Evaluation of ‘Matrix of Change’ intervention

The Matrix of Change seeks to improve implementation of Government programs on nutrition, adolescent health and child protection in 4 states – through a consortium of organizations to enable public awareness, demand generation and engagement with Governments for programmatic change and decision making. 

Development Solutions and Sociometrik evaluated the matrix of change pilot intervention, to: 

• Assess impact of the intervention 
• Understand contribution of the intervention to larger change in implementation of the Government programs 
• Assess the effectiveness of the consortium model 

The evaluation sought to establish ‘proof of concept’. 

Approach and Methodology

Development Solutions worked in Angul, Odisha; Sitapur, UP; Jaipur, Rajasthan; and Gaya, Bihar 

A mixed-method, pre- and post-evaluation design, comprising quantitative and qualitative data collection and location analytics was used. Details of the methodology are as follows:

• Theory of Change mapped to validate assumptions and create a framework for contribution analysis
• Quantitative impact survey
• Location intelligence to Identify change in pockets of vulnerable populations 
• Catchment style geospatial analysis – to assess differential impact, given service access points 
• Model for impact extrapolation 
• In-depth qualitative interactions – to assess contribution and understand stakeholder perspectives 

Quantitative survey was done through CAPI. The data collections was conducted with the following respondents: 

• 1200 women – pregnant and with children less than 11 months of age, each at baseline and end line 
• 10 adolescent friendly health clinics, over a period of 9 months 
• 200 vulnerable households in Gaya, Bihar; interactions in 4 shelter homes in Jaipur

In addition, qualitative interactions were undertaken with key program beneficiaries (pregnant women and mothers of children less than 11 months of age for nutrition; adolescents; and vulnerable families); health and nutrition service providers and Government officials at districts and state levels. Further, in-depth interviews were conducted with partner organizations to assess effectiveness of the consortium function.

Results

Data and information from impact estimation (OLS regression estimates), and from location intelligence, geospatial analysis and secondary data were triangulated to enable the assessment. Insights from qualitative interactions, enabled an understanding of intervention contribution and consortium effectiveness. The evaluation enabled impact estimates and provided insights and recommendations for scale up.