UK frameworks for Malaria and Reproductive, Maternal and Newborn Health: Assessing evaluability
This project has provided important evidence around DFID’s approaches to key health interventions. As a part of the Global Evaluations Framework Agreement (GEFA), DFID contracted the e-Pact consortium led by OPM and ITAD to carry out an evaluability assessment and mid-term review (MTR) of its Frameworks for Results on malaria and Reproductive, Maternal and Newborn Health (RMNH). The team reviewed the theory of change and developed the evaluation matrix for a midterm review and final evaluation of the Frameworks. Mixed-methods data collection and analysis techniques were adopted including document reviews, key informant interviews (KIIs), quantitative data analysis (of data on global outcomes and aid activities and from DFID monitoring and reporting systems), and an online survey of DFID country programmes. Three countries: Ethiopia, Nigeria and India were selected for in-depth case studies. The findings of the midterm review have helped inform DFID policy-making and supported the revision of management processes related to the Frameworks.
Published in 2010, the Frameworks for Results (FfR) set out how DFID plans to deliver on its commitment to halve malaria deaths in high burden countries and save the lives of at least 50,000 women during pregnancy and childbirth and 250,000 newborn babies across the world by 2015. The Frameworks reflect a shift in UK development policy towards a greater focus on results, seeking to direct DFID resources towards interventions that provide strong international evidence of their effectiveness.
This project addressed the need for greater information on the effectiveness of the approach itself and the contribution of each Framework towards achieving UK government objectives.
Working as part of a consortium, we conducted a mid-term review and evaluability assessment of the Frameworks. The team reviewed and developed theories of change, highlighting indicators for programme outputs, outcomes and impacts and used these as a basis for the assessments.
Work focused around a number of analytical steps including setting out the global context for each Framework by identifying trends in international actions, resources and key health indicators and gathering and analysing information on those activities already implemented by DFID under each Framework.
In addition, the team conducted in-depth analysis of selected bilatreral and multilateral programmes and developed detailed case studies for three countries: Ethiopia, India and Nigeria. The case studies sought to review evidence along the whole results chain, examining DFID’s activities and the influence of the Frameworks within national and international contexts.
Specific activities undertaken by the team included:
- Reviewing and developing Theories of Change (ToC) for each Framework
- Developing evaluation matrices for midterm reviews and final evaluations
- Conducting documentation reviews and key informant interviews
- Completing quantitative data analysis from a number of sources including an online survey of DFID country programmes
- Developing country case studies for Ethiopia, India and Nigeria
- Producing a midterm review report incorporating strategic recommendations for each Framework
This project has helped improve DFID’s understanding of its two Frameworks for Results – highlighting the outcomes achieved by each to date as well as the measurability of their impacts. In turn, this evidence is supporting the strengthening of the frameworks and the improvement of management processes around each one. Over the longer-term, the implementation of OPM’s recommendations should support more efficient and effective delivery of programmes that help combat malaria and lead to improvements in RMNH.