Evaluation Matters
Evaluation Office Newsletter: Issue 11, March 2024
2024: Evaluation for Global Health Impact
Our mission is to champion evaluation as a core component of results-based management, with a strong emphasis on fostering collaboration. We are embarking on an innovative approach to engage diverse audiences and maximize the impact of evaluation on the achievement of results to improve global health outcomes.
This year promises a series of pivotal evaluations to assess our work in critical domains such as cholera, disability inclusion, WASH, WHO contributions at the country level and a joint evaluation involving 14 other agencies dedicated to assessing the Global Action Plan for Healthy Lives and Well-being for All (SDG3 GAP)
In the Spotlight
Here is a summary of all the recently completed evaluations, including key findings. Reports and management responses are published on our website here.
Reflecting on the 13th Global Programme of Work (GPW 13)
The WHO Thirteenth General Programme of Work (GPW 13) has played a pivotal role in shaping global health initiatives. It had a dual purpose: to outline WHO strategies for making a tangible impact at the country level; and to provide a framework for measuring progress toward the health-related Sustainable Development Goals (SDGs).
An independent evaluation of GPW 13, which was presented at the Executive Board Meeting in January, offers insights into the achievements and challenges of its implementation and sets the stage for the development of GPW 14.
The evaluation highlighted the need for greater clarity regarding the positioning and duration of GPW 13. Achieving targets proved challenging, with disparities in progress towards universal health coverage (UHC) and financial hardships impacting vulnerable populations. Weaknesses in health systems were exposed during health emergencies, affecting immunization and surveillance.
The recommendations included bridging data gaps, addressing immediate COVID-19 impacts, refining the results framework backed up by robust theories of change, enhancing data collection, scaling up result-oriented delivery, and aligning financing more effectively with strategic priorities. The insights gained from this evaluation offer a roadmap to strengthen the organization's impact on global health for the development of GPW 14.
For the full evaluation report, please visit here.
WHO Normative Function at the Country Level
In a dedicated effort to strengthen its core normative function, WHO conducted a comprehensive evaluation on how its normative products have been applied and used at the country level and have made a difference in national health systems and practices. The evaluation examined the use and effectiveness of six essential normative products: 22nd WHO Model List of Essential Medicines (EML), 2021; Guidance for Conducting a Country COVID-19 Intra-action Review (IAR), 2021; HEARTS, Technical Package for Cardiovascular Disease Management in Primary Health Care (PHC), 2020; Mental Health Global Action Programme (mhGAP) Intervention Guide, 2016; Guidelines for the Treatment of Malaria, 2015; and WHO Guidelines for Indoor Air Quality: Household Fuel Combustion, 2014. These products were assessed through in-depth case studies in seven countries.
Findings reveal variations in normative product development at WHO headquarters, challenges in integrating products into national planning and the significant contributions of these products to global health goals. Recommendations include aligning product development with Member States' needs, involving regional and country offices in development processes, emphasizing practical implementation and establishing a robust monitoring framework to consider gender equality and health equity.
For the full evaluation report, please visit here.
Evaluation of the WHO Special Programme on Primary Health Care
In January 2020, WHO established the Special Programme on Primary Health Care (SP-PHC) with the objective of enhancing the integration of the Primary Health Care (PHC) approach within the organization. The SP-PHC was introduced as a response to global challenges in PHC aligned with WHO goals. The evaluation assessed the effectiveness of SP-PHC in supporting WHO PHC objectives and provides recommendations for its future development in advancing UHC.
The evaluation encompassed all levels of WHO, including the global, regional and country levels, as well as initiatives led by external partners. Recommendations include prioritizing PHC outcomes in GPW 14 to enhance collaboration and budget allocations. The evaluation also suggests creating a new entity for global PHC promotion, revisiting the design of existing structures like SP-PHC and strengthening support mechanisms for scaling up PHC approaches. These measures align with the WHO goal of reorienting health systems towards PHC to advance UHC.
Mid-term Evaluation of the Global Strategy to Eliminate Yellow Fever Epidemics (EYE) 2017-2026
In response to the 2016 yellow fever outbreak in Angola, Gavi, UNICEF, and WHO collaboratively devised the Eliminate Yellow Fever Epidemics (EYE) strategy (2017-2026). This initiative aimed to protect vulnerable populations, curb international disease transmission and rapidly respond to outbreaks. A mid-term evaluation conducted by the Evaluation Office and partner organizations unveiled significant achievements, including the vaccination of 185 million people in high-risk African countries, improved vaccine supplies and streamlined disease confirmation processes. Brazil demonstrated swift responses to yellow fever outbreaks and successful approaches were noted in Ghana. The challenges identified by the evaluation included insufficient implementation capacity, overall low prioritization of yellow fever and sustainability concerns, with heavy reliance on external support.
The evaluation recommendations highlighted the need to address resource constraints through collaborative resource mobilization, relaunching the EYE strategy with high-level events for renewed political commitment, expanding governance structures and conducting subnational risk assessments. Enhanced integration with related health programmes, strengthened supply chains, and improved surveillance and coordination are also essential components for future success.
2024 Evaluations
Take a look at the evaluations coming up in 2024. Reports and management responses are published on our website here.
Evaluation of the Member State Mechanism on Substandard and Falsified Medical Products
In 2012, the 65th World Health Assembly (WHA) established the Member State Mechanism (MSM) to combat substandard and falsified (SF) medical products, aiming to foster collaboration among Member States and the WHO Secretariat to prevent and control these products, ensuring public health protection and access to safe, affordable, efficacious and quality medical products. It has become a global forum where Member States collaborate on various objectives, including enhancing regulatory oversight and sharing best practices.
Low- and middle-income countries spend an estimated US$30.5 billion on such medicines, representing 10.5% of their supply chains.
Following the recommendation of MSM, the 76th WHA requested the WHO Director-General to facilitate an independent evaluation of the MSM. The evaluation will document progress, identify achievements, challenges and gaps, assess the effectiveness of its tools against other approaches, and review its format and governance structures to ensure relevance and fitness for purpose.
Evaluation of the WHO Policy on Disability
The purpose of the evaluation is to assess the effective implementation of disability inclusion across all levels of WHO. This evaluation is expected to play a vital role in promoting institutional learning and accountability, aligning with the UN Convention on the Rights of Persons with Disabilities (CRPD) and the pursuit of the SDGs, particularly the commitment to leave no one behind. The findings and insights derived from this evaluation will contribute to the annual report to the UN Secretariat and the WHO Executive Board, providing valuable input for Member States and the organization's leadership. Additionally, it will guide the UN Disability Inclusion Strategy (UNDIS) custodian departments, regional and country offices, offering recommendations to advance their work on disability inclusion.
Evaluation of Functional Reviews of WHO African Region Country Offices
This evaluation is currently progressing and is expected to be completed by the second quarter of 2024. The evaluation aims to identify the key achievements of functional reviews in the 47 countries of the region, highlighting good practices, performance and results achieved, experiences of existing and former staff in the implementation of the functional reviews and improvements in WHO governance across the region. It will also identify lessons learned and core areas of work that need to be initiated or accelerated as a result of the analysis of the recommendations, to achieve the results set out by the functional review and contribute to the WHO Regional Office for Africa's Transformation Agenda.
WHO Contributions at the Country Level
In collaboration with the WHO Regional Office for the Eastern Mediterranean (EMRO) and WHO country offices, the Evaluation Office has conducted joint evaluations of the organization's contributions at the country level in Djibouti, Iraq and Tunisia. The evaluations aim to provide accountability to the countries and offer the basis to direct future county cooperation strategies. The reports for these three evaluations are expected to be finalised in the first quarter of 2024. New evaluations will be launched in several more countries in EMRO and other regions in 2024.
Priorities and Strategic Direction
Take a look at an overview of the Evaluation Office workplan (2024 – 2025).
Our new organization-wide workplan encompasses a range of essential activities aimed at advancing evaluation efforts. These initiatives include conducting comprehensive thematic evaluations across various topics of significance to both the organization and its Member States. We will provide support for major programme evaluations, collaborating closely with technical departments and regional offices to ensure their success.
We are dedicated to promoting and facilitating evaluations of WHO contributions at the country level. At an Executive Board Meeting session in 2024, Member States requested an increase in the number of country evaluations. These evaluations will be undertaken jointly, led by the respective regional offices in conjunction with the Evaluation Office. We will launch three country evaluations assessing WHO contributions in Djibouti, Iraq and Tunisia.
We will actively engage in the development and support of evaluation planning and implementation by WHO country offices, working collaboratively with technical departments and regional offices to enhance evaluation practices.
To strengthen joint action and enhance results delivery, we will continue to participate in joint evaluations covering broader themes, including those aligned with the SDG3 Global Action Plan (SDG3 GAP) and the Inter-Agency Humanitarian Evaluation Framework.
To facilitate these endeavours, we will review our evaluation policy and consider necessary enhancements. This may involve revisiting aspects such as coverage criteria, funding principles and the role and responsibilities of programme managers during the independent evaluation process.
Regional and Country Focus
Evaluation news and updates from WHO regional and country offices.
WHO Regional Office for Africa on a journey to strengthen evaluation
The WHO Regional Office for Africa (AFRO) is taking steps to reinforce its evaluation function. This strategic move aligns closely with the unwavering organizational commitment to achieving results. To cultivate a stronger evaluation culture, AFRO is bolstering foundations, including leveraging the AFRO evaluation dashboard to facilitate cross-country collaboration, allocating resources efficiently and pinpointing evaluation needs. The Evaluation Office will initiate joint evaluations of WHO contributions in at least three countries to showcase the benefits of such evaluations to other country offices.
Events
Evaluation staff share and exchange expertise globally. Here are the latest events.
Asian Evaluation Week 2023
11 – 14 September 2023: Evaluation staff participated in Asian Evaluation Week, specifically in a panel discussion titled: Learning in action: Expanding the evaluation toolbox?
The discussion focused on the challenges that COVID-19 and the identification of recovery measures presented for traditional forms of independent post-evaluation. Evidence-based decision-making in an evolving context called for rapid assessments and actionable, timely and succinct findings and insights. Participants reflected on their experience as evaluators in developing and using a decision-maker-oriented evaluative learning approach and shared strengths, challenges and potential synergies inherent in early and “fast-lane” assessments. They also explored how this approach can work beyond crisis response as a complementary tool to traditional post-evaluation.
To watch the recording of the panel discussion, visit here.
United Nations Evaluation Group Annual Meeting: EvalWeek 2024
29 January – 2 February 2024: This year’s UN Evaluation Group (UNEG) EvalWeek 2024 took place in Malaga, Spain. It featured a rich line-up of seminars including: 'Evaluation in Crisis,' 'System-Wide Evaluations,' 'Inter-Agency Humanitarian Evaluations,' ‘Evaluating Policy Influence’, and the UNEG Annual General Meeting (AGM). At the AGM, as co-chairs of the National Evaluation Capacity Development Working Group, WFP and WHO presented the following studies: Comparative analysis of UN evaluation policies with respect to national evaluation capacities development; Lessons learnt from joint evaluations and support to country-led evaluations.
The Evaluation Office also presented the 2024 Interagency Humanitarian Evaluation: Process Guidelines, crafted by the Working Group chaired by WHO and composed of representatives from IFRC, ICVA, OCHA, OHCHR, UNICEF, and UNHCR.
For more information about the event, please visit here.
