Evaluation Matters
WHO Evaluation News: Issue 12, August 2024
Cultivating an inclusive evaluation culture
In this edition, we delve into the intersection of evaluation and global health impact, emphasizing efforts to foster a participatory evaluation culture. Our focus spans recent evaluations examining WHO’s performance in areas ranging from country-level contributions to a comprehensive assessment of a global programme to deliver people-centred primary health care. In June, the WHO Executive Board requested a revision of the evaluation policy based on a recent comparative study of WHO's evaluation function and those of similarly sized and structured UN agencies. This decision supports an ongoing commitment to strengthen the organization's evaluation capacity.
In the Spotlight
MOPAN stresses evaluation's key role and recommends investment
The latest Multilateral Organization Performance Assessment Network (MOPAN) report about WHO’s performance and capabilities (2019-2023) underscored the critical role of evaluation in demonstrating contributions to global health outcomes and results. The evaluation of WHO's 13th General Programme of Work (GPW13) in December 2023 was cited as a pivotal opportunity to substantiate the organization's achievements and effectiveness. MOPAN recommended WHO invests further in evaluation to meet UN norms and adopts a more strategic and comprehensive approach to strengthen corporate learning.
Read the report here.
Comparative study of WHO evaluation informs way forward
Following a recommendation from the Executive Board, the Evaluation Office conducted a comparative study of other evaluation functions and their coverage across UN entities comparable to WHO in size and structure. Recommendations focused on strengthening independence, credibility and use.
Key findings underscored the need for WHO to align evaluation policies more closely with strategic goals. Resource allocation remains a significant hurdle, with the evaluation budget representing a mere fraction of the organization's overall funding.
A key recommendation, agreed upon by the Executive Board, is to revise the evaluation policy. Other recommendations include bolstering financial and human resources, and strengthening the dissemination of evaluation results.
Read the study here.
Overview of recent evaluations
Reports and management responses are published on our website here.
Preliminary evaluation of WHO Special Programme on Primary Health Care
WHO established the Special Programme on Primary Health Care in 2020, following the Global Conference on Primary Health Care and the UN high-level meeting on universal health coverage, where countries declared their commitment to achieving universal health coverage through a primary health care approach.
The evaluation highlighted the programme's effectiveness in advocating for primary health care globally but noted a need for greater technical support and wider dissemination of resources at the country level. Structural limitations within WHO affected direct country support. The programme's commitment to equity, gender equality and human rights was evident, though gender considerations require further attention, the evaluation found.
Case studies in Chile, Kenya and Tajikistan revealed both successes and challenges, such as resource allocation and sustainability. The evaluation recommended a clearer strategy, better alignment with WHO departments and partners, and a focus on sustainable, multisectoral actions and community empowerment.
Read the full report here.
Evaluating WHO's Contributions at the Country Level
In 2023, WHO advanced its efforts to evaluate its impact at the country level. Jointly commissioned with regional offices, these evaluations aim to review WHO’s contributions within countries, considering national priorities and needs, as well as the roles of partners, to promote the national public health agenda and improve population well-being. The Evaluation Office, in collaboration with regional offices, aims to perform at least one evaluation annually for each WHO region, focusing on WHO's contributions at the country level.
In the Eastern Mediterranean Region, evaluations were conducted in Djibouti, Iraq and Tunisia. The lessons from these evaluations will be shared with other regional offices. The work plan for 2024-2025 now includes similar evaluations for countries in three additional regions. The Evaluation Office is assessing the demand for supporting country-led, sector-wide evaluations, as advocated by the UN General Assembly in resolution 77/283 (2023) with a focus on SDG3.
Overall, the evaluations found that WHO's work aligns with and contributes to national health needs and policies but lacks a long-term vision developed with key stakeholders. The organization is well-regarded for setting norms and standards, particularly in emergencies, and has effectively reached vulnerable groups. However, responsible disengagement post-emergency remains crucial.
The COVID-19 response demonstrated WHO's ability to provide real-time information, coordinate authorities and establish emergency preparedness. Monitoring and data issues hinder the assessment of contributions, and a greater focus on gender equality and human rights and social determinants is needed to provide further support to vulnerable populations. A multisectoral approach is essential for effective support and collaboration beyond health ministries.
Read the Djibouti report here
Read the Iraq report here
The evaluation report for WHO's contribution in Tunisia will be shared in the next issue.
Ongoing evaluations
An overview of evaluations in progress.
Evaluation of Global Health Days (2019-2023)
The evaluation seeks to assess the effectiveness and impact of WHO's Global Health Days, ensuring alignment with global health priorities and improving campaign execution. It will provide recommendations to enhance future campaigns.
Mid-term evaluation of the Global Task Force on Cholera Control (2017-2030)
The evaluation seeks to assess the relevance, effectiveness and sustainability of the Global Task Force on Cholera Control and its roadmap strategy midway through implementation. It aims to inform future strategic decisions and operational adjustments considering evolving global and local cholera dynamics.
Joint evaluation of the Global Action Plan for Healthy Lives and Well-being for All (SDG3 GAP)
The SDG3 Global Action Plan (GAP), involving 13 multilateral agencies, aims to accelerate progress on health-related SDG targets. This joint evaluation will assess the effectiveness of this collaboration at global, regional and country levels, focusing on alignment with national priorities, coordination, and shared accountability. The findings will guide future efforts to strengthen partnerships and expedite progress towards achieving health-related SDG targets by 2030.
Evaluation of Member State Mechanism on Substandard and Falsified Medical Products
Created in 2012, the Member State Mechanism on Substandard and Falsified Medical Products has become the global forum at which Member States convene, coordinate, decide and organize activities to address substandard and falsified medical products. The evaluation will assess the effectiveness of the Mechanism in strengthening national and regional regulatory authorities, enhancing surveillance and monitoring systems, conducting training and developing risk communication campaigns.
Evaluation of WHO’s Contribution to WASH and Health
The evaluation of WHO’s Global WASH Strategy 2018-2025 will assess progress, achievements, challenges and opportunities. It will recommend adaptations for the 2025 strategy revision. Case studies have been conducted in Ethiopia and the Philippines.
Evaluation of the WHO Policy on Disability
The evaluation aims to provide WHO staff with insights into the effectiveness of disability inclusion initiatives, ensuring alignment with the UN Disability Inclusion Strategy and the WHO Policy on Disability. It seeks to enhance staff awareness, assess the relevance and implementation of activities, identify challenges, share best practices, evaluate resource allocation and ensure the sustainability of disability inclusion efforts.
Regional and Country Focus
News and updates from WHO regional and country offices.
Enhancing PAHO’s Transparency and Accountability through Evaluation
The evaluation landscape for WHO interventions in the Americas has undergone a transformative shift since the implementation of the Pan American Health Organization (PAHO) new evaluation policy. Since June 2023, PAHO began a systematic practice of publicly releasing all external corporate evaluations at the regional level, marking a pivotal moment in its commitment to transparency and accountability.
Watch the video about the organization’s evaluation progress here.
PAHO has launched a dedicated portal and repository of past evaluations since 2009. The platform provides easy access to evaluation reports and a progress tracking tool for recommendations to empower Member States, donors, partners and the public with comprehensive information about performance and results. This initiative aligns closely with the PAHO forward agenda, which aims to bolster organizational effectiveness through rigorous evaluation practices.
In the past year, PAHO has conducted or initiated five major corporate evaluations. These evaluations have engaged virtually all regional, subregional and national entities, ensuring that lessons learned from previous assessments are actively integrated into future strategies. Recommendations stemming from these evaluations are being actively implemented across the organization.
Recent evaluations include comprehensive assessments of critical areas such as PAHO's response to the COVID-19 pandemic, technical cooperation in noncommunicable diseases prevention and control across the Americas, and the implementation of PAHO's results-based management framework. Ongoing evaluations, such as “The Integration of Gender Equality in PAHO's Technical Cooperation in Health,” underscore its commitment to continuous improvement, with results expected by the end of 2024.
Field case studies conducted in 15 countries, coupled with extensive interviews and surveys involving over one thousand PAHO staff and stakeholders, have yielded invaluable insights into the successes and challenges of technical cooperation efforts on the ground. These findings serve as crucial learning opportunities, not only for PAHO and WHO, but also for the broader public health community at national, regional and global levels.
For evaluations and related resources, visit PAHO Evaluation here.
Country-level evaluations: Lessons Learnt in Djibouti, Iraq and Tunisia
WHO Eastern Mediterranean Regional Office (EMRO) reflects on the importance of country-led evaluations.
Why is an evaluation culture important? Fostering a culture of evaluation in WHO Country Offices is crucial for continuous assessment, improvement and alignment with national health goals. Evaluations help ensure that WHO's contributions are relevant, strategic, effective and focused on equity, which enhances their overall impact.
What prompted the recent country-level evaluations in Djibouti, Iraq and Tunisia? In 2023, the Eastern Mediterranean Regional Office (EMRO) and the Evaluation Office initiated strategic country-level evaluations to assess WHO contributions in these countries. The evaluations were aligned with the evaluation of WHO’s 13th General Programme of Work (GPW13), providing valuable insights into subsequent country cooperation strategies.
What are the key recommendations? The key recommendations are as follows:
- Formalizing long-term strategies and cooperation frameworks with national stakeholders.
- Simplifying and integrating monitoring and reporting systems for better effectiveness assessment.
- Enhancing focus on and incorporation of gender equality, health equity and social determinants in all interventions.
- Strengthening WHO’s leadership, strategic support roles and coordination capacity at the country level.
- Institutionalizing emergency preparedness and response mechanisms based on COVID-19 response experiences.
- Strengthening WHO's support role to governments, especially in health systems support.
How have the recommendations from these evaluations and related evaluations been used? Evaluative evidence is being used to inform the design of new country cooperation strategies between WHO and the respective governments, alongside implementing actions to other key recommendations such as the WHO 14th General Programme of Work (GPW14) strategic planning and the formulation of a new strategic direction. The findings and lessons are being incorporated into the EMRO Regional Strategic Operational Plan for 2025-2028, which aims to operationalize GPW14 in the region.
Recommendations from the GPW13 evaluation and related evaluations such as the Results Based Management Evaluation have been utilized by the EMRO Programme Monitoring and Evaluation team to enhance programme management in the region. Regional workshops based on these recommendations have already benefited multiple countries, with additional workshops planned to disseminate and implement these insights further.
For the evaluations, see the reports for Djibouti here and Iraq here .
Ongoing Regional Evaluations
Evaluation of functional reviews of WHO country offices in the African Region
As the functional review of 47 country offices concludes in the African region, this evaluation will inform discussions on strengthening WHO's country presence, improving efficiency and meeting the WHO Transformation Agenda and African Regional Office objectives.
Events
Staff share and exchange expertise at global conferences and other events.
gLOCAL Evaluation Week
3-7 June 2024: This year’s gLOCAL Evaluation Week, organized by the Global Evaluation Initiative, centred on the theme "Evaluation for Transformational Change: Balancing Ambition and Realism". The Evaluation Office participated in sessions on topics such as country-led and joint evaluations and strengthening national evaluation capacities.
OECD DAC EvalNet
18-19 June 2024: The Evaluation Office participated in the OECD DAC EvalNet meeting in Paris, France, which convenes a network of evaluation bodies from bilateral development aid agencies known for setting standards. Discussions focused on policy insights, including lessons from the Agence Française de Développement's evaluation policy revision.
European Evaluation Society
23-27 September 2024: The 15th European Evaluation Biennial Conference in Rimini, Italy, organized by the European Evaluation Society, will explore how collaborative approaches enhance the role of evaluation in tackling urgent challenges like environmental degradation, social injustice and economic instability.
