Are Government M&E Systems Effectively Institutionalized and Coordinated?

The development community is increasingly interested in institutionalizing monitoring and evaluation (M&E) in aid recipient governments as one strategy for improving aid effectiveness. Institutional structures within government M&E systems that improve coordination, central oversight, and feedback from different stakeholders can support more effective decision-making.

This second in a series of four posts presenting evidence and trends from the Evans School Policy Analysis and Research (EPAR) Group’s report on aid-recipient government M&E systems focuses on findings on institutionalization and coordination of these systems. EPAR reviewed evidence from 42 government M&E systems, including 17 general national M&E systems that aim to monitor and evaluate progress across multiple sectors and 25 systems that are national in scope but sector-specific. The first post in the series provides an introduction to aid-recipient country M&E systems and outlines the theoretical framework, data, and methods for the research. This post examines how these government M&E systems are organized, summarizing findings on internal demand for M&E, clarity of institutional M&E roles, and harmonization of M&E activities (Figure 1).

Figure 1: Institutionalization and Coordination of Government M&E Systems

Internal Demand for M&E

A review of 20 Sub-Saharan aid-recipient countries (Holvoet, et. al 2012) suggests a strong external demand from donors for M&E information. While external demand can support the development of M&E systems, internal demand from domestic government or civil society is claimed by some to lead to more sustainable government M&E systems. Other authors argue that strong sources of internal demand motivates resources being allocated towards M&E, encourages domestic ownership of M&E activities, and ensures that skills and knowledge remain in the domestic core structure of government.

Among M&E systems reviewed, systems with “high” internal demand (13 of 42 systems) are characterized by a wide range of stakeholders engaged in strategic planning processes, a focus on reviewing results and performance to inform planning, and high-level political commitment.

Elected officials, especially Presidents or Prime Ministers, appear to be key drivers of demand for general national M&E systems. In Kenya, for example, the 2010 Constitution includes accountability requirements promoting M&E. Multi-sectoral agencies can also be key drivers of demand. Demand for HIV/AIDS M&E systems, for instance, is often led by multi-sectoral agencies established to monitor progress in achieving HIV/AIDS outcomes (5 of 8 systems). Finally, stakeholders from civil society, such as academic institutions and local non-profit organizations, can contribute significantly to internal demand for M&E information, as noted in 15 of 42 systems.

Half of the systems reviewed (21 of 42) describe challenges for increasing internal demand for M&E. These challenges include limited internal capacity and dedicated resources for responding to increased demand for M&E activities, as well as difficulty managing the pace and sequencing of M&E reforms to overcome internal resistance. Three systems also report difficulties in incentivizing M&E use due to separate systems for monitoring budgets and outcomes or fears that M&E information will jeopardize funding.

Clarity of Institutional M&E Roles

As observed by Kusek & Rist, clear institutional roles and responsibilities not only contribute to a systematic and accountability-driven M&E system, but are necessary for M&E activities to be fully implemented. EPAR finds that defining and clarifying roles and leadership is a key challenge in many M&E systems. A review of health sector M&E systems in Rwanda and Uganda concludes that this challenge may emerge from overlapping mandates, loose coordination of M&E activities, and unclear boundaries for M&E functions.

In spite of these challenges, nearly half of systems reviewed (19 of 42) rate “high” on the clarity of institutional roles, with government plans assigning responsibility for different M&E roles to specific offices and evidence to suggest that these responsibilities are being fulfilled. On the other hand, nine systems have a “low” rating, indicating limited evidence of how M&E activities are assigned and implemented.

Many systems have one ministry or office responsible for M&E activities, but in some cases the responsibilities are spread across a variety of different offices. Several systems, including the majority of HIV/AIDS M&E systems (8 of 14), confer responsibility to a single multi-sectoral agency drawing representatives from different offices. Of the nine systems with a single multi-sectoral agency responsible for M&E, seven rate “high” for institutional clarity and two rate “medium”, indicating that this approach may help make M&E implementation more effective. For the remaining M&E systems, however, there is no clear association between the types of government offices involved and the clarity of institutional roles.

Harmonization of M&E Activities

Porter and Goldman (2013) write that aid-recipient countries generally have several separate M&E systems, usually including a general national system as well as specific M&E systems for different sectors. This setup can create challenges for efficiency and for coordinating and harmonizing M&E activities. In their 2007 review of results-based national development strategies, the World Bank suggests many systems are challenged by a lack of clarity or mandate for the office in charge of M&E coordination, and others face difficulties in coordinating M&E activities across multiple sectors and levels of government. Some studies also find systems have insufficient capacity for coordinating activities, often due to limited funding for staff and relevant skills training.

EPAR finds that just six M&E systems have clearly coordinated and aligned M&E roles and activities across government offices and levels. Five of these systems are HIV/AIDS systems where multi-sectoral government agencies have clear responsibilities for coordinating M&E activities. The sixth is Mexico’s general national M&E system, where M&E activities are managed by an independent agency, CONEVAL.

Among the remaining 37 systems 14 are given “medium” and 11 “low” ratings for M&E harmonization due to undefined roles and responsibilities, weak coordination, and/or overlap or confusion in implementing M&E activities across poorly aligned systems. The final 11 systems do not have enough evidence to assess the harmonization of their M&E activities.

Improving Institutionalization & Coordination of M&E Systems

Beyond addressing capacity and resource constraints, the systems reviewed describe a variety of methods for addressing institutionalization and coordination challenges. A common approach to increasing internal demand for M&E in the documents reviewed is increasing communication with, and involvement of, stakeholders by making monitoring information and evaluation findings more accessible. Currently, less than a third of systems (12 of 42) have publicly available raw M&E data.

Evidence from some M&E systems also indicates that implementing new M&E systems in phases can both reduce resource constraints and allow government offices time to adapt and become sources of internal demand. Also, assigning a clear mandate for M&E coordination to a specific office appears to help clarify and harmonize M&E roles and responsibilities. An example is Nepal’s HIV/AIDS system operating as a multi-sectoral agency with several staff responsible for M&E coordination, regional officers tasked with coordinating M&E functions, clear plans for building regional M&E capacity, and provisions for coordinating monitoring activities with regional health offices.

Several countries have developed well-institutionalized M&E systems in spite of challenges defining and clarifying roles and leadership, aligning and coordinating M&E activities across sectors and levels of government, and building internal capacity for M&E activities. Institutionalizing roles is just one factor in developing an effective M&E system, however. The next two posts in this series examine two other facets of aid-recipient M&E: systems for data collection, analysis and use, and alignment and harmonization of government and development partner M&E systems.

References to evidence linked in this post are included in the full report. This series is guest-written by the Evans School Policy Analysis and Research (EPAR) Group at the University of Washington in collaboration with Development Gateway and the Results Data Initiative. View the first post in this series here.