In development policy jargon, a “program evaluation” is a systematic and objective assessment of an ongoing or completed project or policy, including its design, implementation, and results. Very few aid agencies have incorporated corruption considerations into their standard program evaluations–despite the fact that these same agencies have focused heavily on corruption measurement (as a separate endeavor) since the mid-1990s. This is a mistake: A good evaluation should include consideration of corruption, given that program success can be threatened by waste, leakage, and outright theft of resources, and also that evaluations can be useful tools for corruption risk management, accountability, and learning about how to build better anticorruption mechanisms. Part of the explanation for the failure to integrate corruption considerations into program evaluation may be the difficulty of rigorously measuring corruption levels and impact, yet as I have argued elsewhere, this difficulty should not be used as an excuse for not evaluating anticorruption efforts systematically.
Aid organizations can and should incorporate corruption issues into their standard evaluation policies. But not all program evaluators are anticorruption experts, and so the anticorruption community needs to provide more guidance on how to integrate anticorruption analysis into program evaluation. This is one of the things we are trying to do at U4. Based on our work, and helpful discussions with other experts, here are some suggestions for how this can be achieved:
First, corruption concerns should be explicitly integrated into the terms of reference for program evaluators, in order to encourage the evaluators to explicitly analyze corruption risks and costs. The four questions below are all relevant and can be adapted for different programs:
- What proportion of resources are lost directly to corruption within programs?
- How much is lost to waste and leakage?
- What is the impact of corruption and leakage on attainment of your overall development objective?
- How cost-effective would anticorruption measures be, or how cost-effective have they been?
Given the right designs and adequate resources, these are all answerable questions. Obtaining the information will not only help organizations learn how to reduce corruption costs, but will also provide valuable information to the public and the politicians who support development work. Of course, not all evaluations can answer the above four questions at the same time. Questions three and four in particular need good planning and resources. Nonetheless, organizations need a policy for how to respond to these questions as part of the overall evaluation portfolio.
Second, in addition to revising the terms of reference to include anticorruption considerations, evaluators need guidance on how corruption ought to be considered in the context of a program evaluation. A good starting point is to situate anticorruption considerations in the context of the familiar OECD evaluation criteria: (1) relevance, (2) effectiveness, (3) efficiency, (4) impact, and (5) sustainability. Each of these five standard evaluation criteria can easily be connected to corruption, and doing so will help program evaluators integrate corruption considerations into a familiar framework they already employ. Those connections can be made as follows:
- Relevance has to do with assessing whether interventions pursue the right objectives and activities. Since anticorruption is meant to be mainstreamed in most development cooperation, it would be relevant for many programs to consider corruption issues.
- Effectiveness is a measure of the extent to which an activity reaches its objectives. Here, it is straightforward to show that corruption can often be a major factor in the achievement or non-achievement of objectives.
- Efficiency includes both qualitative and quantitative measures of outputs in relation to inputs (i.e., whether a project could have achieved the same or better results with the same or fewer resources). “Leakage,” or other sorts of resources lost due to corruption, obviously reduces program efficiency, and effective anticorruption measures may therefore result in a net gain of resources, and improvement in overall cost-effectiveness, as I have discussed in more depth elsewhere.
- Impact captures the positive and negative changes produced by an intervention, including both intended and unintended results, taking into account the impact of external factors. Although the “external factors” considered are usually things like terms of trade and financial conditions, a change in corruption levels in a country or sector is also a relevant external factor (or perhaps even an unintended consequence of the program) that should be captured in the evaluation.
- Sustainability involves assessing whether the benefits of an activity are likely to continue after the program ends. If you don’t design and implement effective corruption risk management systems it can undermine sustainability as funds may be diverted from intended use.
Third, we need to recognize that one reason that evaluators may be discouraged from digging too deep into corruption issues is that they are told that writing about corruption is simply too political. Aid agencies should squarely address this concern by promulgating clearer guidelines that make it explicit that corruption must be considered as part of the evaluation process, and that it is not “too political” to take this into account.