Improving Quality: Current Evidence on What Affects the Quality of Commissioned Evaluations

Abstract

With the increase in resources that organisations are dedicating to evaluation the issue of evaluation quality has risen up the agenda and a growing number of commissioners are now looking at how to ensure the studies they commission are of sufficient quality. While a plethora of evaluation quality standards exist that identify the factors that shape quality, most are experiential rather than based on research evidence. Particularly in the context of commissioning and implementing evaluation in bilateral donors, there has been limited empirical research on identifying the factors that underlie evaluation quality. Drawing on the findings of two recent studies into the quality of evaluations and other recent work in this area, this CDI Practice Paper by Rob Lloyd and Florian Schatz starts to fill this gap in evidence. The paper argues that the current debate on evaluation quality has become fixated on the issue of methodology to the neglect of other equally important issues. While methodological rigour is important, a singular focus on this issue is unwise. Considerations of quality need to permeate all stages of the evaluation process and evaluation quality needs to be recognised as a product of the capacities of the evaluation commissioner and evaluation team, the relationship between them, and the wider institutional environment in which the evaluation is being conducted.