Going Beyond Mixed Methods to Mixed Approaches: A Systems Perspective for Asking the Right Questions

Abstract

An impact evaluation's primary task is to determine which impacts were caused by an intervention, distinguishing them from those produced by other causes. However, in complex systems, interventions may contribute towards less apparent forms of impact (such as negative, unintended, indirect and secondary) that are no less significant, but which require a different way of asking questions. Rather than advocating for a particular evaluation approach or methodology, we propose that (1) beyond the intervention's theory of change (TOC), the complex system within which the intervention has taken place and intends to make an impact must also be examined, and (2) specific elements of an intervention's TOC and the larger system must be understood to have a systematic basis for determining what evaluation questions need to be asked. Only then can the appropriate evaluation approaches and methods be identified, and adapted as this understanding increases over the course of the evaluation.