The Institute of Development Studies and Partner Organisations
Browse

Evaluation of Programs with Multiple Objectives: Multidimensional Methods and Empirical Application to Progresa in Mexico

report
posted on 2024-10-04, 13:46 authored by Ana Vaz, Bilal Malaeb, Natalie Naïri Quinn
Development programs and policy interventions frequently have multiple simultaneous objectives. Existing quantitative evaluation approaches fail to fully accommodate this multiplicity of objectives. In this paper we adapt the multidimensional poverty measurement approach developed by Alkire and Foster (2011) to the estimation of treatment effects for programs with multiple objectives. We use the potential outcomes framework to show that differences in Alkire-Foster indices between treated and control samples correspond to average treatment effects estimates of outcomes of interest under experimental conditions, and develop further methods of analysis to explore these multidimensional treatment effects. We discuss issues of index design encountered in practice and provide an illustrative example. We apply the methods developed to evaluate the conditional cash transfer program Progresa in Mexico, finding significant multidimensional effects of the program. Further analysis shows that these treatment effects are driven mainly by impacts on school attendance and health visits, objectives that correspond directly to the conditions of the program. There is no evidence for heterogeneity of the treatment effects by the extent to which the beneficiary failed to achieve the objectives at baseline. This study complements the extensive literature on the evaluation of Progresa and other development programs, comprising studies that focus on particular objectives or outcomes of the program. We hope that the methods developed here will find wide application to the evaluation of programs with multiple objectives.||Development programs and policy interventions frequently have multiple simultaneous objectives. Existing quantitative evaluation approaches fail to fully accommodate this multiplicity of objectives. In this paper we adapt the multidimensional poverty measurement approach developed by Alkire and Foster (2011) to the estimation of treatment effects for programs with multiple objectives. We use the potential outcomes framework to show that differences in Alkire-Foster indices between treated and control samples correspond to average treatment effects estimates of outcomes of interest under experimental conditions, and develop further methods of analysis to explore these multidimensional treatment effects. We discuss issues of index design encountered in practice and provide an illustrative example. We apply the methods developed to evaluate the conditional cash transfer program Progresa in Mexico, finding significant multidimensional effects of the program. Further analysis shows that these treatment effects are driven mainly by impacts on school attendance and health visits, objectives that correspond directly to the conditions of the program. There is no evidence for heterogeneity of the treatment effects by the extent to which the beneficiary failed to achieve the objectives at baseline. This study complements the extensive literature on the evaluation of Progresa and other development programs, comprising studies that focus on particular objectives or outcomes of the program. We hope that the methods developed here will find wide application to the evaluation of programs with multiple objectives.

History

Publisher

University of Oxford

Citation

Vaz, A., Malaeb, B. and Quinn, N.N. (2019). ‘Evaluation of programs with multiple objectives: Multidimensional methods and empirical application to Progresa in Mexico’, OPHI Research in Progress 55a, University of Oxford

IDS Item Types

Series paper (non-IDS)

Copyright holder

Oxford Poverty & Human Development Initiative

Identifier Ag

ES/N01457X/1

Usage metrics

    Impact Initiative - Governance

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC