Nothing Special   »   [go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/cca/wpaper/292.html
   My bibliography  Save this paper

Assignment procedure biases in randomized policy experiments

Author

Listed:
  • Gani Aldashev
  • Georg Kirchsteiger
  • Alexander Sebald
Abstract
Randomized controlled trials (RCT) have become a dominant empirical tool in applied economics. The internal validity of RCTs crucially depends on the (implicit) assumption that the procedure assigning subjects to treatment and control groups has no effect on behavior. We show theoretically that this assumption is violated when people are motivated by belief-dependent preferences and care about the intentions of others. The choice of assignment procedure influences subjects’ behavior and, consequently, the RCTs’ findings. Strikingly, even a credible and explicit randomization procedure does not guarantee an unbiased prediction of the impact of a general introduction of the policy.

Suggested Citation

  • Gani Aldashev & Georg Kirchsteiger & Alexander Sebald, 2012. "Assignment procedure biases in randomized policy experiments," Carlo Alberto Notebooks 292, Collegio Carlo Alberto.
    • Handle: RePEc:cca:wpaper:292
    as

    Download full text from publisher

    File URL: https://www.carloalberto.org/wp-content/uploads/2018/11/no.292.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    2. Charness, Gary & Dufwenberg, Martin, 2003. "Promises & Partnership," Research Papers in Economics 2003:3, Stockholm University, Department of Economics.
    3. James J. Heckman, 2010. "Building Bridges between Structural and Program Evaluation Approaches to Evaluating Policy," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 356-398, June.
    4. Esther Duflo & William Gale & Jeffrey Liebman & Peter Orszag & Emmanuel Saez, 2006. "Saving Incentives for Low- and Middle-Income Families: Evidence from a Field Experiment with H&R Block," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 121(4), pages 1311-1346.
    5. Abbring, Jaap H. & Heckman, James J., 2007. "Econometric Evaluation of Social Programs, Part III: Distributional Treatment Effects, Dynamic Treatment Effects, Dynamic Discrete Choice, and General Equilibrium Policy Evaluation," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 72, Elsevier.
    6. Michalopoulos, Charles & Robins, Philip K. & Card, David, 2005. "When financial work incentives pay for themselves: evidence from a randomized social experiment for welfare recipients," Journal of Public Economics, Elsevier, vol. 89(1), pages 5-29, January.
    7. Guido W. Imbens, 2010. "Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 399-423, June.
    8. Joshua Angrist & Eric Bettinger & Erik Bloom & Elizabeth King & Michael Kremer, 2002. "Vouchers for Private Schooling in Colombia: Evidence from a Randomized Natural Experiment," American Economic Review, American Economic Association, vol. 92(5), pages 1535-1558, December.
    9. Dufwenberg, Martin & Kirchsteiger, Georg, 2004. "A theory of sequential reciprocity," Games and Economic Behavior, Elsevier, vol. 47(2), pages 268-298, May.
    10. Claudio Ferraz & Frederico Finan, 2008. "Exposing Corrupt Politicians: The Effects of Brazil's Publicly Released Audits on Electoral Outcomes," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 123(2), pages 703-745.
    11. Suresh de Mel & David McKenzie & Christopher Woodruff, 2009. "Returns to Capital in Microenterprises: Evidence from a Field Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 124(1), pages 423-423.
    12. James J. Heckman & Jeffrey A. Smith, 1995. "Assessing the Case for Social Experiments," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 85-110, Spring.
    13. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    14. David Card & Dean R. Hyslop, 2005. "Estimating the Effects of a Time-Limited Earnings Subsidy for Welfare-Leavers," Econometrica, Econometric Society, vol. 73(6), pages 1723-1770, November.
    15. Sebald, Alexander, 2010. "Attribution and reciprocity," Games and Economic Behavior, Elsevier, vol. 68(1), pages 339-352, January.
    16. Sylvain Chassang & Gerard Padro I Miquel & Erik Snowberg, 2012. "Selective Trials: A Principal-Agent Approach to Randomized Controlled Experiments," American Economic Review, American Economic Association, vol. 102(4), pages 1279-1309, June.
    17. Geanakoplos, John & Pearce, David & Stacchetti, Ennio, 1989. "Psychological games and sequential rationality," Games and Economic Behavior, Elsevier, vol. 1(1), pages 60-79, March.
    18. Battigalli, Pierpaolo & Dufwenberg, Martin, 2009. "Dynamic psychological games," Journal of Economic Theory, Elsevier, vol. 144(1), pages 1-35, January.
    19. Paul Schultz, T., 2004. "School subsidies for the poor: evaluating the Mexican Progresa poverty program," Journal of Development Economics, Elsevier, vol. 74(1), pages 199-250, June.
    20. Ruffle, Bradley J., 1999. "Gift giving with emotions," Journal of Economic Behavior & Organization, Elsevier, vol. 39(4), pages 399-420, July.
    21. Gary Charness & Martin Dufwenberg, 2006. "Promises and Partnership," Econometrica, Econometric Society, vol. 74(6), pages 1579-1601, November.
    22. Paul Gertler, 2004. "Do Conditional Cash Transfers Improve Child Health? Evidence from PROGRESA's Control Randomized Experiment," American Economic Review, American Economic Association, vol. 94(2), pages 336-341, May.
    23. Joshua Angrist & Eric Bettinger & Michael Kremer, 2006. "Long-Term Educational Consequences of Secondary School Vouchers: Evidence from Administrative Records in Colombia," American Economic Review, American Economic Association, vol. 96(3), pages 847-862, June.
    24. Rabin, Matthew, 1993. "Incorporating Fairness into Game Theory and Economics," American Economic Review, American Economic Association, vol. 83(5), pages 1281-1302, December.
    25. Zephyr, 2010. "The city," City, Taylor & Francis Journals, vol. 14(1-2), pages 154-155, February.
    26. Pierpaolo Battigalli & Martin Dufwenberg, 2007. "Guilt in Games," American Economic Review, American Economic Association, vol. 97(2), pages 170-176, May.
    27. Anup Malani, 2006. "Identifying Placebo Effects with Data from Clinical Trials," Journal of Political Economy, University of Chicago Press, vol. 114(2), pages 236-256, April.
    28. Card, David & Robins, Philip K., 2005. "How important are "entry effects" in financial incentive programs for welfare recipients? Experimental evidence from the Self-Sufficiency Project," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 113-139.
    29. Gary Burtless, 1995. "The Case for Randomized Field Trials in Economic and Policy Research," Journal of Economic Perspectives, American Economic Association, vol. 9(2), pages 63-84, Spring.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kiessling, Lukas & Radbruch, Jonas & Schaube, Sebastian, 2018. "The Impact of Self-Selection on Performance," IZA Discussion Papers 11365, Institute of Labor Economics (IZA).
    2. Bierbrauer, Felix & Netzer, Nick, 2016. "Mechanism design and intentions," Journal of Economic Theory, Elsevier, vol. 163(C), pages 557-603.
    3. Dufwenberg, Martin & Kirchsteiger, Georg, 2019. "Modelling kindness," Journal of Economic Behavior & Organization, Elsevier, vol. 167(C), pages 228-234.
    4. Jörg Peters & Jörg Langbein & Gareth Roberts, 2018. "Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity," The World Bank Research Observer, World Bank, vol. 33(1), pages 34-64.
    5. Gwenolé Le Velly & Céline Dutilly, 2016. "Evaluating Payments for Environmental Services: Methodological Challenges," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-20, February.
    6. Omar Al-Ubaydli & John List & Claire Mackevicius & Min Sok Lee & Dana Suskind, 2019. "How Can Experiments Play a Greater Role in Public Policy? 12 Proposals from an Economic Model of Scaling," Artefactual Field Experiments 00679, The Field Experiments Website.
    7. Bellemare, Charles & Sebald, Alexander, 2019. "Measuring Belief-Dependent Preferences without Information about Beliefs," IZA Discussion Papers 12153, Institute of Labor Economics (IZA).
    8. Elias Bouacida & Renaud Foucart, 2020. "The acceptability of lotteries in allocation problems," Working Papers 301646245, Lancaster University Management School, Economics Department.
    9. Pierpaolo Battigalli & Martin Dufwenberg, 2019. "Psychological Game Theory," Working Papers 646, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    10. Elias Bouacida & Renaud Foucart, 2022. "Rituals of Reason," Working Papers 344119591, Lancaster University Management School, Economics Department.
    11. Fellner-Röhling, Gerlinde & Hromek, Kristijan & Kleinknecht, Janina & Ludwig, Sandra, 2023. "Reciprocal reactions to (in)transparent task assignments: An experimental investigation," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 106(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    2. Kirchsteiger, Georg & Aldashev, Gani & Sebald, Alexander, 2009. "Decision-making Procedures: A General Theory and Its Field Experimental Test," CEPR Discussion Papers 7365, C.E.P.R. Discussion Papers.
    3. Miettinen, Topi, 2013. "Promises and conventions – An approach to pre-play agreements," Games and Economic Behavior, Elsevier, vol. 80(C), pages 68-84.
    4. Bierbrauer, Felix & Netzer, Nick, 2016. "Mechanism design and intentions," Journal of Economic Theory, Elsevier, vol. 163(C), pages 557-603.
    5. Pikulina, Elena S. & Tergiman, Chloe, 2020. "Preferences for power," Journal of Public Economics, Elsevier, vol. 185(C).
    6. Sarkar, Sumit, 2019. "Gratitude, conscience, and reciprocity: Models of supplier motivation when quality is non-contractible," European Journal of Operational Research, Elsevier, vol. 277(2), pages 633-642.
    7. Philippe Grégoire, 2018. "Psychology at work," International Review of Economics, Springer;Happiness Economics and Interpersonal Relations (HEIRS), vol. 65(2), pages 119-135, June.
    8. Bracht, Jürgen & Regner, Tobias, 2013. "Moral emotions and partnership," Journal of Economic Psychology, Elsevier, vol. 39(C), pages 313-326.
    9. Jagau, Stephan & Perea, Andrés, 2022. "Common belief in rationality in psychological games," Journal of Mathematical Economics, Elsevier, vol. 100(C).
    10. Dufwenberg, Martin & Patel, Amrish, 2019. "Introduction to special issue on psychological game theory," Journal of Economic Behavior & Organization, Elsevier, vol. 167(C), pages 181-184.
    11. Jiabin Wu, 2018. "Indirect higher order beliefs and cooperation," Experimental Economics, Springer;Economic Science Association, vol. 21(4), pages 858-876, December.
    12. Dufwenberg, Martin & Gächter, Simon & Hennig-Schmidt, Heike, 2011. "The framing of games and the psychology of play," Games and Economic Behavior, Elsevier, vol. 73(2), pages 459-478.
    13. Dhami, Sanjit & Wei, Mengxing & al-Nowaihi, Ali, 2019. "Public goods games and psychological utility: Theory and evidence," Journal of Economic Behavior & Organization, Elsevier, vol. 167(C), pages 361-390.
    14. Anderlini, Luca & Terlizzese, Daniele, 2017. "Equilibrium trust," Games and Economic Behavior, Elsevier, vol. 102(C), pages 624-644.
    15. Maroš Servátka & Steven Tucker & Radovan Vadovič, 2011. "Building Trust—One Gift at a Time," Games, MDPI, vol. 2(4), pages 1-22, September.
    16. Eric Cardella, 2016. "Exploiting the guilt aversion of others: do agents do it and is it effective?," Theory and Decision, Springer, vol. 80(4), pages 523-560, April.
    17. Tobias Regner & Gerhard Riener, 2011. "Motivational Cherry Picking," Jena Economics Research Papers 2011-029, Friedrich-Schiller-University Jena.
    18. Martin Dufwenberg & Simon Gaechter & Heike Hennig-Schmidt, 2006. "The Framing of Games and the Psychology of Strategic Choice," Discussion Papers 2006-20, The Centre for Decision Research and Experimental Economics, School of Economics, University of Nottingham.
    19. Battigalli, Pierpaolo & Dufwenberg, Martin, 2009. "Dynamic psychological games," Journal of Economic Theory, Elsevier, vol. 144(1), pages 1-35, January.
    20. Peeters, Ronald & Vorsatz, Marc, 2021. "Simple guilt and cooperation," Journal of Economic Psychology, Elsevier, vol. 82(C).

    More about this item

    Keywords

    Randomized controlled trials; Policy experiments; Internal validity; Procedural concerns; Psychological game theory.;
    All these keywords.

    JEL classification:

    • C70 - Mathematical and Quantitative Methods - - Game Theory and Bargaining Theory - - - General
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • D63 - Microeconomics - - Welfare Economics - - - Equity, Justice, Inequality, and Other Normative Criteria and Measurement
    • I38 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - Government Programs; Provision and Effects of Welfare Programs
    • O12 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Microeconomic Analyses of Economic Development

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cca:wpaper:292. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Giovanni Bert (email available below). General contact details of provider: https://edirc.repec.org/data/fccaait.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.