Nothing Special   »   [go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp14594.html
   My bibliography  Save this paper

Estimating Coherency between Survey Data and Incentivized Experimental Data

Author

Listed:
  • Belzil, Christian

    (Ecole Polytechnique, Paris)

  • Pernaudet, Julie

    (University of Chicago)

  • Poinas, François

    (Toulouse School of Economics)

Abstract
Imagine the situation in which an econometrician can infer the distribution of welfare gains induced by the provision of higher education financial aid using survey data obtained from a set of individuals, and can estimate the same distribution using a highly incentivized field experiment in which the same set of individuals participated. In the experimental setting relying on incentivized choices, making the wrong decision can be costly. In the survey, the stakes are null and reporting false intentions and expectations is costless. In this paper, we evaluate the extent to which the decomposition of the two welfare gain distributions into latent factors are coherent. We find that individuals often put a much different weight to a specific set of determinants in the experiment and in the survey and that the valuations of financial aid are rank incoherent. About 66% of Biased Incoherency (defined as the tendency to have a higher valuation rank in the experiment than in the survey) is explained by individual heterogeneity in subjective benefits, costs and other factors and about half of these factors affect the welfare gains of financial aid in the survey and in the experiment in opposite directions. Ex-ante policy evaluation of a potential expansion of the Canadian higher education financial aid system may therefore depend heavily on whether or not the data have been obtained in an "incentivized" context.

Suggested Citation

  • Belzil, Christian & Pernaudet, Julie & Poinas, François, 2021. "Estimating Coherency between Survey Data and Incentivized Experimental Data," IZA Discussion Papers 14594, Institute of Labor Economics (IZA).
    • Handle: RePEc:iza:izadps:dp14594
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp14594.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Arcidiacono, Peter & Hotz, V. Joseph & Kang, Songman, 2012. "Modeling college major choices using elicited measures of expectations and counterfactuals," Journal of Econometrics, Elsevier, vol. 166(1), pages 3-16.
    2. Christian Belzil & Arnaud Maurel & Modibo Sidibé, 2021. "Estimating the Value of Higher Education Financial Aid: Evidence from a Field Experiment," Journal of Labor Economics, University of Chicago Press, vol. 39(2), pages 361-395.
    3. Filip Matêjka & Alisdair McKay, 2015. "Rational Inattention to Discrete Choices: A New Foundation for the Multinomial Logit Model," American Economic Review, American Economic Association, vol. 105(1), pages 272-298, January.
    4. James J. Heckman & John Eric Humphries & Gregory Veramendi, 2018. "Returns to Education: The Causal Effects of Education on Earnings, Health, and Smoking," Journal of Political Economy, University of Chicago Press, vol. 126(S1), pages 197-246.
    5. Lochner, L. & Monge-Naranjo, A., 2016. "Student Loans and Repayment," Handbook of the Economics of Education,, Elsevier.
    6. Benjamin Williams, 2020. "Identification of the linear factor model," Econometric Reviews, Taylor & Francis Journals, vol. 39(1), pages 92-109, January.
    7. Uri Gneezy & John A. List & Jeffrey A. Livingston & Xiangdong Qin & Sally Sadoff & Yang Xu, 2019. "Measuring Success in Education: The Role of Effort on the Test Itself," American Economic Review: Insights, American Economic Association, vol. 1(3), pages 291-308, December.
    8. Belzil, Christian, 2007. "The return to schooling in structural dynamic models: a survey," European Economic Review, Elsevier, vol. 51(5), pages 1059-1105, July.
    9. Christian Belzil & Jörgen Hansen, 2002. "Unobserved Ability and the Return to Schooling," Econometrica, Econometric Society, vol. 70(5), pages 2075-2091, September.
    10. Levon Barseghyan & Francesca Molinari & Matthew Thirkettle, 2021. "Discrete Choice under Risk with Limited Consideration," American Economic Review, American Economic Association, vol. 111(6), pages 1972-2006, June.
    11. Ola Andersson & Håkan J. Holm & Jean-Robert Tyran & Erik Wengström, 2020. "Robust inference in risk elicitation tasks," Journal of Risk and Uncertainty, Springer, vol. 61(3), pages 195-209, December.
    12. Frenette, Marc, 2014. "An Investment of a Lifetime? The Long-term Labour Market Premiums Associated with a Postsecondary Education," Analytical Studies Branch Research Paper Series 2014359e, Statistics Canada, Analytical Studies Branch.
    13. Jared Ashworth & V. Joseph Hotz & Arnaud Maurel & Tyler Ransom, 2021. "Changes across Cohorts in Wage Returns to Schooling and Early Work Experiences," Journal of Labor Economics, University of Chicago Press, vol. 39(4), pages 931-964.
    14. Matthew Wiswall & Basit Zafar, 2015. "Determinants of College Major Choice: Identification using an Information Experiment," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 82(2), pages 791-824.
    15. Ralph Stinebrickner & Todd R. Stinebrickner, 2014. "A Major in Science? Initial Beliefs and Final Outcomes for College Major and Dropout," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 81(1), pages 426-472.
    16. Adeline Delavande & Basit Zafar, 2019. "University Choice: The Role of Expected Earnings, Nonpecuniary Outcomes, and Financial Constraints," Journal of Political Economy, University of Chicago Press, vol. 127(5), pages 2343-2393.
    17. Peter Arcidiacono, 2005. "Affirmative Action in Higher Education: How Do Admission and Financial Aid Rules Affect Future Earnings?," Econometrica, Econometric Society, vol. 73(5), pages 1477-1524, September.
    18. Tomáš Jagelka, 2024. "Are Economists’ Preferences Psychologists’ Personality Traits? A Structural Approach," Journal of Political Economy, University of Chicago Press, vol. 132(3), pages 910-970.
    19. Yifan Gong & Lance Lochner & Ralph Stinebrickner & Todd R. Stinebrickner, 2019. "The Consumption Value of College," NBER Working Papers 26335, National Bureau of Economic Research, Inc.
    20. Benjamin Williams, 2018. "Identification of the Linear Factor Model," Working Papers 2018-002, The George Washington University, Department of Economics, H. O. Stekler Research Program on Forecasting.
    21. Patrick Bajari & Jeremy T. Fox & Stephen P. Ryan, 2007. "Linear Regression Estimation of Discrete Choice Models with Nonparametric Distributions of Random Coefficients," American Economic Review, American Economic Association, vol. 97(2), pages 459-463, May.
    22. Manski, Charles F. & Molinari, Francesca, 2010. "Rounding Probabilistic Expectations in Surveys," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(2), pages 219-231.
    23. Zvi Eckstein & Kenneth I. Wolpin, 1999. "Why Youths Drop Out of High School: The Impact of Preferences, Opportunities, and Abilities," Econometrica, Econometric Society, vol. 67(6), pages 1295-1340, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Christian Belzil & Jörgen Hansen & Julie Pernaudet, 2024. "Les déterminants cognitifs et non-cognitifs du choix de filière et leur impact sur la phase initiale du cycle professionnel," CIRANO Project Reports 2024rp-06, CIRANO.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Christian Belzil & Jörgen Hansen, 2020. "The evolution of the US family income–schooling relationship and educational selectivity," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(7), pages 841-859, November.
    2. Xavier D'Haultfoeuille & Christophe Gaillac & Arnaud Maurel, 2021. "Rationalizing rational expectations: Characterizations and tests," Quantitative Economics, Econometric Society, vol. 12(3), pages 817-842, July.
    3. Pamela Giustinelli & Charles F. Manski, 2018. "Survey Measures Of Family Decision Processes For Econometric Analysis Of Schooling Decisions," Economic Inquiry, Western Economic Association International, vol. 56(1), pages 81-99, January.
    4. Christian Belzil & Jorgen Hansen, 2020. "Reconciling Changes in Wage Inequality with Changes in College Selectivity using a Behavioral Model," Working Papers 20005, Concordia University, Department of Economics.
    5. Kemptner, Daniel & Tolan, Songül, 2018. "The role of time preferences in educational decision making," Economics of Education Review, Elsevier, vol. 67(C), pages 25-39.
    6. Johannes S. Kunz & Kevin E. Staub, 2016. "Subjective Completion Beliefs and the Demand for Post-Secondary Education," SOEPpapers on Multidisciplinary Panel Data Research 878, DIW Berlin, The German Socio-Economic Panel (SOEP).
    7. Pamela Giustinelli, 2016. "Group Decision Making With Uncertain Outcomes: Unpacking Child–Parent Choice Of The High School Track," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 57(2), pages 573-602, May.
    8. Romuald Meango & Esther Mirjam Girsberger, 2023. "Identification of Ex ante Returns Using Elicited Choice Probabilities: an Application to Preferences for Public-sector Jobs," Papers 2303.03009, arXiv.org, revised Jun 2024.
    9. Fernanda Estevan & Thomas Gall & Louis-Philippe Morin, 2019. "Can Affirmative Action Affect Major Choice?," Boston University - Department of Economics - The Institute for Economic Development Working Papers Series dp-324, Boston University - Department of Economics.
    10. Grewenig, Elisabeth & Lergetporer, Philipp & Werner, Katharina & Woessmann, Ludger, 2022. "Incentives, search engines, and the elicitation of subjective beliefs: Evidence from representative online survey experiments," Journal of Econometrics, Elsevier, vol. 231(1), pages 304-326.
    11. Yifan Gong & Todd Stinebrickner & Ralph Stinebrickner, 2020. "Perceived and actual option values of college enrollment," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(7), pages 940-959, November.
    12. D'Haultfoeuille, Xavier & Gaillac, Christophe & Maurel, Arnaud, 2018. "Rationalizing Rational Expectations? Tests and Deviations," IZA Discussion Papers 11989, Institute of Labor Economics (IZA).
    13. Gizem Koşar & Cormac O'Dea, 2022. "Expectations Data in Structural Microeconomic Models," Staff Reports 1018, Federal Reserve Bank of New York.
    14. Peter Arcidiacono & Esteban M. Aucejo & V. Joseph Hotz, 2016. "University Differences in the Graduation of Minorities in STEM Fields: Evidence from California," American Economic Review, American Economic Association, vol. 106(3), pages 525-562, March.
    15. Gong, Yifan & Stinebrickner, Ralph & Stinebrickner, Todd, 2022. "Marriage, children, and labor supply: Beliefs and outcomes," Journal of Econometrics, Elsevier, vol. 231(1), pages 148-164.
    16. Tyler Ransom & Esteban Aucejo & Arnaud Maurel & Peter Arcidiacono, 2014. "College Attrition and the Dynamics of Information Revelation," 2014 Meeting Papers 529, Society for Economic Dynamics.
    17. Michele Giannola, 2024. "Parental Investments and Intra-household Inequality in Child Human Capital: Evidence from a Survey Experiment," The Economic Journal, Royal Economic Society, vol. 134(658), pages 671-727.
    18. Bhalotra, Sonia & Delavande, Adeline & Font-Gilabert, Paulino & Maselko, Joanna, 2022. "Maternal Investments in Children : The Role of Expected Effort and Returns," CAGE Online Working Paper Series 637, Competitive Advantage in the Global Economy (CAGE).
    19. Fernando Saltiel, 2019. "What's Math Got to Do With It? Multidimensional Ability and the Gender Gap in STEM," 2019 Meeting Papers 1201, Society for Economic Dynamics.
    20. Bhalotra, Sonia & Delavande, Adeline & Font-Gilabert, Paulino & Maselko, Joanna, 2022. "Maternal Investments in Children : The Role of Expected Effort and Returns," The Warwick Economics Research Paper Series (TWERPS) 1423, University of Warwick, Department of Economics.

    More about this item

    Keywords

    field experiment; survey data; coherency; incentives;
    All these keywords.

    JEL classification:

    • I2 - Health, Education, and Welfare - - Education
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • D12 - Microeconomics - - Household Behavior - - - Consumer Economics: Empirical Analysis
    • D9 - Microeconomics - - Micro-Based Behavioral Economics
    • D91 - Microeconomics - - Micro-Based Behavioral Economics - - - Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp14594. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.