Nothing Special   »   [go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/27736.html
   My bibliography  Save this paper

Hiring as Exploration

Author

Listed:
  • Danielle Li
  • Lindsey R. Raymond
  • Peter Bergman
Abstract
This paper views hiring as a contextual bandit problem: to find the best workers over time, firms must balance “exploitation” (selecting from groups with proven track records) with “exploration” (selecting from under-represented groups to learn about quality). Yet modern hiring algorithms, based on “supervised learning” approaches, are designed solely for exploitation. Instead, we build a resume screening algorithm that values exploration by evaluating candidates according to their statistical upside potential. Using data from professional services recruiting within a Fortune 500 firm, we show that this approach improves the quality (as measured by eventual hiring rates) of candidates selected for an interview, while also increasing demographic diversity, relative to the firm's existing practices. The same is not true for traditional supervised learning based algorithms, which improve hiring rates but select far fewer Black and Hispanic applicants. In an extension, we show that exploration-based algorithms are also able to learn more effectively about simulated changes in applicant hiring potential over time. Together, our results highlight the importance of incorporating exploration in developing decision-making algorithms that are potentially both more efficient and equitable.

Suggested Citation

  • Danielle Li & Lindsey R. Raymond & Peter Bergman, 2020. "Hiring as Exploration," NBER Working Papers 27736, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:27736
    Note: CF ED LE LS POL PR TWP
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w27736.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Mitchell Hoffman & Lisa B Kahn & Danielle Li, 2018. "Discretion in Hiring," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(2), pages 765-800.
    2. Janet M. Currie & W. Bentley MacLeod, 2020. "Understanding Doctor Decision Making: The Case of Depression Treatment," Econometrica, Econometric Society, vol. 88(3), pages 847-878, May.
    3. DiNardo, John & Fortin, Nicole M & Lemieux, Thomas, 1996. "Labor Market Institutions and the Distribution of Wages, 1973-1992: A Semiparametric Approach," Econometrica, Econometric Society, vol. 64(5), pages 1001-1044, September.
    4. Peter J. Kuhn & Lizi Yu, 2019. "How Costly is Turnover? Evidence from Retail," NBER Working Papers 26179, National Bureau of Economic Research, Inc.
    5. Anja Lambrecht & Catherine Tucker, 2019. "Algorithmic Bias? An Empirical Study of Apparent Gender-Based Discrimination in the Display of STEM Career Ads," Management Science, INFORMS, vol. 65(7), pages 2966-2981, July.
    6. Zhengyuan Zhou & Susan Athey & Stefan Wager, 2023. "Offline Multi-Action Policy Learning: Generalization and Optimization," Operations Research, INFORMS, vol. 71(1), pages 148-183, January.
    7. Amanda Agan & Sonja Starr, 2016. "Ban the Box, Criminal Records, and Statistical Discrimination: A Field Experiment," Natural Field Experiments 00539, The Field Experiments Website.
    8. Abadie, Alberto, 2003. "Semiparametric instrumental variable estimation of treatment response models," Journal of Econometrics, Elsevier, vol. 113(2), pages 231-263, April.
    9. Alan Benson & Danielle Li & Kelly Shue, 2019. "Promotions and the Peter Principle," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 134(4), pages 2085-2134.
    10. Olof Åslund & Oskar Nordströum Skans, 2012. "Do Anonymous Job Application Procedures Level the Playing Field?," ILR Review, Cornell University, ILR School, vol. 65(1), pages 82-107, January.
    11. David J. Deming, 2017. "The Growing Importance of Social Skills in the Labor Market," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(4), pages 1593-1640.
    12. Patrick Kline & Christopher Walters, 2021. "Reasonable Doubt: Experimental Detection of Job‐Level Employment Discrimination," Econometrica, Econometric Society, vol. 89(2), pages 765-792, March.
    13. Jeffrey R. Kling, 2006. "Incarceration Length, Employment, and Earnings," American Economic Review, American Economic Association, vol. 96(3), pages 863-876, June.
    14. Ashesh Rambachan & Jon Kleinberg & Sendhil Mullainathan & Jens Ludwig, 2020. "An Economic Approach to Regulating Algorithms," NBER Working Papers 27111, National Bureau of Economic Research, Inc.
    15. Maximilian Kasy & Anja Sautmann, 2021. "Adaptive Treatment Assignment in Experiments for Policy Choice," Econometrica, Econometric Society, vol. 89(1), pages 113-132, January.
    16. A Stefano Caria & Grant Gordon & Maximilian Kasy & Simon Quinn & Soha Osman Shami & Alexander Teytelboym, 2024. "An Adaptive Targeted Field Experiment: Job Search Assistance for Refugees in Jordan," Journal of the European Economic Association, European Economic Association, vol. 22(2), pages 781-836.
    17. Sanath Kumar Krishnamurthy & Susan Athey, 2020. "Survey Bandits with Regret Guarantees," Papers 2002.09814, arXiv.org.
    18. Cecilia Rouse & Claudia Goldin, 2000. "Orchestrating Impartiality: The Impact of "Blind" Auditions on Female Musicians," American Economic Review, American Economic Association, vol. 90(4), pages 715-741, September.
    19. Ray Reagans & Ezra W. Zuckerman, 2001. "Networks, Diversity, and Productivity: The Social Capital of Corporate R&D Teams," Organization Science, INFORMS, vol. 12(4), pages 502-517, August.
    20. Amanda Agan & Sonja Starr, 2018. "Ban the Box, Criminal Records, and Racial Discrimination: A Field Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(1), pages 191-235.
    21. Terry‐Ann Craigie, 2020. "Ban The Box, Convictions, And Public Employment," Economic Inquiry, Western Economic Association International, vol. 58(1), pages 425-445, January.
    22. Luc Behaghel & Bruno Crépon & Thomas Le Barbanchon, 2015. "Unintended Effects of Anonymous Résumés," American Economic Journal: Applied Economics, American Economic Association, vol. 7(3), pages 1-27, July.
    23. Luc Behaghel & Bruno Crépon & Thomas Le Barbanchon, 2015. "Unintended Effects of Anonymous," Post-Print halshs-01203070, HAL.
    24. repec:hal:pseose:halshs-01203070 is not listed on IDEAS
    25. Susan Athey & Scott Stern, 1998. "An Empirical Framework for Testing Theories About Complimentarity in Organizational Design," NBER Working Papers 6600, National Bureau of Economic Research, Inc.
    26. Jon Kleinberg & Himabindu Lakkaraju & Jure Leskovec & Jens Ludwig & Sendhil Mullainathan, 2018. "Human Decisions and Machine Predictions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(1), pages 237-293.
    27. Julian Kolev & Yuly Fuentes-Medel & Fiona Murray, 2019. "Is Blinded Review Enough? How Gendered Outcomes Arise Even Under Anonymous Evaluation," NBER Working Papers 25759, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Peter Leopold S. Bergman & Elizabeth Kopko & Julio Rodriguez, 2021. "Using Predictive Analytics to Track Students: Evidence from a Seven-College Experiment," CESifo Working Paper Series 9157, CESifo.
    2. Mario Alberto García-Meza, 2021. "The Cost of Work Discrimination: A Market Capture Differential Game Model," Mathematics, MDPI, vol. 9(19), pages 1-10, September.
    3. Jentjens, Sabine & Cherbib, Jihène, 2023. "Trust me if you can – Do trust propensities in granting working-from-home arrangements change during times of exogenous shocks?," Journal of Business Research, Elsevier, vol. 161(C).
    4. Zhang, Lixuan & Yencha, Christopher, 2022. "Examining perceptions towards hiring algorithms," Technology in Society, Elsevier, vol. 68(C).
    5. Richard T. Carson & Joshua Graff Zivin & Jordan J. Louviere & Sally Sadoff & Jeffrey G. Shrader, 2022. "The Risk of Caution: Evidence from an Experiment," Management Science, INFORMS, vol. 68(12), pages 9042-9060, December.
    6. Diana Moreira & Santiago Pérez, 2022. "Who Benefits from Meritocracy?," NBER Working Papers 30113, National Bureau of Economic Research, Inc.
    7. Jonas Radbruch & Amelie Schiprowski, 2020. "Interview Sequences and the Formation of Subjective Assessments," ECONtribute Discussion Papers Series 045, University of Bonn and University of Cologne, Germany.
    8. Laura Blattner & Scott Nelson & Jann Spiess, 2021. "Unpacking the Black Box: Regulating Algorithmic Decisions," Papers 2110.03443, arXiv.org, revised May 2024.
    9. Laschever, Ron A. & Weinstein, Russell, 2021. "Preference Signaling and Worker-Firm Matching: Evidence from Interview Auctions," IZA Discussion Papers 14622, Institute of Labor Economics (IZA).
    10. Claudio Cardoso Flores & Marcelo Cunha Medeiros, 2020. "Online Action Learning in High Dimensions: A Conservative Perspective," Papers 2009.13961, arXiv.org, revised Mar 2024.
    11. Jin Li & Ye Luo & Xiaowei Zhang, 2021. "Causal Reinforcement Learning: An Instrumental Variable Approach," Papers 2103.04021, arXiv.org, revised Sep 2022.
    12. Fumagalli, Elena & Rezaei, Sarah & Salomons, Anna, 2022. "OK computer: Worker perceptions of algorithmic recruitment," Research Policy, Elsevier, vol. 51(2).
    13. Luca Coraggio & Marco Pagano & Annalisa Scognamiglio & Joacim Tåg, 2022. "JAQ of All Trades: Job Mismatch, Firm Productivity and Managerial Quality," EIEF Working Papers Series 2205, Einaudi Institute for Economics and Finance (EIEF), revised Mar 2022.
    14. Marina Chugunova & Wolfgang J. Luhan, 2022. "Ruled by robots: Preference for algorithmic decision makers and perceptions of their choices," Working Papers in Economics & Finance 2022-03, University of Portsmouth, Portsmouth Business School, Economics and Finance Subject Group.
    15. Benedikt Holtgen & Robert C. Williamson, 2024. "Causal modelling without introducing counterfactuals or abstract distributions," Papers 2407.17385, arXiv.org, revised Aug 2024.
    16. Lepage, Louis Pierre, 2021. "Endogenous learning, persistent employer biases, and discrimination," CLEF Working Paper Series 34, Canadian Labour Economics Forum (CLEF), University of Waterloo.
    17. Moran Koren, 2023. "The Gatekeeper Effect: The Implications of Pre-Screening, Self-selection, and Bias for Hiring Processes," Papers 2312.17167, arXiv.org.
    18. Pisanelli, Elena, 2022. "Your resume is your gatekeeper: Automated resume screening as a strategy to reduce gender gaps in hiring," Economics Letters, Elsevier, vol. 221(C).
    19. Victor Alfonso Naya & Guillaume Bied & Philippe Caillou & Bruno Crépon & Christophe Gaillac & Elia Pérennes & Michèle Sebag, 2021. "Designing labor market recommender systems: the importance of job seeker preferences and competition," Post-Print hal-03540319, HAL.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ohto Kanninen & Sanni Kiviholma & Tuomo Virkola, "undated". "Anatomy of An Anonymous Hiring Pilot," Working Papers 338, Työn ja talouden tutkimus LABORE, The Labour Institute for Economic Research LABORE.
    2. Lee, Logan M. & Waddell, Glen R., 2021. "Diversity and the timing of preference in hiring decisions," Journal of Economic Behavior & Organization, Elsevier, vol. 184(C), pages 432-459.
    3. Marianne Bertrand & Esther Duflo, 2016. "Field Experiments on Discrimination," NBER Working Papers 22014, National Bureau of Economic Research, Inc.
    4. David Neumark, 2018. "Experimental Research on Labor Market Discrimination," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 799-866, September.
    5. Bharti, Nitin Kumar & Roy, Sutanuka, 2023. "The early origins of judicial stringency in bail decisions: Evidence from early childhood exposure to Hindu-Muslim riots in India," Journal of Public Economics, Elsevier, vol. 221(C).
    6. Shuai Chen & Jan C. van Ours, 2022. "Mental health effects of same‐sex marriage legalization," Health Economics, John Wiley & Sons, Ltd., vol. 31(1), pages 42-56, January.
    7. Amanda Y. Agan & Michael D. Makowsky, 2023. "The Minimum Wage, EITC, and Criminal Recidivism," Journal of Human Resources, University of Wisconsin Press, vol. 58(5), pages 1712-1751.
    8. Sylvain Chareyron & Yannick L'Horty & Pascale Petit & Souleymane Mbaye, 2019. "Mesurer l’impact d’un courrier d’alerte sur les discriminations liées à l’origine," Working Papers hal-02333840, HAL.
    9. Darolia, Rajeev & Mueser, Peter & Cronin, Jacob, 2021. "Labor market returns to a prison GED," Economics of Education Review, Elsevier, vol. 82(C).
    10. Moshe A. Barach & John Horton, 2017. "How Do Employers Use Compensation History?: Evidence from a Field Experiment," CESifo Working Paper Series 6559, CESifo.
    11. Runshan Fu & Ginger Zhe Jin & Meng Liu, 2022. "Does Human-algorithm Feedback Loop Lead to Error Propagation? Evidence from Zillow’s Zestimate," NBER Working Papers 29880, National Bureau of Economic Research, Inc.
    12. Marianne Bertrand & Esther Duflo, 2016. "Field Experiments on Discrimination," NBER Working Papers 22014, National Bureau of Economic Research, Inc.
    13. Bauer, Kevin & Pfeuffer, Nicolas & Abdel-Karim, Benjamin M. & Hinz, Oliver & Kosfeld, Michael, 2020. "The terminator of social welfare? The economic consequences of algorithmic discrimination," SAFE Working Paper Series 287, Leibniz Institute for Financial Research SAFE.
    14. Berson, Clémence & Laouénan, Morgane & Valat, Emmanuel, 2020. "Outsourcing recruitment as a solution to prevent discrimination: A correspondence study," Labour Economics, Elsevier, vol. 64(C).
    15. Bastien Michel & Camille Hémet, 2022. "Custodial versus non-custodial sentences: Long-run evidence from an anticipated reform," PSE Working Papers halshs-03899897, HAL.
    16. Nicolás Grau & Damián Vergara, "undated". "A Simple Test for Prejudice in Decision Processes: The Prediction-Based Outcome Test," Working Papers wp493, University of Chile, Department of Economics.
    17. Joshua M. Congdon-Hohman, 2018. "The persistent labor market effects of a criminal conviction and �Ban the Box� reforms," Working Papers 1808, College of the Holy Cross, Department of Economics.
    18. Huber, Martin, 2019. "An introduction to flexible methods for policy evaluation," FSES Working Papers 504, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    19. Jeffrey Penney & Steven Lehrer & Emilia Galan, 2024. "Mandatory minimum sentencing and its effect on sentencing distributions: Evidence from Canada," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 57(1), pages 55-77, February.
    20. Ingrid Haegele, 2022. "Talent Hoarding in Organizations," Papers 2206.15098, arXiv.org, revised Mar 2024.

    More about this item

    JEL classification:

    • D80 - Microeconomics - - Information, Knowledge, and Uncertainty - - - General
    • J20 - Labor and Demographic Economics - - Demand and Supply of Labor - - - General
    • M15 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Business Administration - - - IT Management
    • M51 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Personnel Economics - - - Firm Employment Decisions; Promotions
    • O33 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Technological Change: Choices and Consequences; Diffusion Processes

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:27736. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.