Nothing Special   »   [go: up one dir, main page]
IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2306.11689.html
   My bibliography  Save this paper

Statistical Tests for Replacing Human Decision Makers with Algorithms

Author

Listed:
  • Kai Feng
  • Han Hong
  • Ke Tang
  • Jingyuan Wang
Abstract
This paper proposes a statistical framework with which artificial intelligence can improve human decision making. The performance of each human decision maker is first benchmarked against machine predictions; we then replace the decisions made by a subset of the decision makers with the recommendation from the proposed artificial intelligence algorithm. Using a large nationwide dataset of pregnancy outcomes and doctor diagnoses from prepregnancy checkups of reproductive age couples, we experimented with both a heuristic frequentist approach and a Bayesian posterior loss function approach with an application to abnormal birth detection. We find that our algorithm on a test dataset results in a higher overall true positive rate and a lower false positive rate than the diagnoses made by doctors only. We also find that the diagnoses of doctors from rural areas are more frequently replaceable, suggesting that artificial intelligence assisted decision making tends to improve precision more in less developed regions.

Suggested Citation

  • Kai Feng & Han Hong & Ke Tang & Jingyuan Wang, 2023. "Statistical Tests for Replacing Human Decision Makers with Algorithms," Papers 2306.11689, arXiv.org.
    • Handle: RePEc:arx:papers:2306.11689
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2306.11689
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Andre Esteva & Brett Kuprel & Roberto A. Novoa & Justin Ko & Susan M. Swetter & Helen M. Blau & Sebastian Thrun, 2017. "Correction: Corrigendum: Dermatologist-level classification of skin cancer with deep neural networks," Nature, Nature, vol. 546(7660), pages 686-686, June.
    2. Toru Kitagawa & Aleksey Tetenov, 2018. "Who Should Be Treated? Empirical Welfare Maximization Methods for Treatment Choice," Econometrica, Econometric Society, vol. 86(2), pages 591-616, March.
    3. Susan Athey & Stefan Wager, 2021. "Policy Learning With Observational Data," Econometrica, Econometric Society, vol. 89(1), pages 133-161, January.
    4. Andre Esteva & Brett Kuprel & Roberto A. Novoa & Justin Ko & Susan M. Swetter & Helen M. Blau & Sebastian Thrun, 2017. "Dermatologist-level classification of skin cancer with deep neural networks," Nature, Nature, vol. 542(7639), pages 115-118, February.
    5. Jonathan Gruber & Maria Owings, 1996. "Physician Financial Incentives and Cesarean Section Delivery," RAND Journal of Economics, The RAND Corporation, vol. 27(1), pages 99-123, Spring.
    6. Janet Currie & W. Bentley MacLeod, 2017. "Diagnosing Expertise: Human Capital, Decision Making, and Performance among Physicians," Journal of Labor Economics, University of Chicago Press, vol. 35(1), pages 1-43.
    7. Gruber, Jon & Kim, John & Mayzlin, Dina, 1999. "Physician fees and procedure intensity: the case of cesarean delivery," Journal of Health Economics, Elsevier, vol. 18(4), pages 473-490, August.
    8. Jiaming Zeng & Berk Ustun & Cynthia Rudin, 2017. "Interpretable classification models for recidivism prediction," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(3), pages 689-722, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Barili, Emilia & Bertoli, Paola & Grembi, Veronica, 2021. "Fee equalization and appropriate health care," Economics & Human Biology, Elsevier, vol. 41(C).
    2. Gabriel A. Facchini Palma, 2020. "Low Staffing in the Maternity Ward: Keep Calm and Call the Surgeon," Working Papers wpdea2009, Department of Applied Economics at Universitat Autonoma of Barcelona.
    3. Heike Hennig‐Schmidt & Hendrik Jürges & Daniel Wiesen, 2019. "Dishonesty in health care practice: A behavioral experiment on upcoding in neonatology," Health Economics, John Wiley & Sons, Ltd., vol. 28(3), pages 319-338, March.
    4. Sofia Amaral-Garcia & Paola Bertoli & Veronica Grembi, 2015. "Does Experience Rating Improve Obstetric Practices? Evidence From Geographical Discontinuities in Italy," CEIS Research Paper 342, Tor Vergata University, CEIS, revised 08 May 2015.
    5. David Card & Alessandra Fenizia & David Silver, 2023. "The Health Impacts of Hospital Delivery Practices," American Economic Journal: Economic Policy, American Economic Association, vol. 15(2), pages 42-81, May.
    6. Barili, E; & Bertoli, P; & Grembi, V;, 2020. "Title: Fees equalization and Appropriate Health Care," Health, Econometrics and Data Group (HEDG) Working Papers 20/09, HEDG, c/o Department of Economics, University of York.
    7. Toru Kitagawa & Shosei Sakaguchi & Aleksey Tetenov, 2021. "Constrained Classification and Policy Learning," Papers 2106.12886, arXiv.org, revised Jul 2023.
    8. Facchini, Gabriel, 2022. "Low staffing in the maternity ward: Keep calm and call the surgeon," Journal of Economic Behavior & Organization, Elsevier, vol. 197(C), pages 370-394.
    9. Karen Norberg & Juan Pantano, 2016. "Cesarean sections and subsequent fertility," Journal of Population Economics, Springer;European Society for Population Economics, vol. 29(1), pages 5-37, January.
    10. Yi Zhang & Kosuke Imai, 2023. "Individualized Policy Evaluation and Learning under Clustered Network Interference," Papers 2311.02467, arXiv.org, revised Feb 2024.
    11. Manski, Charles F., 2023. "Probabilistic prediction for binary treatment choice: With focus on personalized medicine," Journal of Econometrics, Elsevier, vol. 234(2), pages 647-663.
    12. Lin Lu & Laurent Dercle & Binsheng Zhao & Lawrence H. Schwartz, 2021. "Deep learning for the prediction of early on-treatment response in metastatic colorectal cancer from serial medical imaging," Nature Communications, Nature, vol. 12(1), pages 1-11, December.
    13. Jeffrey Clemens & Joshua D. Gottlieb & Jeffrey Hicks, 2021. "How Would Medicare for All Affect Health System Capacity? Evidence from Medicare for Some," Tax Policy and the Economy, University of Chicago Press, vol. 35(1), pages 225-262.
    14. Garbero, Alessandra & Sakos, Grayson & Cerulli, Giovanni, 2023. "Towards data-driven project design: Providing optimal treatment rules for development projects," Socio-Economic Planning Sciences, Elsevier, vol. 89(C).
    15. Zheng Yan & Wenqian Robertson & Yaosheng Lou & Tom W. Robertson & Sung Yong Park, 2021. "Finding leading scholars in mobile phone behavior: a mixed-method analysis of an emerging interdisciplinary field," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9499-9517, December.
    16. Thomas Buchmueller & John C. Ham & Lara D. Shore-Sheppard, 2015. "The Medicaid Program," NBER Chapters, in: Economics of Means-Tested Transfer Programs in the United States, Volume 1, pages 21-136, National Bureau of Economic Research, Inc.
    17. Freddy Gabbay & Rotem Lev Aharoni & Ori Schweitzer, 2022. "Deep Neural Network Memory Performance and Throughput Modeling and Simulation Framework," Mathematics, MDPI, vol. 10(21), pages 1-20, November.
    18. Undral Byambadalai, 2022. "Identification and Inference for Welfare Gains without Unconfoundedness," Papers 2207.04314, arXiv.org.
    19. Carine Milcent & Julie Rochut, 2009. "Tarification hospitalière et pratique médicale. La pratique de la césarienne en France," Revue économique, Presses de Sciences-Po, vol. 60(2), pages 489-506.
    20. Yuchen Hu & Henry Zhu & Emma Brunskill & Stefan Wager, 2024. "Minimax-Regret Sample Selection in Randomized Experiments," Papers 2403.01386, arXiv.org, revised Jun 2024.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2306.11689. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.