Using AI uncertainty quantification to improve human decision-making
Article No.: 1421, Pages 34949 - 34960
Abstract
AI Uncertainty Quantification (UQ) has the potential to improve human decision-making beyond AI predictions alone by providing additional probabilistic information to users. The majority of past research on AI and human decision-making has concentrated on model explainability and interpretability, with little focus on understanding the potential impact of UQ on human decision-making. We evaluated the impact on human decision-making for instance-level UQ, calibrated using a strict scoring rule, in two online behavioral experiments. In the first experiment, our results showed that UQ was beneficial for decision-making performance compared to only AI predictions. In the second experiment, we found UQ had generalizable benefits for decision-making across a variety of representations for probabilistic information. These results indicate that implementing high quality, instance-level UQ for AI may improve decision-making with real systems compared to AI predictions alone.
References
[1]
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., Makarenkov, V., and Nahavandi, S. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Inf. Fusion, 76 (C):243-297, dec 2021a. ISSN 1566-2535.
[2]
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information Fusion, 76:243-297, 2021b.
[3]
Alufaisan, Y., Marusich, L. R., Bakdash, J. Z., Zhou, Y., and Kantarcioglu, M. Does explainable artificial intelligence improve human decision-making? Proceedings of the AAAI Conference on Artificial Intelligence, 35(8):6618-6626, May 2021. URL https://ojs.aaai.org/index.php/AAAI/article/view/16819.
[4]
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. Guidelines for human-ai interaction. In Proceedings of the 2019 CHI conference on human factors in computing systems, pp. 1-13, 2019.
[5]
Bhatt, U., Antorán, J., Zhang, Y., Liao, Q. V., Sattigeri, P., Fogliato, R., Melançon, G., Krishnan, R., Stanley, J., Tickoo, O., Nachman, L., Chunara, R., Srikumar, M., Weller, A., and Xiang, A. Uncertainty as a form of transparency: Measuring, communicating, and using uncertainty. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, AIES '21, pp. 401-413, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450384735.
[6]
Buçinca, Z., Malaya, M. B., and Gajos, K. Z. To trust or to think: Cognitive forcing functions can reduce overreliance on ai in ai-assisted decision-making. Proc. ACM Hum.-Comput. Interact., 5(CSCW1), apr 2021.
[7]
Chen, C., Ding, N., and Carin, L. On the convergence of stochastic gradient mcmc algorithms with high-order integrators. In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2, NIPS'15, pp. 2278-2286, Cambridge, MA, USA, 2015. MIT Press.
[8]
Chen, T., Fox, E., and Guestrin, C. Stochastic gradient hamiltonian monte carlo. In Xing, E. P. and Jebara, T. (eds.), Proceedings of the 31st International Conference on Machine Learning, volume 32 of Proceedings of Machine Learning Research, pp. 1683-1691, Bejing, China, 22-24 Jun 2014. PMLR. URL https://proceedings.mlr.press/v32/cheni14.html.
[9]
Cokely, E. T., Galesic, M., Schulz, E., Ghazal, S., and Garcia-Retamero, R. Measuring risk literacy: The berlin numeracy test. Judgment and Decision making, 7(1): 25-47, 2012.
[10]
Cummings, M. M. Man versus machine or man+ machine? IEEE Intelligent Systems, 29(5):62-69, 2014.
[11]
Dawes, R. M., Faust, D., and Meehl, P. E. Clinical versus actuarial judgment. Science, 243(4899):1668-1674, 1989.
[12]
De Leeuw, J. R. jspsych: A javascript library for creating behavioral experiments in a web browser. Behavior research methods, 47:1-12, 2015.
[13]
Ding, N., Fang, Y., Babbush, R., Chen, C., Skeel, R. D., and Neven, H. Bayesian sampling using stochastic gradient thermostats. In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., and Weinberger, K. (eds.), Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014. URL https://proceedings.neurips.cc/paper_files/paper/2014/file/21fe5b8ba755eeaece7a450849876228-Paper.pdf.
[14]
Dua, D. and Graff, C. UCI machine learning repository, 2017. URL http://archive.ics.uci.edu/ml.
[15]
Dwaracherla, V., Wen, Z., Osband, I., Lu, X., Asghari, S. M., and Roy, B. V. Ensembles for uncertainty estimation: Benefits of prior functions and bootstrapping. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id=IqJsyulDUX.
[16]
Farquhar, S., Smith, L., and Gal, Y. Try depth instead of weight correlations: Mean-field is a less restrictive assumption for deeper networks. CoRR, abs/2002.03704, 2020. URL https://arxiv.org/abs/2002.03704.
[17]
Frick, J. and Hegg, C. Can end-users' flood management decision making be improved by information about forecast uncertainty? Atmospheric Research, 100(2-3):296-303, 2011.
[18]
Gelman, A. and Hill, J. Data analysis using regression and multilevel/hierarchical models. Cambridge University Press, 2006.
[19]
Ghosh, P., Sajjadi, M. S. M., Vergari, A., Black, M. J., and Schölkopf, B. From variational to deterministic autoencoders. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020. URL https://openreview.net/forum?id=S1g7tpEYDS.
[20]
Gigerenzer, G. and Brighton, H. Homo heuristicus: Why biased minds make better inferences. Topics in cognitive science, 1(1):107-143, 2009.
[21]
Gigerenzer, G., Gaissmaier, W., Kurz-Milcke, E., Schwartz, L. M., and Woloshin, S. Helping doctors and patients make sense of health statistics. Psychological science in the public interest, 8(2):53-96, 2007.
[22]
Gneiting, T. and Raftery, A. E. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):359-378, 2007.
[23]
Gong, W., Tschiatschek, S., Nowozin, S., Turner, R. E., Hernández-Lobato, J. M., and Zhang, C. Icebreaker: Element-wise efficient information acquisition with a bayesian deep latent gaussian model. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper_files/paper/2019/file/c055dcc749c2632fd4dd806301f05ba6-Paper.pdf.
[24]
Hullman, J., Qiao, X., Correll, M., Kale, A., and Kay, M. In pursuit of error: A survey of uncertainty visualization evaluation. IEEE transactions on visualization and computer graphics, 25(1):903-913, 2018.
[25]
Jain, S., Liu, G., Mueller, J., and Gifford, D. Maximizing overall diversity for improved uncertainty estimates in deep ensembles. Proceedings of the AAAI Conference on Artificial Intelligence, 34(04): 4264-4271, Apr. 2020. URL https://ojs.aaai.org/index.php/AAAI/article/view/5849.
[26]
Jalaian, B., Lee, M., and Russell, S. Uncertain context: Uncertainty quantification in machine learning. AI Magazine, 40(4):40-49, 2019.
[27]
Jiang, H., Kim, B., Guan, M. Y., and Gupta, M. To trust or not to trust a classifier. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS'18, pp. 5546-5557, Red Hook, NY, USA, 2018. Curran Associates Inc.
[28]
Joslyn, S. and LeClerc, J. Decisions with uncertainty: The glass half full. Current Directions in Psychological Science, 22(4):308-315, 2013.
[29]
Kendall, A. and Gal, Y. What uncertainties do we need in bayesian deep learning for computer vision? In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper_files/paper/2017/file/2650d6089a6d640c5e85b2b88265dc2b-Paper.pdf.
[30]
Lahlou, S., Jain, M., Nekoei, H., Butoi, V. I., Bertin, P., Rector-Brooks, J., Korablyov, M., and Bengio, Y. DEUP: Direct epistemic uncertainty prediction. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id=eGLdVRvvfQ.
[31]
Lai, V., Chen, C., Liao, Q. V., Smith-Renner, A., and Tan, C. Towards a science of human-ai decision making: a survey of empirical studies. arXiv preprint arXiv:2112.11471, 2021.
[32]
Lakshminarayanan, B., Pritzel, A., and Blundell, C. Simple and scalable predictive uncertainty estimation using deep ensembles. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS'17, pp. 6405-6416, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964.
[33]
Li, C., Stevens, A., Chen, C., Pu, Y., Gan, Z., and Carin, L. Learning weight uncertainty with stochastic gradient mcmc for shape classification. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 5666-5675. IEEE Computer [email protected], December 2016. ISBN 9781467388504. Generated from Scopus record by KAUST IRTS on 2021-02-09.
[34]
Liu, H., Ji, R., Li, J., Zhang, B., Gao, Y., Wu, Y., and Huang, F. Universal adversarial perturbation via prior driven uncertainty approximation. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, pp. 2941-2949. IEEE, 2019a.
[35]
Liu, J., Paisley, J., Kioumourtzoglou, M.-A., and Coull, B. Accurate uncertainty estimation and decomposition in ensemble learning. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019b. URL https://proceedings.neurips.cc/paper_files/paper/2019/file/1cc8a8ea51cd0adddf5dab504a285915-Paper.pdf.
[36]
Louizos, C. and Welling, M. Multiplicative normalizing flows for variational bayesian neural networks. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML'17, pp. 2218-2227. JMLR.org, 2017.
[37]
Marusich, L. R., Bakdash, J. Z., Onal, E., Yu, M. S., Schaffer, J., O'Donovan, J., Höllerer, T., Buchler, N., and Gonzalez, C. Effects of information availability on commandand-control decision making: Performance, trust, and situation awareness. Human Factors, 58(2):301-321, 2016. 26822796.
[38]
Morss, R. E., Demuth, J. L., and Lazo, J. K. Communicating uncertainty in weather forecasts: A survey of the us public. Weather and forecasting, 23(5):974-991, 2008.
[39]
Nadav-Greenberg, L. and Joslyn, S. L. Uncertainty forecasts improve decision making among nonexperts. Journal of Cognitive Engineering and Decision Making, 3(3):209-227, 2009.
[40]
Neal, R. M. Bayesian learning for neural networks, volume 118. Springer Science & Business Media, 2012.
[41]
Oberdiek, P., Fink, G. A., and Rottmann, M. UQGAN: A unified model for uncertainty quantification of deep classifiers trained via conditional GANs. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=djOANbV2zSu.
[42]
Prabhudesai, S., Yang, L., Asthana, S., Huan, X., Liao, Q. V., and Banovic, N. Understanding uncertainty: How lay decision-makers perceive and interpret uncertainty in human-ai decision making. In Proceedings of the 28th International Conference on Intelligent User Interfaces, IUI '23, pp. 379-396, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9798400701061.
[43]
Salakhutdinov, R. and Mnih, A. Bayesian probabilistic matrix factorization using markov chain monte carlo. In Proceedings of the 25th International Conference on Machine Learning, ICML '08, pp. 880-887, New York, NY, USA, 2008. Association for Computing Machinery. ISBN 9781605582054.
[44]
Salimans, T., Kingma, D., and Welling, M. Markov chain monte carlo and variational inference: Bridging the gap. In Bach, F. and Blei, D. (eds.), Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp. 1218-1226, Lille, France, 07-09 Jul 2015. PMLR. URL https://proceedings.mlr.press/v37/salimans15.html.
[45]
Schemmer, M., Hemmer, P., Nitsche, M., Kühl, N., and Vössing, M. A meta-analysis of the utility of explainable artificial intelligence in human-ai decision-making. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society, pp. 617-626, 2022.
[46]
Sensoy, M., Kaplan, L., and Kandemir, M. Evidential deep learning to quantify classification uncertainty. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS'18, pp. 3183-3193, Red Hook, NY, USA, 2018. Curran Associates Inc.
[47]
Subedar, M., Krishnan, R., Meyer, P. L., Tickoo, O., and Huang, J. Uncertainty-aware audiovisual activity recognition using deep bayesian variational inference. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019.
[48]
Vasconcelos, F., He, B., Singh, N. M., and Teh, Y. W. UncertaINR: Uncertainty quantification of end-to-end implicit neural representations for computed tomography. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id=jdGMBgYvfX.
[49]
Wang, G., Li, W., Aertsen, M., Deprest, J., Ourselin, S., and Vercauteren, T. Aleatoric uncertainty estimation with testtime augmentation for medical image segmentation with convolutional neural networks. NEUROCOMPUTING, 338:34-45, April 2019. ISSN 0925-2312.
[50]
Watson-Daniels, J., Parkes, D. C., and Ustun, B. Predictive multiplicity in probabilistic classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 10306-10314, 2023.
[51]
Wickelgren, W. A. Speed-accuracy tradeoff and information processing dynamics. Acta Psychologica, 41(1):67-85, 1977.
[52]
Wilson, A. G. and Izmailov, P. Bayesian deep learning and a probabilistic perspective of generalization. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS'20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.
[53]
Zhang, Y., Liao, Q. V., and Bellamy, R. K. Effect of confidence and explanation on accuracy and trust calibration in ai-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency, pp. 295-305, 2020.
Index Terms
- Using AI uncertainty quantification to improve human decision-making
Index terms have been assigned to the content through auto-classification.
Recommendations
Numerical approach for quantification of epistemic uncertainty
In the field of uncertainty quantification, uncertainty in the governing equations may assume two forms: aleatory uncertainty and epistemic uncertainty. Aleatory uncertainty can be characterised by known probability distributions whilst epistemic ...
2nd Workshop on Uncertainty Reasoning and Quantification in Decision Making
KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data MiningUncertainty reasoning and quantification play a critical role in decision making across various domains, prompting increased attention from both academia and industry. As real-world applications become more complex and data-driven, effectively handling ...
Decision-making under uncertainty: beyond probabilities: Challenges and perspectives
AbstractThis position paper reflects on the state-of-the-art in decision-making under uncertainty. A classical assumption is that probabilities can sufficiently capture all uncertainty in a system. In this paper, the focus is on the uncertainty that goes ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
July 2024
63010 pages
Copyright © 2024.
Publisher
JMLR.org
Publication History
Published: 21 July 2024
Qualifiers
- Research-article
- Research
- Refereed limited
Acceptance Rates
Overall Acceptance Rate 140 of 548 submissions, 26%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025