Abstract
Recent advances in autonomous vehicles promise to revolutionize the transportation system. This perspective has led to new research on a number of open questions, such as how the self-driving system should behave in unavoidable crash situations. Our study aims to contribute to this investigation. In most ongoing research, this question is presented as a moral dilemma, drawing on established research on the trolley dilemma. However, more recent studies have shifted the focus from morality to risk analysis. We investigated the dual contribution of moral judgment and risk analysis in subjects facing dangerous situations. To this end, we use virtual reality to recreate a driving situation that allows subjects to immerse themselves in the road environment. Our results show a strong dissociation between quick decisions, when risk analysis seems to suggest the best choice, and conscious decisions, when proper moral judgment is at stake.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Alaieri F, Vellino A (2016) Ethical decision making in robots: Autonomy, trust and responsibility. In: Agah A, Cabibihan JJ, Howard AM, Salichs MA, He H (eds) Social robotics. Springer, Cham, pp 159–168
Allen C, Smit I, Wallach W (2005) Artificial morality: top-down, bottom-up, and hybrid approaches. Ethics Inf Technol 7(3):149–155
Appiah KA (2008) Experiments in ethics. Harvard University Press, Cambridge
Awad E, Dsouza S, Kim R, Schulz J, Henrich J, Shariff A, Rahwan I (2018) The moral machine experiment. Nature, 563(7729):59–64
Bartels DM (2008) Principled moral sentiment and the flexibility of moral judgment and decision making. Cognition 108:381–417
Bengtsson, P. (2018). Attuning the pedestrian-vehicle and driver-vehicle: why attributing a mind to a vehicle matters. In: W. Karwowski, T. Ahram (Eds) International conference on intelligent human systems integration. Springer, Cham, pp 19–22
Bergmann LT, Schlicht L, Meixner C, König P, Pipa G, Boshammer S, Stephan A (2018) Autonomous vehicles require socio-political acceptance an empirical and philosophical perspective on the problem of moral decision making. Front Behav Neurosci 12:31
Bjorndahl A, London AJ, Zollman KJ (2017) Kantian decision making under uncertainty: dignity, price, and consistency. Philosopher's Imprint 17(7)
Bojarski M, Testa DD, Dworakowski D, Firner B, Flepp B, Goyal P, Zhao J (2016) End-to-end learning for self-driving cars. arXiv abs/1604.07316
Bojarski M, YeresP, Choromanaska A, Choromanski K, Firner B, Jackel L, Muller U (2017) Explaining how a deep neural network trained with end-to-end learning steers a car. arXivabs/1704.07911
Bonnefon JF, Shariff A, Rahwan I (2016) The social dilemma of autonomous vehicles. Science 352(6293):1573–1576
Chen LC, Schwing A, Yuille A, Urtasun R (2015) Learning deep structured models. In International Conference on Machine Learning (pp. 1785–1794). PMLR
Crisman JD, Thorpe CE (1993) SCARF: a color vision system that tracks roads and intersections. IEEE Trans Robot Autom 9:49–58
Davnall R (2019) Solving the single-vehicle self-driving car trolley problem using risk theory and vehicle dynamics. Sci Eng Ethics 26:1–19
Edmonds D (2014) Would you kill the fat man? The trolley problem and what your answer tells us about right and wrong. Princeton University Press, Princeton
Faulhaber AK, Dittmer A, Blind F, Wächter MA, Timm S, Sütfeld LR, König P (2019) Human decisions in moral dilemmas are largely described by utilitarianism: virtual car driving study provides guidelines for autonomous driving vehicles. Sci Eng Ethics 25(2):399–418
Fayyad J, Jaradat MA, Gruyer D, Najjaran H (2020) Deep learning sensor fusion for autonomous vehicle perception and localization: a review. Sensors 20(15):4220
Foot P (1967) The problem of abortion. Oxford Rev 5:5–15
Frank DA, Chrysochou P, Mitkidis P, Ariely D (2019) Human decision-making biases in the moral dilemmas of autonomous vehicles. Sci Rep 9(1):1–19
Gold N, Pulford BD, Colman AM (2014) The outlandish, the realistic, and the real: contextual manipulation and agent role effects in trolley problems. Front Psychol 5:35
Grasso GM, Lucifora C, Perconti P, Plebe A (2020) Integrating human acceptable morality in autonomous vehicles. In: Ahram T, Karwowski W, Vergnano A, Leali F, Taiar R (eds) Intelligent human systems integration 2020. IHSI 2020. Advances in intelligent systems and computing. Springer, Cham. https://doi.org/10.1007/978-3-030-39512-4_7
Greene JD, Sommerville RB, Nystrom LE, Darley JM, Cohen JD (2001) fMRI investigation of emotional engagement in moral judgment. Science 293:2105–2108
Greene JD, Nystrom LE, Engell AD, Darley JM, Cohen JD (2004) The neural bases of cognitive conflict and control in moral judgment. Neuron 44(2):389–400
Gross D, Ehlers W, Wriggers P, Schröder J, Müller R (2017) Statics-formulas and problems. Springer, Berlin
Hansson SO (2012) A panorama of the philosophy of risk. In Handbook of risk theory. Springer, Dordrecht, pp 27–54
Himmelreich J (2018) Never mind the trolley: The ethics of autonomous vehicles in mundane situations. In: «Ethical Theory & Moral Practice» vol. XXI, n. 3:669–684
Jorge CC, Rossetti RJF (2018) On social interactions and the emergence of autonomous vehicles. In International conference on vehicle technology and intelligent transport systems (pp. 423–430)
Kagan S (1989) The limits of morality
Kahneman D (2011) Thinking, fast and slow, Macmillan, Basingstoke
Kallioinen N, Pershina M, Zeiser J, Nosrat Nezami F, Pipa G, Stephan A, König P (2019) Moral judgements on the actions of self-driving carsand human drivers in dilemma situations from different perspectives. Front Psychol 10:2415
Kant I (1785) The categorical imperative
Keeling G (2020) Why trolley problems mater for the ethics of automated vehicles. In: «Science and Engineering Ethics», vol. XXVI, n. 1:293–307
Kastrinaki V, Zervakis M, Kalaitzakis K (2003) A survey of video processing techniques for traffic applications. Image Vis Comput 21:359–381
Koenings M, Young L, Adolphs R, Tranel D, Cushman F, Hauser M, Damasio A (2007) Damage to the prefrontal cortex increases utilitarian moral judgments. Nature 446:908–911
Kuutti S, Fallah S, Bowden R, Barber P (2019) Deep learning for autonomous vehicle control—algorithms, state-of-the-art, and future prospects. Synth Lect Adv Automot Technol 3:1–80
LaViola JJ Jr (2000) A discussion of cybersickness in virtual environments. ACM Sigchi Bull 32(1):47–56
Li S, Zhang J, Li P, Wang Y, Wang Q (2019) Influencing factors of driving decision-making under the moral dilemma. IEEE Access 7:104132–104142
Lieberman MD (2007) Social cognitive neuroscience: a review of core processes. Annu Rev Psychol 58:259–289
Lucifora C, Grasso GM, Perconti P, Plebe A (2020) Moral dilemmas in self-driving cars. Rivista internazionale di Filosofia e Psicologia 11(2):238–250
Marchand R (1992) The designer go to the fair II: Norman Bel Geddes, the general motors “futurama” and the visit to the factory transformed. Design Issues 8(2):23–40
Masaki I (ed) (1992) Vision-based vehicle guidance. Springer-Verlag, Berlin
McCall JC, Trivedi MM (2006) Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation. IEEE Trans Intell Transp Syst 7:20–37
McCubbins MD, Turner M (2012) Going cognitive: tools for rebuilding the social sciences. In: Sun R (ed) Grounding social sciences in cognitive sciences. MIT Press, Cambridge, pp 387–414
Meder B, Fleischhut N, Krumnau NC, Waldmann MR (2019) How should autonomous cars drive? A preference for defaults in moral judgments under risk and uncertainty. Risk Anal 39(2):295–314
Mikhail J (2007) Universal moral grammar: theory, evidence and the future. Trends Cogn Sci 11:143–152
Nyholm S, Smids J (2016) The ethics of accident-algorithms for self-driving cars: An applied trolley problem?. In: «Ethical Theory & Moral Practice» vol. XIX, n. 5:1275–1289
Ohn-Bar E, Trivedi MM (2016) Looking at humans in the age of self-driving and highly automated vehicles. IEEE Trans Intell Transp Syst 29:90–104
Patil I, Cogoni C, Zangrando N, Chittaro L, Silani G (2014) Affective basis of judgment-behavior discrepancy in virtual experiences of moral dilemmas. Soc Neurosci 9(1):94–107
Pomerleau, D. A. (1989). ALVINN: An autonomous land vehicle in a neural network. In Ieee intelligent vehicles symposium (pp. 305–313)
Rhim J, Lee GB, Lee JH (2020) Human moral reasoning types in autonomous vehicle moral dilemma: a cross-cultural comparison of Korea andCanada. Comput Hum Behav 102:39–56
Riaz F, Niazi M (2017) Towards social autonomous vehicles: efficient collision avoidance scheme using Richardson’s arms race model. PLoS ONE 12:e0186103
Richards DC (2010) Relationship between speed and risk of fatal injury: pedestrians and car occupants. Department for Transport, London
Rumiati RI, Humphreys GW (2015) Cognitive neuroscience goes social. Cortex 70:1–4
Samuel S, Yahoodik S, Yamani Y, Valluru K, Fisher DL (2020) Ethical decision making behind the wheel–a driving simulator study. Transport Res Interdisip Perspect 5:100147
Schwarting W, Pierson A, Alonso-Mora J, Karaman S, Rus D (2019) Social behavior for autonomous vehicles. Proc Nat Acad Sci USA 116:24972–24978
Stanovich KE, West RF (2000) Individual differences in reasoning: implications for the rationality debate? Behav Brain Sci 23:645–726
Thomson JJ (1976) Killing, let die, and the trolley problem. Monist An Int Quart J General Philisophical Inquiry 59:204–217
Thomson JJ (1985) The trolley problem. Yale Law J 94:1395–1415
Turner M (2001) Cognitive dimensions of social science. Oxford University Press, Oxford
Vanderhaegen F (2016) A rule-based support system for dissonance discovery and control applied to car driving. Expert Syst Appl 65:361–371
Vanderhaegen F (2021) Weak signal-oriented investigation of ethical dissonance applied to unsuccessful mobility experiences linked to human-machine interactions. Sci Eng Ethics 27(1):1–25
VanRullen R (2017) Perception science in the age of deep neural networks. Front Psychol 8:142
Winfield AF, Blum C, Liu W (2014) Towards an ethical robot: internal models, consequences and ethical action selection. In: Mistry M, Leonardis A, Witkowski M, Melhuish C (eds) Advances in autonomous robotic systems, TAROS 2014. Lecture Notes in Computer Science. Springer, Cham, pp. 85–96. https://doi.org/10.1007/978-3-319-10401-08
Wilson H, Theodorou A, Bryson JJ (2019). Slam the Brakes: Perceptions of Moral Decisions in Driving Dilemmas. In AISafety@ IJCAI
Wood A (2013) Humanity as end in itself. In: Scheffler S (ed) On what matters, vol 2. Oxford University Press, Oxford, pp 58–82
Yokoi R, Nakayachi K (2020) Trust in autonomous cars: exploring the role of shared moral values, reasoning, and emotion in safety-critical decisions. Hum Factors. https://doi.org/10.1177/0018720820933041
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lucifora, C., Grasso, G.M., Perconti, P. et al. Moral reasoning and automatic risk reaction during driving. Cogn Tech Work 23, 705–713 (2021). https://doi.org/10.1007/s10111-021-00675-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10111-021-00675-y