Abstract
AI systems are bringing an augmentation of human capabilities to shape the world. They may also drag a replacement of human conscience in large chunks of life. AI systems can be designed to leave moral control in human hands, to obstruct or diminish that moral control, or even to prevent it, replacing human morality with pre-packaged or developed ‘solutions’ by the ‘intelligent’ machine itself. Artificial Intelligent systems (AIS) are increasingly being used in multiple applications and receiving more attention from the public and private organisations. The purpose of this article is to offer a mapping of the technological architectures that support AIS, under the specific focus of the moral agency. Through a literature research and reflection process, the following areas are covered: a brief introduction and review of the literature on the topic of moral agency; an analysis using the BDI logic model (Bratman 1987); an elemental review of artificial ‘reasoning’ architectures in AIS; the influence of the data input and the data quality; AI systems’ positioning in decision support and decision making scenarios; and finally, some conclusions are offered about regarding the potential loss of moral control by humans due to AIS. This article contributes to the field of Ethics and Artificial Intelligence by providing a discussion for developers and researchers to understand how and under what circumstances the ‘human subject’ may, totally or partially, lose moral control and ownership over AI technologies. The topic is relevant because AIS often are not single machines but complex networks of machines that feed information and decisions into each other and to human operators. The detailed traceability of input-process-output at each node of the network is essential for it to remain within the field of moral agency. Moral agency is then at the basis of our system of legal responsibility, and social approval is unlikely to be obtained for entrusting important functions to complex systems under which no moral agency can be identified.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
We keep here the word 'Desires' because it was used by Bratman and it continues being used in the related literature. However, it has a psychological-Humean taste that does not seem necessary. Maybe better than 'Desires' we should speak of 'Purposes', with the same logical content and less psychological charge.
See for example Faucher and Roques (2018).
We shall not enter in the much discussed issue of moral and legal responsibilities of AI-endowed systems. A good summary of both issues can be found in Chinen (2019).
We use the word 'decision' here to group together what Bratman (1987) calls 'intentions' and 'plans'.
This author was the first in adopting this terminology. It may also be the first paper to propose common sense reasoning ability as the key to AI.
The contrary is also frequent: A machine under the regular control by a human operator, that passes to an automatic system in case of catastrophic failure of that human operator (for example: in case that the driver of a car becomes distracted or asleep and the car threatens to leave the road). This are rarely AI systems: they don't have time/experience enough to 'learn' from their own performance. They are rather emergency, fully programmed mechanisms.
Abbreviations
- AAN:
-
Artificial neural networks
- AI:
-
Artificial intelligence
- AIS:
-
Artificial intelligence Systems
- AS:
-
Autonomous systems
- DSS:
-
Decision support systems
- GAN:
-
Generative adversarial networks
- ML:
-
Machine learning
- MTT:
-
Moral turing test
- RL:
-
Reinforcement learning
- SAS:
-
Semi-autonomous systems
References
Anderson M, Anderson SL (2014) GenEth: a general ethical dilemma analyser. In Twenty-Eighth AAAI Conference on Artificial Intelligence.
Aristotle, Crisp R (2000) Nicomachean ethics. Cambridge University Press, Cambridge
Arnold T, Scheutz M (2016) Against the moral Turing test: accountable design and the moral reasoning of autonomous systems. Ethics Inf Technol 18(2):103–115
Arvan M (2018). Mental time-travel, semantic flexibility, and AI ethics. AI & SOCIETY, pp. 1–20
Autili M, Di Ruscio D, Inverardi P, Pelliccione P, Tivoli M (2019) A software exoskeleton to protect and support citizen’s ethics and privacy in the digital world. IEEE Access 7:62011–62021
Balakrishnan A, Bouneffouf D, Mattei N, & Rossi F (2018) Using contextual bandits with behavioral constraints for constrained online movie recommendation. In IJCAI (pp. 5802–5804)
Barredo AA et al. (2019) Explainable artificial intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. arXiv preprint arXiv:1910.10045
Bauer WA (2018) Virtuous vs utilitarian artificial moral agents. AI & SOCIETY, pp. 1–9
Boddington P (2017) Towards a code of ethics for artificial intelligence. Springer, Cham
Bowles C (2018) Future ethics. NowNext Press, East Sussex
Boyd D, Crawford K (2012) Critical questions for Big Data. Inform Commun Soc 15(5):662–679. https://doi.org/10.1080/1369118X.2012.678878
Bratman M (1987) Intention, plans, and practical reason. Harvard University Press, Cambridge
Broussard M (2018) Artificial unintelligence: how computers misunderstand the world. MIT Press, Cambridge
Caliskan Aylin, Bryson Joanna J, Narayanan Arvind (2017) Semantics derived automatically from language corpora contain human-like biases. Science 356(6334):183–186
Carter SM, Mayes C, Eagle L, Dahl S (2017) A code of ethics for social marketing? Bridging procedural ethics and ethics-in-practice. J Nonprofit Public Sect Mark 29(1):20–38
Charisi V, Dennis L, Fisher M, Lieck R, Matthias A, Slavkovik M, Sombetzki J, Winfield AF, Yampolskiy R (2017). Towards moral autonomous systems. arXiv preprint arXiv:1703.04741
Chinen M (2019) Law and autonomous machines: the co-evolution of legal responsibility and technology, UK. Edward Elgar Publishing, Cheltenham
Ekbia H, Mattioli M, Kouper I, Arave G, Ghazinejad A, Bowman T, Suri VR, Tsou A, Weingart S, Sugimoto CR (2015) Big data, bigger dilemmas: a critical review. J Assn Inf Sci Tech 66:1523–1545. https://doi.org/10.1002/asi.23294
Faucher N, Roques M (2018) The ontology, psychology and axiology of habits (habitus) in medieval philosophy. Springer, New York
Floridi L, Sanders JW (2004) On the morality of artificial agents. Mind Mach 14(3):349–379
Friedler SA, Scheidegger C, Venkatasubramanian S, Choudhary S, Hamilton EP, Roth D (2018) A comparative study of fairness-enhancing interventions in machine learning. arXiv preprint arXiv:1802.04422
Gelernter David Hillel (1992) Mirror worlds. Oxford University Press, Oxford
Gerdes A, Øhrstrøm P (2015) Issues in robot ethics seen through the lens of a moral Turing test. J Inform Commun Ethics Soc
Gibson S (2019) Arguing, obeying and defying: a rhetorical perspective on Stanley Milgram’s obedience experiments. Cambridge University Press, New York
Greene D, Hoffmann AL, Stark L (2019) Better, nicer, clearer, fairer: a critical assessment of the movement for ethical artificial intelligence and machine learning. In Proceedings of the 52nd Hawaii International Conference on System Sciences
Hacker P (2018) Teaching fairness to artificial intelligence: existing and novel strategies against algorithmic discrimination under EU law. Common Market Law Review 55(4):1143–1185
Hester PT, Adams KM (2017) Systemic decision-making fundamentals for addressing problems and messes. Springer, New York
AI HLEG, High-Level Expert Group on Artificial Intelligence (2018) Draft ethics guidelines for trustworthy AI. https://ec.europa.eu/newsroom/dae/document.cfm?doc_id=57112
Jobin A, Ienca M, Vayena E (2019) The global landscape of AI ethics guidelines. Nat Mach Intell 1(9):389–399
Kahneman D (2011) Thinking, fast and slow. Farrar, Straus and Giroux, New York
Loreggia A, Mattei N, Rossi F, Venable KB (2018) Preferences and ethical principles in decision making. In 2018 AAAI Spring Symposium Series
Markham AN, Tiidenberg K, Herman A (2018) Ethics as methods: doing ethics in the Era of big data research—introduction. Soc Media Soc. https://doi.org/10.1177/2056305118784502
McCarthy (1958). “Programs with common sense.” Proceedings of Teddington Conference on the mechanisation of thought processes
McQuillan D (2018) People’s councils for ethical machine learning. Soc Media Soc 4(2):2056305118768303
Metcalf J, Emily FK, Danah B (2019) Perspectives on big data, ethics, and society. Council for big data, ethics, and society. https://bdes.datasociety.net/council-output/perspectives-on-big-data-ethics-and-society/
Meyer, John-Jules CH, Broersen Jan, Herzig Andreas (2015) BDI logics. In: Ditmarsch HV (ed) Handbook of epistemic logic. College Publications, London
Mingers J, Walsham G (2010) Toward ethical information systems: the contribution of discourse ethics. Mis Quart 34(4):833–854
Mnich M (2018) Big data algorithms beyond machine learning. KI-Künstliche Intelligenz 32(1):9–17
Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, Petersen S, Beattie C, Sadik A, Antonoglou I, King H, Kumaran D, Wierstra D, Legg S, Hassabis D (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533
Moor JH (2006) The nature, importance, and difficulty of machine ethics. IEEE Intell Syst 21(4):18–21
Nath R, Sahu V (2017) The problem of machine ethics in artificial intelligence. AI Soc 35(1):103–111
Plato et al (2018) The republic. Cambridge University Press, Cambridge
AI NOW Report (2018) Artificial Intelligence Institute. New York
Rossi F, Mattei N (2019). Building ethically bounded AI. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 33, pp. 9785–9789)
Russell SJ, Norvig P, Davis E (2010) Artificial intelligence: a modern approach, 3rd edn. Prentice Hall, Upper Saddle River
Salgues B (2018) Society 5.0. Wiley, Hoboken
Shortliffe EH (1976) Computer-based medical consultations: MYCIN. Elsevier, Amsterdam
Silver N (2012) The signal and the noise: why so many predictions fail–but some don’t. Penguin Press, New York
Smith G (2018) The AI delusion. Oxford University Press, Oxford
Sutton RS, Barto AG (2018) Reinforcement learning: an introduction, 2nd edn. MIT Press, Cambridge
Tiberius V (2015) Moral psychology: a contemporary introduction. Taylor & Francis Group, New York
Tomasello M (2018) Precís of a natural history of human morality. Philos Psychol 31(5):661–668
Torrance S (2013) Artificial agents and the expanding ethical circle. AI & Soc 28(4):399–414
Vakkuri V, Abrahamsson P (2018) The key concepts of ethics of artificial intelligence. In 2018 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC) (pp. 1–6). IEEE
Wallach W (2008) Implementing moral decision making faculties in computers and robots. AI & Soc 22(4):463–475
Wallach W, Allen C (2009) Moral machines: teaching robots right from wrong. Oxford University Press, Oxford; New York
Wallach W, Allen C (2012) Hard problems: framing the chinese room in which a robot takes a moral turing test. University of Birmingham, AISB/IACAP, p 5
Winfield AF, Jirotka M (2018) Ethical governance is essential to building trust in robotics and artificial intelligence systems. Philos Trans R Soc A 376(2133):20180085
Winfield AF, Michael K, Pitt J, Evers V (2019) Machine ethics: the design and governance of ethical AI and autonomous systems. Proc IEEE 107(3):509–517
Wu YH, Lin SD (2018) A low-cost ethics shaping approach for designing reinforcement learning agents. In Thirty-Second AAAI Conference on Artificial Intelligence
Yu H, Shen Z, Miao C, Leung C, Lesser VR, Yang Q (2018) Building ethics into artificial intelligence. arXiv preprint arXiv:1812.02953
Zilberstein S (2015) Building strong semi-autonomous systems. Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence
Zubiri X (1986) Sobre el hombre. Alianza, Madrid
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Gonzalez Fabre, R., Camacho Ibáñez, J. & Tejedor Escobar, P. Moral control and ownership in AI systems. AI & Soc 36, 289–303 (2021). https://doi.org/10.1007/s00146-020-01020-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00146-020-01020-z