Every day, corporations are connecting the dots about our personal behaviorsilently scrutinizing clues left behind by our work habits and Internet use. The data compiled and portraits created are incredibly detailed, to the point of being invasive. But who connects the dots about what firms are doing with this information? The Black Box Society argues that we all need to be able to do soand to set limits on how big data affects our lives. Hidden algorithms can make (or ruin) reputations, decide the destiny of entrepreneurs, or even devastate an entire economy. Shrouded in secrecy and complexity, decisions at major Silicon Valley and Wall Street firms were long assumed to be neutral and technical. But leaks, whistleblowers, and legal disputes have shed new light on automated judgment. Self-serving and reckless behavior is surprisingly common, and easy to hide in code protected by legal and real secrecy. Even after billions of dollars of fines have been levied, underfunded regulators may have only scratched the surface of this troubling behavior. Frank Pasquale exposes how powerful interests abuse secrecy for profit and explains ways to rein them in. Demanding transparency is only the first step. An intelligible society would assure that key decisions of its most important firms are fair, nondiscriminatory, and open to criticism. Silicon Valley and Wall Street need to accept as much accountability as they impose on others.
Cited By
- Andalibi N, Pyle C, Barta K, Xian L, Jacobs A and Ackerman M Conceptualizing Algorithmic Stigmatization Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, (1-18)
- Davidson E, Wessel L, Winter J and Winter S (2023). Future directions for scholarship on data governance, digital innovation, and grand challenges, Information and Organization, 33:1, Online publication date: 1-Mar-2023.
- Kingsley S, Sinha P, Wang C, Eslami M and Hong J (2022). "Give Everybody [..] a Little Bit More Equity": Content Creator Perspectives and Responses to the Algorithmic Demonetization of Content Associated with Disadvantaged Groups, Proceedings of the ACM on Human-Computer Interaction, 6:CSCW2, (1-37), Online publication date: 7-Nov-2022.
- Vigna F Co-regulation Approach for Governing Big Data: Thoughts on Data Protection Law Proceedings of the 15th International Conference on Theory and Practice of Electronic Governance, (59-63)
- Gansky B and McDonald S CounterFAccTual: How FAccT Undermines Its Organizing Principles 2022 ACM Conference on Fairness, Accountability, and Transparency, (1982-1992)
- Susser D Decision Time: Normative Dimensions of Algorithmic Speed 2022 ACM Conference on Fairness, Accountability, and Transparency, (1410-1420)
- Fischer M, Hirsbrunner S, Jentner W, Miller M, Keim D and Helm P Promoting Ethical Awareness in Communication Analysis: Investigating Potentials and Limits of Visual Analytics for Intelligence Applications 2022 ACM Conference on Fairness, Accountability, and Transparency, (877-889)
- Norval C, Cornelius K, Cobbe J and Singh J Disclosure by Design: Designing information disclosures to support meaningful transparency and accountability 2022 ACM Conference on Fairness, Accountability, and Transparency, (679-690)
- Suryanarayana S, Sarne D and Kraus S Justifying Social-Choice Mechanism Outcome for Improving Participant Satisfaction Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, (1246-1255)
- Grill G and Andalibi N (2022). Attitudes and Folk Theories of Data Subjects on Transparency and Accuracy in Emotion Recognition, Proceedings of the ACM on Human-Computer Interaction, 6:CSCW1, (1-35), Online publication date: 30-Mar-2022.
- Kennedy J and Holcombe-James I (2022). “It’s almost impossible to buy a dumb TV” Experiences of automated decision-making and smart technologies in low-income homes, Telematics and Informatics, 68:C, Online publication date: 1-Mar-2022.
- Seidelin C, Moreau T, Shklovski I and Holten Møller N (2022). Auditing Risk Prediction of Long-Term Unemployment, Proceedings of the ACM on Human-Computer Interaction, 6:GROUP, (1-12), Online publication date: 14-Jan-2022.
- Ryoo J, Morris A and Margolis J (2021). “What Happens to the Raspado man in a Cash-free Society?”: Teaching and Learning Socially Responsible Computing, ACM Transactions on Computing Education, 21:4, (1-28), Online publication date: 31-Dec-2022.
- Cloete R, Norval C and Singh J (2021). Auditable Augmented/Mixed/Virtual Reality, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 5:4, (1-24), Online publication date: 27-Dec-2022.
- DeVito M (2021). Adaptive Folk Theorization as a Path to Algorithmic Literacy on Changing Platforms, Proceedings of the ACM on Human-Computer Interaction, 5:CSCW2, (1-38), Online publication date: 13-Oct-2021.
- Posada J, Weller N and Wong W We Haven't Gone Paperless Yet: Why the Printing Press Can Help Us Understand Data and AI Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, (864-872)
- Ochigame R and Ye K Search Atlas: Visualizing Divergent Search Results Across Geopolitical Borders Designing Interactive Systems Conference 2021, (1970-1983)
- Kaspersen M, Bilstrup K, Van Mechelen M, Hjorth A, Bouvin N and Petersen M VotestratesML: A High School Learning Tool for Exploring Machine Learning and its Societal Implications FabLearn Europe / MakeEd 2021 - An International Conference on Computing, Design and Making in Education, (1-10)
- Carter D, Acker A and Sholler D (2021). Investigative approaches to researching information technology companies, Journal of the Association for Information Science and Technology, 72:6, (655-666), Online publication date: 14-May-2021.
- Ariadne Muller D The Computer for the 21st Century – Second Edition for Europe Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, (1-11)
- Whitney C, Naval T, Quepons E, Singh S, Rick S and Irani L HCI Tactics for Politics from Below: Meeting the Challenges of Smart Cities Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, (1-15)
- Cobbe J, Lee M and Singh J Reviewable Automated Decision-Making Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, (598-609)
- Albert K and Delano M This Whole Thing Smacks of Gender Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, (342-352)
- Kaspersen M, Bilstrup K and Petersen M The Machine Learning Machine: A Tangible User Interface for Teaching Machine Learning Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction, (1-12)
- Shneiderman B (2020). Bridging the Gap Between Ethics and Practice, ACM Transactions on Interactive Intelligent Systems, 10:4, (1-31), Online publication date: 3-Dec-2020.
- Hatwell J, Gaber M and Azad R (2020). CHIRPS: Explaining random forest classification, Artificial Intelligence Review, 53:8, (5747-5788), Online publication date: 1-Dec-2020.
- de Fine Licht K and de Fine Licht J (2020). Artificial intelligence, transparency, and public decision-making, AI & Society, 35:4, (917-926), Online publication date: 1-Dec-2020.
- Cunha J and Aguiar Y Reflections on the role of nudges in human-computer interaction for behavior change Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems, (1-6)
- Udoh E Is the data fair? Proceedings of the 13th International Conference on Theory and Practice of Electronic Governance, (1-7)
- Rajtmajer S and Susser D Automated influence and the challenge of cognitive security Proceedings of the 7th Symposium on Hot Topics in the Science of Security, (1-9)
- Jones K, Rubel A and LeClere E (2019). A matter of trust, Journal of the Association for Information Science and Technology, 71:10, (1227-1241), Online publication date: 11-Sep-2020.
- Stark L, Stanhaus A and Anthony D (2020). “I Don't Want Someone to Watch Me While I'm Working”, Journal of the Association for Information Science and Technology, 71:9, (1074-1088), Online publication date: 6-Aug-2020.
- Bilstrup K, Kaspersen M and Petersen M Staging Reflections on Ethical Dilemmas in Machine Learning Proceedings of the 2020 ACM Designing Interactive Systems Conference, (1211-1222)
- Robbins S (2019). AI and the path to envelopment: knowledge as a first step towards the responsible regulation and use of AI-powered machines, AI & Society, 35:2, (391-400), Online publication date: 1-Jun-2020.
- Carabantes M (2019). Black-box artificial intelligence: an epistemological and critical analysis, AI & Society, 35:2, (309-317), Online publication date: 1-Jun-2020.
- Hagendorff T and Wezel K (2019). 15 challenges for AI: or what AI (currently) can’t do, AI & Society, 35:2, (355-365), Online publication date: 1-Jun-2020.
- Poulain R and Tarissan F (2020). Investigating the lack of diversity in user behavior, Information Processing and Management: an International Journal, 57:2, Online publication date: 1-Mar-2020.
- Hagendorff T (2020). The Ethics of AI Ethics: An Evaluation of Guidelines, Minds and Machines, 30:1, (99-120), Online publication date: 1-Mar-2020.
- Awad E, Dsouza S, Bonnefon J, Shariff A and Rahwan I (2020). Crowdsourcing moral machines, Communications of the ACM, 63:3, (48-55), Online publication date: 24-Feb-2020.
- Robbins S (2019). A Misdirected Principle with a Catch: Explicability for AI, Minds and Machines, 29:4, (495-514), Online publication date: 1-Dec-2019.
- Elsden C, Symons K, Bunduchi R, Speed C and Vines J (2019). Sorting Out Valuation in the Charity Shop, Proceedings of the ACM on Human-Computer Interaction, 3:CSCW, (1-25), Online publication date: 7-Nov-2019.
- Pinter A, Jiang J, Gach K, Sidwell M, Dykes J and Brubaker J (2019). "Am I Never Going to Be Free of All This Crap?", Proceedings of the ACM on Human-Computer Interaction, 3:CSCW, (1-23), Online publication date: 7-Nov-2019.
- Frey S, Krafft P and Keegan B (2019). "This Place Does What It Was Built For", Proceedings of the ACM on Human-Computer Interaction, 3:CSCW, (1-31), Online publication date: 7-Nov-2019.
- Guidotti R, Monreale A, Ruggieri S, Turini F, Giannotti F and Pedreschi D (2018). A Survey of Methods for Explaining Black Box Models, ACM Computing Surveys, 51:5, (1-42), Online publication date: 30-Sep-2019.
- Watson D (2019). The Rhetoric and Reality of Anthropomorphism in Artificial Intelligence, Minds and Machines, 29:3, (417-440), Online publication date: 1-Sep-2019.
- Seo H, Xiong A and Lee D Trust It or Not Proceedings of the 10th ACM Conference on Web Science, (265-274)
- J. Domanski R The A.I. Pandorica Proceedings of the 20th Annual International Conference on Digital Government Research, (409-416)
- Pham A and Castro C (2019). The moral limits of the market, Ethics and Information Technology, 21:2, (117-126), Online publication date: 1-Jun-2019.
- Marciano A (2019). Reframing biometric surveillance, Ethics and Information Technology, 21:2, (127-136), Online publication date: 1-Jun-2019.
- Eslami M, Vaccaro K, Lee M, Elazari Bar On A, Gilbert E and Karahalios K User Attitudes towards Algorithmic Opacity and Transparency in Online Reviewing Platforms Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, (1-14)
- Geoffroy-Terryn L Regulating 'Social Scores' in Data-Driven Societies Proceedings of the 12th International Conference on Theory and Practice of Electronic Governance, (516-518)
- Benthall S Situated information flow theory Proceedings of the 6th Annual Symposium on Hot Topics in the Science of Security, (1-10)
- Mittelstadt B, Russell C and Wachter S Explaining Explanations in AI Proceedings of the Conference on Fairness, Accountability, and Transparency, (279-288)
- Passi S and Barocas S Problem Formulation and Fairness Proceedings of the Conference on Fairness, Accountability, and Transparency, (39-48)
- Susser D Invisible Influence Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, (403-408)
- Castillo C (2019). Fairness and Transparency in Ranking, ACM SIGIR Forum, 52:2, (64-71), Online publication date: 17-Jan-2019.
- Passi S and Jackson S (2018). Trust in Data Science, Proceedings of the ACM on Human-Computer Interaction, 2:CSCW, (1-28), Online publication date: 1-Nov-2018.
- Khalil M, Prinsloo P and Slade S The unbearable lightness of consent Proceedings of the Fifth Annual ACM Conference on Learning at Scale, (1-11)
- Pierce J, Fox S, Merrill N, Wong R and DiSalvo C An Interface without A User Proceedings of the 2018 Designing Interactive Systems Conference, (1345-1358)
- Ramadan Q, Ahmadian A, Strüber D, Jürjens J and Staab S Model-based discrimination analysis Proceedings of the International Workshop on Software Fairness, (22-28)
- Skirpan M, Cameron J and Yeh T More Than a Show Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, (1-13)
- Elsden C, Manohar A, Briggs J, Harding M, Speed C and Vines J Making Sense of Blockchain Applications Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, (1-14)
- Eslami M, Krishna Kumaran S, Sandvig C and Karahalios K Communicating Algorithmic Process in Online Behavioral Advertising Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, (1-13)
- Diaz M, Johnson I, Lazar A, Piper A and Gergle D Addressing Age-Related Bias in Sentiment Analysis Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, (1-14)
- DeVito M, Birnholtz J, Hancock J, French M and Liu S How People Form Folk Theories of Social Media Feeds and What it Means for How We Study Self-Presentation Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, (1-12)
- Rader E, Cotter K and Cho J Explanations as Mechanisms for Supporting Algorithmic Transparency Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, (1-13)
- Shackell C and Sitbon L Cognitive Externalities and HCI Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, (1-10)
- DeVito M, Hancock J, French M, Birnholtz J, Antin J, Karahalios K, Tong S and Shklovski I The Algorithm and the User Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, (1-6)
- Robertson R, Lazer D and Wilson C Auditing the Personalization and Composition of Politically-Related Search Engine Results Pages Proceedings of the 2018 World Wide Web Conference, (955-965)
- Epstein R, Robertson R, Lazer D and Wilson C (2017). Suppressing the Search Engine Manipulation Effect (SEME), Proceedings of the ACM on Human-Computer Interaction, 1:CSCW, (1-22), Online publication date: 6-Dec-2017.
- Obar J and Oeldorf-Hirsch A Clickwrap Impact Proceedings of the 8th International Conference on Social Media & Society, (1-5)
- Karanasiou A and Pinotsis D Towards a legal definition of machine intelligence Proceedings of the 16th edition of the International Conference on Articial Intelligence and Law, (119-128)
- Brkan M AI-supported decision-making under the general data protection regulation Proceedings of the 16th edition of the International Conference on Articial Intelligence and Law, (3-8)
- Lee M and Baykal S Algorithmic Mediation in Group Decisions Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, (1035-1048)
- Paraschakis D Recommender Systems from an Industrial and Ethical Perspective Proceedings of the 10th ACM Conference on Recommender Systems, (463-466)
- Doneda D and Almeida V (2016). What Is Algorithm Governance?, IEEE Internet Computing, 20:4, (60-63), Online publication date: 1-Jul-2016.
- Frizzo-Barker J, Chow-White P, Mozafari M and Ha D (2016). An empirical study of the rise of big data in business scholarship, International Journal of Information Management: The Journal for Information Professionals, 36:3, (403-413), Online publication date: 1-Jun-2016.
- Arnold T and Scheutz M (2016). Against the moral Turing test, Ethics and Information Technology, 18:2, (103-115), Online publication date: 1-Jun-2016.
- Lustig C, Pine K, Nardi B, Irani L, Lee M, Nafus D and Sandvig C Algorithmic Authority Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, (1057-1062)
- Anthes G (2016). Search engine agendas, Communications of the ACM, 59:4, (19-21), Online publication date: 23-Mar-2016.
Index Terms
- The Black Box Society: The Secret Algorithms That Control Money and Information