research-article

Counterfactual explanations and how to find them: literature review and benchmarking

Author:

Riccardo GuidottiAuthors Info & Claims

Data Mining and Knowledge Discovery, Volume 38, Issue 5

Pages 2770 - 2824

https://doi.org/10.1007/s10618-022-00831-6

Published: 28 April 2022 Publication History

Abstract

Interpretable machine learning aims at unveiling the reasons behind predictions returned by uninterpretable classifiers. One of the most valuable types of explanation consists of counterfactuals. A counterfactual explanation reveals what should have been different in an instance to observe a diverse outcome. For instance, a bank customer asks for a loan that is rejected. The counterfactual explanation consists of what should have been different for the customer in order to have the loan accepted. Recently, there has been an explosion of proposals for counterfactual explainers. The aim of this work is to survey the most recent explainers returning counterfactual explanations. We categorize explainers based on the approach adopted to return the counterfactuals, and we label them according to characteristics of the method and properties of the counterfactuals returned. In addition, we visually compare the explanations, and we report quantitative benchmarking assessing minimality, actionability, stability, diversity, discriminative power, and running time. The results make evident that the current state of the art does not provide a counterfactual explainer able to guarantee all these properties simultaneously.

References

[1]

Aamodt A and Plaza E Case-based reasoning: foundational issues, methodological variations, and system approaches AI Commun 1994 7 1 39-59

[2]

Adadi A et al. Peeking inside the black-box: a survey on explainable artificial intelligence (XAI) IEEE Access 2018 6 52138-52160

[3]

Aggarwal CC, Chen C, and Han J The inverse classification problem J Comput Sci Technol 2010 25 3 458-468

Digital Library

[4]

Anjomshoae S, Najjar A, Calvaresi D, Främling K (2019) Explainable agents and robots: results from a systematic literature review. In: Proceedings of the 18th international conference on autonomous agents and multiagent systems, AAMAS’19, Montreal, QC, Canada, May 13–17, 2019, International Foundation for Autonomous Agents and Multiagent Systems, pp 1078–1088

[5]

Arrieta AB, Ser JD (2020) Plausible counterfactuals: auditing deep learning classifiers with realistic adversarial examples. In: 2020 International joint conference on neural networks, IJCNN 2020, Glasgow, United Kingdom, July 19–24, 2020, IEEE, pp 1–7

[6]

Arrieta AB, Rodríguez ND, Ser JD, Bennetot A, Tabik S, Barbado A, García S, Gil-Lopez S, Molina D, Benjamins R, Chatila R, and Herrera F Explainable artificial intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI Inf Fusion 2020 58 82-115

Digital Library

[7]

Artelt A (2019) Ceml: counterfactuals for explaining machine learning models—a python toolbox. https://www.github.com/andreArtelt/ceml

[8]

Artelt A, Hammer B (2019) On the computation of counterfactual explanations—a survey. CoRR arXiv:1911.07749

[9]

Artelt A, Hammer B (2020a) Convex density constraints for computing plausible counterfactual explanations. In: Artificial neural networks and machine learning—ICANN 2020—29th international conference on artificial neural networks, Bratislava, Slovakia, September 15–18, 2020, Proceedings, Part I, Springer, Lecture notes in computer science, vol 12396, pp 353–365

[10]

Artelt A, Hammer B (2020b) Efficient computation of counterfactual explanations of LVQ models. In: 28th European symposium on artificial neural networks, computational intelligence and machine learning, ESANN 2020, Bruges, Belgium, October 2–4, 2020, pp 19–24

[11]

Artelt A, Vaquet V, Velioglu R, Hinder F, Brinkrolf J, Schilling M, Hammer B (2021) Evaluating robustness of counterfactual explanations. CoRR arXiv:2103.02354

[12]

Ates E, Aksar B, Leung VJ, Coskun AK (2021) Counterfactual explanations for machine learning on multivariate time series data. In: 2021 international conference on applied artificial intelligence (ICAPAI), IEEE, pp 1–8

[13]

Balasubramanian R, Sharpe S, Barr B, Wittenbach JD, Bruss CB (2020) Latent-cf: a simple baseline for reverse counterfactual explanations. CoRR arXiv:2012.09301

[14]

Ballet V, Renard X, Aigrain J, Laugel T, Frossard P, Detyniecki M (2019) Imperceptible adversarial attacks on tabular data. CoRR arXiv:1911.03274

[15]

Barbaglia L, Manzan S, Tosetti E (2020) Forecasting loan default in Europe with machine learning. Available at SSRN 3605449

[16]

Beck A and Teboulle M A fast iterative shrinkage-thresholding algorithm for linear inverse problems SIAM J Imaging Sci 2009 2 1 183-202

Digital Library

[17]

Beck SR, Riggs KJ, and Gorniak SL Relating developments in children’s counterfactual thinking and executive functions Think Reason 2009 15 4 337-354

[18]

Bhatt U, Xiang A, Sharma S, Weller A, Taly A, Jia Y, Ghosh J, Puri R, Moura JMF, Eckersley P (2020) Explainable machine learning in deployment. In: FAT*’20: conference on fairness, accountability, and transparency, Barcelona, Spain, January 27–30, 2020, ACM, pp 648–657

[19]

Bien J, Tibshirani R, et al. Prototype selection for interpretable classification Ann Appl Stat 2011 5 4 2403-2424

[20]

Bodria F, Giannotti F, Guidotti R, Naretto F, Pedreschi D, Rinzivillo S (2021) Benchmarking and survey of explanation methods for black box models. CoRR arXiv:2102.13076

[21]

Breunig MM, Kriegel H, Ng RT, Sander J (2000) LOF: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD international conference on management of data, May 16–18, 2000, Dallas, Texas, USA, ACM, pp 93–104

[22]

Brughmans D, Martens D (2021) NICE: an algorithm for nearest instance counterfactual explanations. CoRR arXiv:2104.07411

[23]

Buchsbaum D, Bridgers S, Skolnick Weisberg D, and Gopnik A The power of possibility: causal learning, counterfactual reasoning, and pretend play Philos Trans R Soc B Biol Sci 2012 367 1599 2202-2212

[24]

Byrne RMJ (2019) Counterfactuals in explainable artificial intelligence (XAI): evidence from human reasoning. In: Kraus S (ed) Proceedings of the twenty-eighth international joint conference on artificial intelligence, IJCAI 2019, Macao, China, August 10–16, 2019, ijcai.org, pp 6276–6282

[25]

Carlini N, Wagner DA (2017) Adversarial examples are not easily detected: bypassing ten detection methods. In: Proceedings of the 10th ACM workshop on artificial intelligence and security, AISec@CCS 2017, Dallas, TX, USA, November 3, 2017, ACM, pp 3–14

[26]

Carreira-Perpiñán MÁ, Hada SS (2021) Counterfactual explanations for oblique decision trees: exact, efficient algorithms. In: Thirty-Fifth AAAI conference on artificial intelligence, AAAI 2021, thirty-third conference on innovative applications of artificial intelligence, IAAI 2021, the eleventh symposium on educational advances in artificial intelligence, EAAI 2021, Virtual Event, February 2–9, 2021, AAAI Press, pp 6903–6911

[27]

Carvalho DV, Pereira EM, and Cardoso JS Machine learning interpretability: a survey on methods and metrics Electronics 2019 8 8 832

[28]

Chapman-Rounds M, Schulz M, Pazos E, Georgatzis K (2019) EMAP: explanation by minimal adversarial perturbation. CoRR arXiv:1912.00872

[29]

Chapman-Rounds M, Bhatt U, Pazos E, Schulz M, Georgatzis K (2021) FIMAP: feature importance by minimal adversarial perturbation. In: Thirty-fifth AAAI conference on artificial intelligence, AAAI 2021, thirty-third conference on innovative applications of artificial intelligence, IAAI 2021, the eleventh symposium on educational advances in artificial intelligence, EAAI 2021, Virtual Event, February 2–9, 2021, AAAI Press, pp 11433–11441. https://ojs.aaai.org/index.php/AAAI/article/view/17362

[30]

Cheng F, Ming Y, and Qu H DECE: decision explorer with counterfactual explanations for machine learning models IEEE Trans Vis Comput Graph 2021 27 2 1438-1447

[31]

Choi Y, Choi M, Kim M, Ha J, Kim S, Choo J (2018) Stargan: unified generative adversarial networks for multi-domain image-to-image translation. In: 2018 IEEE conference on computer vision and pattern recognition, CVPR 2018, Salt Lake City, UT, USA, June 18–22, 2018, Computer Vision Foundation/IEEE Computer Society, pp 8789–8797

[32]

Craven MW et al (1995) Extracting tree-structured representations of trained networks. In: Touretzky DS, Mozer M, Hasselmo ME (eds) Advances in neural information processing systems 8, NIPS, Denver, CO, USA, November 27–30, 1995. MIT Press, pp 24–30

[33]

Cui Z, Chen W, He Y, Chen Y (2015) Optimal action extraction for random forests and boosted trees. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, Sydney, NSW, Australia, August 10–13, 2015, ACM, pp 179–188

[34]

Dandl S, Molnar C, Binder M, Bischl B (2020) Multi-objective counterfactual explanations. In: Parallel problem solving from nature - PPSN XVI - 16th international conference, PPSN 2020, Leiden, The Netherlands, September 5–9, 2020, Proceedings, Part I, Springer, Lecture notes in computer science, vol 12269, pp 448–469

[35]

Dhurandhar A, Chen P, Luss R, Tu C, Ting P, Shanmugam K, Das P (2018) Explanations based on the missing: towards contrastive explanations with pertinent negatives. In: Advances in neural information processing systems 31: annual conference on neural information processing systems 2018, NeurIPS 2018, December 3–8, 2018, Montréal, Canada, pp 590–601

[36]

Dhurandhar A, Pedapati T, Balakrishnan A, Chen P, Shanmugam K, Puri R (2019) Model agnostic contrastive explanations for structured data. CoRR arXiv:1906.00117

[37]

Dosilovic FK, Brcic M, Hlupic N (2018) Explainable artificial intelligence: a survey. In: 41st international convention on information and communication technology, electronics and microelectronics, MIPRO 2018, Opatija, Croatia, May 21–25, 2018, IEEE, pp 210–215

[38]

Downs M, Chu JL, Yacoby Y, Doshi-Velez F, Pan W (2020) CRUDS: counterfactual recourse using disentangled subspaces. In: ICML workshop on human interpretability in machine learning

[39]

Fan C, Li P (2020) Classification acceleration via merging decision trees. In: FODS’20: ACM-IMS foundations of data science conference, virtual event, USA, October 19–20, 2020, ACM, pp 13–22

[40]

Fernandez C, Provost FJ, Han X (2020) Explaining data-driven decisions made by AI systems: the counterfactual approach. CoRR arXiv:2001.07417

[41]

Fernández RR, de Diego IM, Aceña V, Fernández-Isabel A, and Moguerza JM Random forest explainability using counterfactual sets Inf Fusion 2020 63 196-207

[42]

Freitas AA Comprehensible classification models: a position paper SIGKDD Explor 2013 15 1 1-10

Digital Library

[43]

Ghazimatin A, Balalau O, Roy RS, Weikum G (2020) PRINCE: provider-side interpretability with counterfactual explanations in recommender systems. In: WSDM’20: the thirteenth ACM international conference on web search and data mining, Houston, TX, USA, February 3–7, 2020, ACM, pp 196–204

[44]

Gilpin LH, Bau D, Yuan BZ, Bajwa A, Specter M, Kagal L (2018) Explaining explanations: an overview of interpretability of machine learning. In: 5th IEEE international conference on data science and advanced analytics, DSAA 2018, Turin, Italy, October 1–3, 2018, IEEE, pp 80–89

[45]

Goebel R, Chander A, Holzinger K, Lécué F, Akata Z, Stumpf S, Kieseberg P, Holzinger A (2018) Explainable AI: the new 42? In: Machine learning and knowledge extraction - second IFIP TC 5, TC 8/WG 8.4, 8.9, TC 12/WG 12.9 International cross-domain conference, CD-MAKE 2018, Hamburg, Germany, August 27–30, 2018, Proceedings, Springer, Lecture notes in computer science, vol 11015, pp 295–303

[46]

Gomez O, Holter S, Yuan J, Bertini E (2020) Vice: visual counterfactual explanations for machine learning models. In: IUI’20: 25th international conference on intelligent user interfaces, Cagliari, Italy, March 17–20, 2020, ACM, pp 531–535

[47]

Goyal Y, Wu Z, Ernst J, Batra D, Parikh D, Lee S (2019) Counterfactual visual explanations. In: Proceedings of the 36th international conference on machine learning, ICML 2019, 9–15 June 2019, Long Beach, California, USA, PMLR, Proceedings of machine learning research, vol 97, pp 2376–2384

[48]

Guidotti R Evaluating local explanation methods on ground truth Artif Intell 2021 291

[49]

Guidotti R, Monreale A (2020) Data-agnostic local neighborhood generation. In: 20th IEEE international conference on data mining, ICDM 2020, Sorrento, Italy, November 17–20, 2020, IEEE, pp 1040–1045

[50]

Guidotti R, Ruggieri S (2019) On the stability of interpretable models. In: International joint conference on neural networks, IJCNN 2019 Budapest, Hungary, July 14–19, 2019, IEEE, pp 1–8

[51]

Guidotti R, Monreale A, Giannotti F, Pedreschi D, Ruggieri S, and Turini F Factual and counterfactual explanations for black box decision making IEEE Intell Syst 2019 34 6 14-23

[52]

Guidotti R, Monreale A, Matwin S, Pedreschi D (2019b) Black box explanation by learning image exemplars in the latent feature space. In: Machine learning and knowledge discovery in databases—European conference, ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part I, Springer, Lecture notes in computer science, vol 11906, pp 189–205

[53]

Guidotti R, Monreale A, Ruggieri S, Turini F, Giannotti F, and Pedreschi D A survey of methods for explaining black box models ACM Comput Surv 2019 51 5 93:1-93:42

Digital Library

[54]

Guidotti R, Monreale A, Spinnato F, Pedreschi D, Giannotti F (2020) Explaining any time series classifier. In: 2nd IEEE international conference on cognitive machine intelligence, CogMI 2020, Atlanta, GA, USA, October 28–31, 2020, IEEE, pp 167–176

[55]

Hashemi M, Fathi A (2020) Permuteattack: counterfactual explanation of machine learning credit scorecards. CoRR arXiv:2008.10138

[56]

He Z, Zuo W, Kan M, Shan S, and Chen X Attgan: facial attribute editing by only changing what you want IEEE Trans Image Process 2019 28 11 5464-5478

Digital Library

[57]

Joshi S, Koyejo O, Vijitbenjaronk W, Kim B, Ghosh J (2019) Towards realistic individual recourse and actionable explanations in black-box decision making systems. CoRR arXiv:1907.09615

[58]

Kanamori K, Takagi T, Kobayashi K, Arimura H (2020) DACE: distribution-aware counterfactual explanation by mixed-integer linear optimization. In: Proceedings of the twenty-ninth international joint conference on artificial intelligence, IJCAI 2020, ijcai.org, pp 2855–2862

[59]

Kanamori K, Takagi T, Kobayashi K, Ike Y, Uemura K, Arimura H (2021) Ordered counterfactual explanation by mixed-integer linear optimization. In: Thirty-Fifth AAAI conference on artificial intelligence, AAAI 2021, thirty-third conference on innovative applications of artificial intelligence, IAAI 2021, the eleventh symposium on educational advances in artificial intelligence, EAAI 2021, Virtual Event, February 2–9, 2021, AAAI Press, pp 11564–11574

[60]

Kang S, Jung H, Won D, Lee S (2020) Counterfactual explanation based on gradual construction for deep networks. CoRR arXiv:2008.01897

[61]

Karimi A, Barthe G, Balle B, Valera I (2020) Model-agnostic counterfactual explanations for consequential decisions. In: The 23rd international conference on artificial intelligence and statistics, AISTATS 2020, 26–28 August 2020, Online [Palermo, Sicily, Italy], PMLR, Proceedings of machine learning research, vol 108, pp 895–905

[62]

Karimi A, Barthe G, Schölkopf B, Valera I (2021a) A survey of algorithmic recourse: definitions, formulations, solutions, and prospects. CoRR arXiv:2010.04050

[63]

Karimi A, Schölkopf B, Valera I (2021b) Algorithmic recourse: from counterfactual explanations to interventions. In: FAccT’21: 2021 ACM conference on fairness, accountability, and transparency, virtual event/Toronto, Canada, March 3–10, 2021, ACM, pp 353–362

[64]

Keane MT, Smyth B (2020) Good counterfactuals and where to find them: a case-based technique for generating counterfactuals for explainable AI (XAI). In: Case-based reasoning research and development—28th international conference, ICCBR 2020, Salamanca, Spain, June 8–12, 2020, Proceedings, Springer, Lecture notes in computer science, vol 12311, pp 163–178

[65]

Keane MT, Kenny EM, Delaney E, Smyth B (2021) If only we had better counterfactual explanations: five key deficits to rectify in the evaluation of counterfactual XAI techniques. In: Proceedings of the thirtieth international joint conference on artificial intelligence, IJCAI 2021, Virtual Event/Montreal, Canada, 19–27 August 2021, ijcai.org, pp 4466–4474

[66]

Kenny EM, Keane MT (2021) On generating plausible counterfactual and semi-factual explanations for deep learning. In: Thirty-fifth AAAI conference on artificial intelligence, AAAI 2021, thirty-third conference on innovative applications of artificial intelligence, IAAI 2021, the eleventh symposium on educational advances in artificial intelligence, EAAI 2021, Virtual Event, February 2–9, 2021, AAAI Press, pp 11575–11585

[67]

Kianpour M, Wen S (2019) Timing attacks on machine learning: state of the art. In: Intelligent systems and applications - proceedings of the 2019 intelligent systems conference, IntelliSys 2019, London, UK, September 5–6, 2019, Volume 1, Springer, Advances in intelligent systems and computing, vol 1037, pp 111–125

[68]

Kim B, Koyejo O, Khanna R (2016) Examples are not enough, learn to criticize! criticism for interpretability. In: Advances in neural information processing systems 29: annual conference on neural information processing systems 2016, December 5–10, 2016, Barcelona, Spain, pp 2280–2288

[69]

Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: 3rd international conference on learning representations, ICLR 2015, San Diego, CA, USA, May 7–9, 2015, Conference track proceedings

[70]

Klys J, Snell J, Zemel RS (2018) Learning latent subspaces in variational autoencoders. In: Advances in neural information processing systems 31: annual conference on neural information processing systems 2018, NeurIPS 2018, December 3–8, 2018, Montréal, Canada, pp 6445–6455

[71]

Koh PW, et al. (2017) Understanding black-box predictions via influence functions. In: Proceedings of the 34th international conference on machine learning, ICML 2017, Sydney, NSW, Australia, 6–11 August 2017, PMLR, Proceedings of machine learning research, vol 70, pp 1885–1894

[72]

Kovalev M, Utkin LV, Coolen FPA, and Konstantinov AV Counterfactual explanation of machine learning survival models Informatica 2021 32 4 817-847

Digital Library

[73]

Kusner MJ, Loftus JR, Russell C, Silva R (2017) Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, December 4–9, 2017, long beach, ca, USA. In: NIPS, pp 4066–4076

[74]

Lampridis O, Guidotti R, Ruggieri S (2020) Explaining sentiment classification with synthetic exemplars and counter-exemplars. In: Discovery science—23rd international conference, DS 2020, Thessaloniki, Greece, October 19–21, 2020, Proceedings, Springer, Lecture notes in computer science, vol 12323, pp 357–373

[75]

Lash MT, Lin Q, Street WN, Robinson JG (2017a) A budget-constrained inverse classification framework for smooth classifiers. In: 2017 IEEE international conference on data mining workshops, ICDM workshops 2017, New Orleans, LA, USA, November 18–21, 2017, IEEE Computer Society, pp 1184–1193

[76]

Lash MT, Lin Q, Street WN, Robinson JG, Ohlmann JW (2017b) Generalized inverse classification. In: Proceedings of the 2017 SIAM international conference on data mining, Houston, Texas, USA, April 27–29, 2017, SIAM, pp 162–170

[77]

Laugel T, Lesot M, Marsala C, Renard X, Detyniecki M (2018) Comparison-based inverse classification for interpretability in machine learning. In: Information processing and management of uncertainty in knowledge-based systems. Theory and foundations—17th international conference, IPMU 2018, Cádiz, Spain, June 11–15, 2018, Proceedings, Part I, Springer, Communications in computer and information science, vol 853, pp 100–111

[78]

Laugel T, Lesot M, Marsala C, Renard X, Detyniecki M (2019) The dangers of post-hoc interpretability: unjustified counterfactual explanations. In: Proceedings of the twenty-eighth international joint conference on artificial intelligence, IJCAI 2019, Macao, China, August 10–16, 2019, ijcai.org, pp 2801–2807

[79]

Le T, Wang S, Lee D (2020) GRACE: generating concise and informative contrastive sample to explain neural network model’s prediction. In: KDD’20: the 26th ACM SIGKDD conference on knowledge discovery and data mining, Virtual Event, CA, USA, August 23–27, 2020, ACM, pp 238–248

[80]

Lee J, Mirrokni VS, Nagarajan V, Sviridenko M (2009) Non-monotone submodular maximization under matroid and knapsack constraints. In: Proceedings of the 41st annual ACM symposium on theory of computing, STOC 2009, Bethesda, MD, USA, May 31–June 2, 2009, ACM, pp 323–332

[81]

Li XH, Cao CC, Shi Y, Bai W, Gao H, Qiu L, Wang C, Gao Y, Zhang S, Xue X, et al (2020) A survey of data-driven and knowledge-aware explainable AI. IEEE Trans Knowl Data Eng

[82]

Lipton P Contrastive explanation R Inst Philos Suppl 1990 27 247-266

[83]

Liu FT, Ting KM, Zhou Z (2008) Isolation forest. In: Proceedings of the 8th IEEE international conference on data mining (ICDM 2008), December 15–19, 2008, Pisa, Italy, IEEE Computer Society, pp 413–422

[84]

Lucic A, Oosterhuis H, Haned H, de Rijke M (2019) Focus: flexible optimizable counterfactual explanations for tree ensembles. CoRR arXiv:1911.12199

[85]

Lucic A, Haned H, de Rijke M (2020) Why does my model fail? Contrastive local explanations for retail forecasting. In: FAT*’20: conference on fairness, accountability, and transparency, Barcelona, Spain, January 27–30, 2020, ACM, pp 90–98

[86]

Lucic A, Ter Hoeve M, Tolomei G, de Rijke M, Silvestri F (2021) Cf-gnnexplainer: counterfactual explanations for graph neural networks. CoRR arXiv:2102.03322

[87]

Lundberg SM, Lee S (2017) A unified approach to interpreting model predictions. In: Advances in neural information processing systems 30: annual conference on neural information processing systems 2017, December 4–9, 2017, Long Beach, CA, USA, pp 4765–4774

[88]

Mahajan D, Tan C, Sharma A (2019) Preserving causal constraints in counterfactual explanations for machine learning classifiers. CoRR arXiv:1912.03277

[89]

Martens D and Provost FJ Explaining data-driven document classifications MIS Q 2014 38 1 73-99

Digital Library

[90]

Martens D, Baesens B, Van Gestel T, and Vanthienen J Comprehensible credit scoring models using rule extraction from support vector machines Eur J Oper Res 2007 183 3 1466-1476

[91]

Mazzine R and Martens D A framework and benchmarking study for counterfactual generating methods on tabular data Appl Sci 2021 11 16 7274

[92]

Mc Grath R, Costabello L, Le Van C, Sweeney P, Kamiab F, Shen Z, Lécué F (2018) Interpretable credit application predictions with counterfactual explanations. CoRR arXiv:1811.05245

[93]

McGill AL et al. Contrastive and counterfactual reasoning in causal judgment J Person Soc Psychol 1993 64 6 897

[94]

Miller T (2018) Contrastive explanation: a structural-model approach. CoRR arXiv:1811.03163

[95]

Miller T Explanation in artificial intelligence: insights from the social sciences Artif Intell 2019 267 1-38

[96]

Mohammadi K, Karimi A, Barthe G, Valera I (2021) Scaling guarantees for nearest counterfactual explanations. In: AIES’21: AAAI/ACM conference on AI, ethics, and society, Virtual Event, USA, May 19–21, 2021, ACM, pp 177–187

[97]

Molnar C (2020) Interpretable machine learning. Lulu. com

[98]

Moore J, Hammerla N, Watkins C (2019) Explaining deep learning models with constrained adversarial examples. In: PRICAI 2019: trends in artificial intelligence—16th Pacific rim international conference on artificial intelligence, Cuvu, Yanuca Island, Fiji, August 26–30, 2019, Proceedings, Part I, Springer, Lecture notes in computer science, vol 11670, pp 43–56

[99]

Mothilal RK, Sharma A, Tan C (2020) Explaining machine learning classifiers through diverse counterfactual explanations. In: FAT*’20: conference on fairness, accountability, and transparency, Barcelona, Spain, January 27–30, 2020, ACM, pp 607–617

[100]

Mothilal RK, Mahajan D, Tan C, Sharma A (2021) Towards unifying feature attribution and counterfactual explanations: different means to the same end. In: AIES’21: AAAI/ACM conference on AI, ethics, and society, virtual event, USA, May 19–21, 2021, ACM, pp 652–663

[101]

Murdoch WJ, Singh C, Kumbier K, Abbasi-Asl R, and Yu B Definitions, methods, and applications in interpretable machine learning Proc Natl Acad Sci 2019 116 44 22071-22080

[102]

Nebro AJ, Durillo JJ, García-Nieto J, Coello CAC, Luna F, Alba E (2009) SMPSO: a new pso-based metaheuristic for multi-objective optimization. In: 2009 IEEE symposium on computational intelligence in multi-criteria decision-making, MCDM 2009, Nashville, TN, USA, March 30–April 2, 2009, IEEE, pp 66–73

[103]

Numeroso D, Bacciu D (2021) MEG: generating molecular counterfactual explanations for deep graph networks. In: 2021 international joint conference on neural networks (IJCNN), IEEE, pp 1–8

[104]

Panigutti C, Perotti A, Pedreschi D (2020) Doctor XAI: an ontology-based approach to black-box sequential data classification explanations. In: FAT*’20: conference on fairness, accountability, and transparency, Barcelona, Spain, January 27–30, 2020, ACM, pp 629–639

[105]

Parmentier A, Vidal T (2021) Optimal counterfactual explanations in tree ensembles. In: Proceedings of the 38th international conference on machine learning, ICML 2021, 18–24 July 2021, Virtual Event, PMLR, Proceedings of machine learning research, vol 139, pp 8422–8431

[106]

Pawelczyk M, Broelemann K, Kasneci G (2020) Learning model-agnostic counterfactual explanations for tabular data. In: WWW’20: the web conference 2020, Taipei, Taiwan, April 20–24, 2020, ACM / IW3C2, pp 3126–3132

[107]

Pawelczyk M, Bielawski S, van den Heuvel J, Richter T, Kasneci G (2021) CARLA: a python library to benchmark algorithmic recourse and counterfactual explanation algorithms. CoRR arXiv:2108.00783

[108]

Pearl J et al. Causal inference in statistics: an overview Stat Surv 2009 3 96-146

[109]

Powell MJD On search directions for minimization algorithms Math Program 1973 4 1 193-201

[110]

Poyiadzi R, Sokol K, Santos-Rodríguez R, De Bie T, Flach PA (2020) FACE: feasible and actionable counterfactual explanations. In: AIES’20: AAAI/ACM conference on AI, ethics, and society, New York, NY, USA, February 7–8, 2020, ACM, pp 344–350

[111]

Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I, et al. Language models are unsupervised multitask learners OpenAI Blog 2019 1 8 9

[112]

Ramakrishnan G, Lee YC, Albarghouthi A (2020) Synthesizing action sequences for modifying model decisions. In: The thirty-fourth AAAI conference on artificial intelligence, AAAI 2020, the thirty-second innovative applications of artificial intelligence conference, IAAI 2020, The tenth AAAI symposium on educational advances in artificial intelligence, EAAI 2020, New York, NY, USA, February 7–12, 2020, AAAI Press, pp 5462–5469

[113]

Ramon Y, Martens D, Provost FJ, and Evgeniou T A comparison of instance-level counterfactual explanation algorithms for behavioral and textual data: Sedc, LIME-C and SHAP-C Adv Data Anal Classif 2020 14 4 801-819

Digital Library

[114]

Rathi S (2019) Generating counterfactual and contrastive explanations using SHAP. CoRR arXiv:1906.09293

[115]

Rawal K, Lakkaraju H (2020) Beyond individualized recourse: interpretable and interactive summaries of actionable recourses. In: Beyond individualized recourse: interpretable and interactive summaries of actionable recourses

[116]

Ribeiro MT, Singh S, Guestrin C (2016) "why should I trust you?": Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, San Francisco, CA, USA, August 13–17, 2016, ACM, pp 1135–1144

[117]

Ribeiro MT, Singh S, Guestrin C (2018) Anchors: high-precision model-agnostic explanations. In: Proceedings of the thirty-second AAAI conference on artificial intelligence, (AAAI-18), the 30th innovative applications of artificial intelligence (IAAI-18), and the 8th AAAI symposium on educational advances in artificial intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2–7, 2018, AAAI Press, pp 1527–1535

[118]

Rockoff JE, Jacob BA, Kane TJ, and Staiger DO Can you recognize an effective teacher when you recruit one? Educ Finance Policy 2011 6 1 43-74

[119]

Russell C (2019) Efficient search for diverse coherent explanations. In: Proceedings of the conference on fairness, accountability, and transparency, FAT* 2019, Atlanta, GA, USA, January 29–31, 2019, ACM, pp 20–28

[120]

Samek W, Montavon G, Vedaldi A, Hansen LK, Müller K (eds) (2019) Explainable AI: interpreting, explaining and visualizing deep learning, Lecture notes in computer science, vol 11700. Springer

[121]

Schleich M, Geng Z, Zhang Y, and Suciu D Geco: quality counterfactual explanations in real time Proc VLDB Endow 2021 14 9 1681-1693

Digital Library

[122]

Setzu M, Guidotti R, Monreale A, Turini F, Pedreschi D, and Giannotti F Glocalx—from local to global explanations of black box AI models Artif Intell 2021 294 103457

[123]

Shakhnarovich G, Darrell T, and Indyk P Nearest-neighbor methods in learning and vision IEEE Trans Neural Netw 2008 19 2 377

[124]

Sharma S, Henderson J, Ghosh J (2019) CERTIFAI: counterfactual explanations for robustness, transparency, interpretability, and fairness of artificial intelligence models. CoRR arXiv:1905.07857

[125]

Slack D, Hilgard S, Lakkaraju H, Singh S (2021) Counterfactual explanations can be manipulated. Advances in Neural Information Processing Systems 34

[126]

Sokol K, Santos-Rodríguez R, Flach PA (2019) FAT forensics: a python toolbox for algorithmic fairness, accountability and transparency. CoRR arXiv:1909.05167

[127]

Stepin I, Alonso JM, Catalá A, and Pereira-Fariña M A survey of contrastive and counterfactual explanation generation methods for explainable artificial intelligence IEEE Access 2021 9 11974-12001

[128]

Strecht P (2015) A survey of merging decision trees data mining approaches. In: Proceedings of 10th doctoral symposium in informatics engineering, pp 36–47

[129]

Tjoa E, Guan C (2019) A survey on explainable artificial intelligence (XAI): towards medical XAI. CoRR arXiv:1907.07374

[130]

Tolomei G, Silvestri F, Haines A, Lalmas M (2017) Interpretable predictions of tree-based ensembles via actionable feature tweaking. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, Halifax, NS, Canada, August 13–17, 2017, ACM, pp 465–474

[131]

Tomsett R, Braines D, Harborne D, Preece AD, Chakraborty S (2018) Interpretable to whom? A role-based model for analyzing interpretable machine learning systems. CoRR arXiv:1806.07552

[132]

Tsirtsis S, Rodriguez MG (2020) Decisions, counterfactual explanations and strategic behavior. In: NeurIPS

[133]

Ustun B, Spangher A, Liu Y (2019) Actionable recourse in linear classification. In: Proceedings of the conference on fairness, accountability, and transparency, FAT* 2019, Atlanta, GA, USA, January 29–31, 2019, ACM, pp 10–19

[134]

Van Der Waa J, Robeer M, Van Diggelen J, Brinkhuis M, Neerincx MA (2019) Contrastive explanations with local foil trees. CoRR arXiv:1806.07470

[135]

Van Looveren A, Klaise J (2021) Interpretable counterfactual explanations guided by prototypes. In: Machine learning and knowledge discovery in databases. Research Track - European conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, Proceedings, Part II, Springer, Lecture notes in computer science, vol 12976, pp 650–665

[136]

Verma S, Dickerson JP, Hines K (2020) Counterfactual explanations for machine learning: a review. CoRR arXiv:2010.10596

[137]

Vermeire T, Martens D (2022) Explainable image classification with evidence counterfactual. Pattern Analysis and Applications, pp 1–21

[138]

Visani G, Bagli E, Chesani F, Poluzzi A, Capuzzo D (2020) Statistical stability indices for LIME: obtaining reliable explanations for machine learning models. CoRR arXiv:2001.11757

[139]

Von Kügelgen J, Bhatt U, Karimi A, Valera I, Weller A, Schölkopf B (2020) On the fairness of causal algorithmic recourse. CoRR arXiv:2010.06529

[140]

Wachter S, Mittelstadt BD, and Russell C Counterfactual explanations without opening the black box: automated decisions and the GDPR Harv JL Tech 2017 31 841

[141]

Wang P, Vasconcelos N (2020) SCOUT: self-aware discriminant counterfactual explanations. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, CVPR 2020, Seattle, WA, USA, June 13–19, 2020, Computer Vision Foundation/IEEE, pp 8978–8987

[142]

Waters A and Miikkulainen R GRADE: machine learning support for graduate admissions AI Mag 2014 35 1 64-75

Digital Library

[143]

Wexler J, Pushkarna M, Bolukbasi T, Wattenberg M, Viégas FB, and Wilson J The what-if tool: interactive probing of machine learning models IEEE Trans Vis Comput Graph 2020 26 1 56-65

[144]

White A, d’Avila Garcez AS (2020) Measurable counterfactual local explanations for any classifier. In: ECAI 2020—24th European conference on artificial intelligence, 29 August–8 September 2020, Santiago de Compostela, Spain, August 29–September 8, 2020 - Including 10th conference on prestigious applications of artificial intelligence (PAIS 2020), IOS Press, Frontiers in Artificial Intelligence and Applications, vol 325, pp 2529–2535

[145]

Wilson DR and Martinez TR Improved heterogeneous distance functions J Artif Intell Res 1997 6 1-34

[146]

Wu T, Ribeiro MT, Heer J, Weld DS (2021) Polyjuice: Generating counterfactuals for explaining, evaluating, and improving models. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing, ACL/IJCNLP 2021, Volume 1: Long Papers, virtual event, August 1–6, 2021, Association for Computational Linguistics, pp 6707–6723

[147]

Yang L, Kenny EM, Ng TLJ, Yang Y, Smyth B, Dong R (2020) Generating plausible counterfactual explanations for deep transformers in financial text classification. In: Proceedings of the 28th international conference on computational linguistics, COLING 2020, Barcelona, Spain (Online), December 8–13, 2020, International Committee on Computational Linguistics, pp 6150–6160

[148]

Zhang Y and Chen X Explainable recommendation: a survey and new perspectives Found Trends Inf Retr 2020 14 1 1-101

Digital Library

[149]

Zhang X, Solar-Lezama A, Singh R (2018) Interpreting neural network judgments via minimal, stable, and symbolic corrections. In: Advances in neural information processing systems 31: annual conference on neural information processing systems 2018, NeurIPS 2018, December 3–8, 2018, Montréal, Canada, pp 4879–4890

[150]

Zhao Y (2020) Fast real-time counterfactual explanations. CoRR arXiv:2007.05684

[151]

Zou H and Hastie T Regularization and variable selection via the elastic net J R Stat Soc Ser B Stat Methodol 2005 67 2 301-320

Cited By

Pan CTian YMa LZhou TOuyang SLi J(2025)CISL-PDExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.125506261:COnline publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1016/j.eswa.2024.125506
Jiang JLeofante FRago AToni FDastani MSichman JAlechina NDignum V(2024)Recourse under Model Multiplicity via Argumentative EnsemblingProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3662950(954-963)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3662950
VanNostrand PHofmann DMa LGenin BHuang RRundensteiner E(2024)Counterfactual Explanation Analytics: Empowering Lay Users to Take Action Against Consequential Automated DecisionsProceedings of the VLDB Endowment10.14778/3685800.368587217:12(4349-4352)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.14778/3685800.3685872
Show More Cited By

Index Terms

Counterfactual explanations and how to find them: literature review and benchmarking
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
  2. Machine learning

Index terms have been assigned to the content through auto-classification.

Recommendations

Counterfactual Explanations for Reinforcement Learning Agents
AAMAS '23: Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems

Reinforcement learning (RL) algorithms often use neural networks to represent agent's policy, making them difficult to interpret. Counterfactual explanations are human-friendly explanations which offer users actionable advice on how to change their ...
Explainability Metrics and Properties for Counterfactual Explanation Methods
Explainable and Transparent AI and Multi-Agent Systems
Abstract
The increasing application of Explainable AI (XAI) methods to enhance the transparency and trustworthiness of AI systems designates the need to quantitatively assess and analyze the theoretical and behavioral characteristics of explanations ...
The Privacy Issue of Counterfactual Explanations: Explanation Linkage Attacks
Black-box machine learning models are used in an increasing number of high-stakes domains, and this creates a growing need for Explainable AI (XAI). However, the use of XAI in machine learning introduces privacy risks, which currently remain largely ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Data Mining and Knowledge Discovery

Data Mining and Knowledge Discovery Volume 38, Issue 5

Sep 2024

776 pages

Issue’s Table of Contents

© The Author(s) 2022. corrected publication 2022.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 28 April 2022

Accepted: 18 March 2022

Received: 01 April 2021

Author Tags

Qualifiers

Research-article

Funding Sources

SoBigData++
HumanE AI Net
XAI
TAILOR

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 31 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Pan CTian YMa LZhou TOuyang SLi J(2025)CISL-PDExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.125506261:COnline publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1016/j.eswa.2024.125506
Jiang JLeofante FRago AToni FDastani MSichman JAlechina NDignum V(2024)Recourse under Model Multiplicity via Argumentative EnsemblingProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3662950(954-963)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3662950
VanNostrand PHofmann DMa LGenin BHuang RRundensteiner E(2024)Counterfactual Explanation Analytics: Empowering Lay Users to Take Action Against Consequential Automated DecisionsProceedings of the VLDB Endowment10.14778/3685800.368587217:12(4349-4352)Online publication date: 8-Nov-2024
https://dl.acm.org/doi/10.14778/3685800.3685872
VanNostrand PHofmann DMa LRundensteiner E(2024)Actionable Recourse for Automated Decisions: Examining the Effects of Counterfactual Explanation Type and Presentation on Lay User UnderstandingProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658997(1682-1700)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658997
Corti LOltmans RJung JBalayn AWijsenbeek MYang J(2024)``It Is a Moving Process": Understanding the Evolution of Explainability Needs of Clinicians in Pulmonary MedicineProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642551(1-21)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642551
Li YCai XWu CLin XCao G(2024)A Trustworthy Counterfactual Explanation Method With Latent Space SmoothingIEEE Transactions on Image Processing10.1109/TIP.2024.344261433(4584-4599)Online publication date: 19-Aug-2024
https://dl.acm.org/doi/10.1109/TIP.2024.3442614
Corbugy SMarion RFrénay B(2024)Gradient-based explanation for non-linear non-parametric dimensionality reductionData Mining and Knowledge Discovery10.1007/s10618-024-01055-638:6(3690-3718)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10618-024-01055-6
Atzmueller MFürnkranz JKliegr TSchmid U(2024)Explainable and interpretable machine learning and data miningData Mining and Knowledge Discovery10.1007/s10618-024-01041-y38:5(2571-2595)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s10618-024-01041-y
Guidotti RMonreale ARuggieri SNaretto FTurini FPedreschi DGiannotti F(2024)Stable and actionable explanations of black-box models through factual and counterfactual rulesData Mining and Knowledge Discovery10.1007/s10618-022-00878-538:5(2825-2862)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s10618-022-00878-5
Schwalbe GFinzel B(2024)A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and conceptsData Mining and Knowledge Discovery10.1007/s10618-022-00867-838:5(3043-3101)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s10618-022-00867-8
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents