Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3583780.3614885acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article
Open access

Flexible and Robust Counterfactual Explanations with Minimal Satisfiable Perturbations

Published: 21 October 2023 Publication History

Abstract

Counterfactual explanations (CFEs) exemplify how to minimally modify a feature vector to achieve a different prediction for an instance. CFEs can enhance informational fairness and trustworthiness, and provide suggestions for users who receive adverse predictions. However, recent research has shown that multiple CFEs can be offered for the same instance or instances with slight differences. Multiple CFEs provide flexible choices and cover diverse desiderata for user selection. However, individual fairness and model reliability will be damaged if unstable CFEs with different costs are returned. Existing methods fail to exploit flexibility and address the concerns of non-robustness simultaneously. To address these issues, we propose a conceptually simple yet effective solution named Counterfactual Explanations with Minimal Satisfiable Perturbations (CEMSP). Specifically, CEMSP constrains changing values of abnormal features with the help of their semantically meaningful normal ranges. For efficiency, we model the problem as a Boolean satisfiability problem to modify as few features as possible. Additionally, CEMSP is a general framework and can easily accommodate more practical requirements, e.g., casualty and actionability. Compared to existing methods, we conduct comprehensive experiments on both synthetic and real-world datasets to demonstrate that our method provides more robust explanations while preserving flexibility.

References

[1]
André Artelt, Valerie Vaquet, Riza Velioglu, Fabian Hinder, Johannes Brinkrolf, Malte Schilling, and Barbara Hammer. 2021. Evaluating robustness of counterfactual explanations. In 2021 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, 01--09.
[2]
Jaroslav Bend'ik, Ivana vC erná, and Nikola Benevs. 2018. Recursive online enumeration of all minimal unsatisfiable subsets. In International symposium on automated technology for verification and analysis. Springer, 143--159.
[3]
KRISHNA BHATT. 2021. Hepatitis C Predictions. https://www.kaggle.com/code/krishnabhatt4/hepatitis-c-predictions
[4]
Emily Black, Zifan Wang, and Matt Fredrikson. 2022. Consistent Counterfactuals for Deep Models. In International Conference on Learning Representations. https://openreview.net/forum?id=St6eyiTEHnG
[5]
Jana Blechschmidt, Meike J Wittmann, and Chantal Blüml. 2020. Climate change and green sea turtle sex ratio-preventing possible extinction. Genes, Vol. 11, 5 (2020), 588.
[6]
USA CDC. 2022. Assessing Your Weight. https://www.cdc.gov/healthyweight/assessing/index.html
[7]
F. Cheng, Y. Ming, and H. Qu. 2021. DECE: Decision Explorer with Counterfactual Explanations for Machine Learning Models. IEEE Transactions on Visualization & Computer Graphics, Vol. 27, 02 (feb 2021), 1438--1447. https://doi.org/10.1109/TVCG.2020.3030342
[8]
Ryan Daws. 2021. Medical chatbot using OpenAI's GPT-3 told a fake patient to kill themselves. AI News (11 2021). https://www.artificialintelligence-news.com/2020/10/28/medical-chatbot-openai-gpt3-patient-kill-themselves/
[9]
Amit Dhurandhar, Pin-Yu Chen, Ronny Luss, Chun-Chen Tu, Paishun Ting, Karthikeyan Shanmugam, and Payel Das. 2018. Explanations based on the missing: Towards contrastive explanations with pertinent negatives. In Advances in Neural Information Processing Systems. 592--603.
[10]
Persi Diaconis and Bradley Efron. 1983. Computer-intensive methods in statistics. Scientific American, Vol. 248, 5 (1983), 116--131.
[11]
M-P Dubuisson and Anil K Jain. 1994. A modified Hausdorff distance for object matching. In Proceedings of 12th international conference on pattern recognition, Vol. 1. IEEE, 566--568.
[12]
Sanghamitra Dutta, Jason Long, Saumitra Mishra, Cecilia Tilli, and Daniele Magazzeni. 2022. Robust counterfactual explanations for tree-based ensembles. In International Conference on Machine Learning. PMLR, 5742--5756.
[13]
Georg Hoffmann, Ralf Lichtinghagen, and Werner Wosniok. 2016. Simple estimation of reference intervals from routine laboratory data. LaboratoriumsMedizin, Vol. 39, s1 (2016).
[14]
Oliver Jesorsky, Klaus J Kirchberg, and Robert W Frischholz. 2001. Robust face detection using the hausdorff distance. In International conference on audio-and video-based biometric person authentication. Springer, 90--95.
[15]
Shalmali Joshi, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. 2019. Towards realistic individual recourse and actionable explanations in black-box decision making systems. arXiv preprint arXiv:1907.09615 (2019).
[16]
Shashank R Joshi et al. 2011. Laboratory evaluation of thyroid function. J Assoc Physicians India, Vol. 59, Suppl (2011), 14--20.
[17]
Amir-Hossein Karimi, Bernhard Schö lkopf, and Isabel Valera. 2021. Algorithmic Recourse: from Counterfactual Explanations to Interventions. In FAccT '21: 2021 ACM Conference on Fairness, Accountability, and Transparency, Virtual Event / Toronto, Canada, March 3--10, 2021, Madeleine Clare Elish, William Isaac, and Richard S. Zemel (Eds.). ACM, 353--362. https://doi.org/10.1145/3442188.3445899
[18]
Amir-Hossein Karimi, Julius von Kügelgen, Bernhard Schölkopf, and Isabel Valera. 2020. Algorithmic recourse under imperfect causal knowledge: a probabilistic approach. Advances in Neural Information Processing Systems, Vol. 33 (2020).
[19]
Atoosa Kasirzadeh and Andrew Smart. 2021. The Use and Misuse of Counterfactuals in Ethical Machine Learning. In FAccT '21: 2021 ACM Conference on Fairness, Accountability, and Transparency, Virtual Event / Toronto, Canada, March 3--10, 2021, Madeleine Clare Elish, William Isaac, and Richard S. Zemel (Eds.). ACM, 228--236. https://doi.org/10.1145/3442188.3445886
[20]
Michael T Lash, Qihang Lin, Nick Street, Jennifer G Robinson, and Jeffrey Ohlmann. 2017. Generalized inverse classification. In Proceedings of the 2017 SIAM International Conference on Data Mining. SIAM, 162--170.
[21]
Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, and Marcin Detyniecki. 2019. Issues with post-hoc counterfactual explanations: a discussion. arXiv preprint arXiv:1906.04774 (2019).
[22]
Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, and Marcin Detyniecki. 2017. Inverse classification for comparison-based interpretability in machine learning. arXiv preprint arXiv:1712.08443 (2017).
[23]
Mark H Liffiton and Ammar Malik. 2013. Enumerating infeasibility: Finding multiple MUSes quickly. In International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research. Springer, 160--175.
[24]
Mark H Liffiton and Karem A Sakallah. 2005. On finding all minimally unsatisfiable subformulas. In International conference on theory and applications of satisfiability testing. Springer, 173--186.
[25]
Ramaravind K Mothilal, Amit Sharma, and Chenhao Tan. 2020. Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 607--617.
[26]
Martin Pawelczyk, Klaus Broelemann, and Gjergji Kasneci. 2020a. Learning Model-Agnostic Counterfactual Explanations for Tabular Data. In Proceedings of The Web Conference 2020. 3126--3132.
[27]
Martin Pawelczyk, Klaus Broelemann, and Gjergji Kasneci. 2020b. On counterfactual explanations under predictive multiplicity. In Conference on Uncertainty in Artificial Intelligence. PMLR, 809--818.
[28]
Judea Pearl. 2009. Causality 2 ed.). Cambridge University Press. https://doi.org/10.1017/CBO9780511803161
[29]
J. R. Quinlan, P. J. Compton, K. A. Horn, and L. Lazarus. 1987. Inductive Knowledge Acquisition: A Case Study. In Proceedings of the Second Australian Conference on Applications of Expert Systems. Addison-Wesley Longman Publishing Co., Inc., USA, 137--156.
[30]
Chris Russell. 2019. Efficient search for diverse coherent explanations. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 20--28.
[31]
Jakob Schoeffer, Niklas Kuehl, and Yvette Machowski. 2022. ?There Is Not Enough Information": On the Effects of Explanations on Perceptions of Informational Fairness and Trustworthiness in Automated Decision-Making. In 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea) (FAccT '22). Association for Computing Machinery, New York, NY, USA, 1616--1628. https://doi.org/10.1145/3531146.3533218
[32]
Shubham Sharma, Jette Henderson, and Joydeep Ghosh. 2020. CERTIFAI: A Common Framework to Provide Explanations and Analyse the Fairness and Robustness of Black-Box Models. Association for Computing Machinery, New York, NY, USA, 166--172. https://doi.org/10.1145/3375627.3375812
[33]
Jianying Shen, Jingying Zhang, Jing Wen, Qiang Ming, Ji Zhang, and Yawei Xu. 2015. Correlation of serum alanine aminotransferase and aspartate aminotransferase with coronary heart disease. International journal of clinical and experimental medicine, Vol. 8, 3 (2015), 4399.
[34]
Robert H. Shmerling, MD. 2020. How's your heart rate and why it matters? Harvard Health (3 2020). https://www.health.harvard.edu/heart-health/hows-your-heart-rate-and-why-it-matters
[35]
Dylan Slack, Anna Hilgard, Himabindu Lakkaraju, and Sameer Singh. 2021. Counterfactual explanations can be manipulated. Advances in Neural Information Processing Systems, Vol. 34 (2021), 62--75.
[36]
Kacper Sokol and Peter A Flach. 2019. Counterfactual explanations of machine learning predictions: opportunities and challenges for AI safety. In SafeAI@ AAAI.
[37]
Gabriele Tolomei, Fabrizio Silvestri, Andrew Haines, and Mounia Lalmas. 2017. Interpretable predictions of tree-based ensembles via actionable feature tweaking. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. 465--474.
[38]
Sohini Upadhyay, Shalmali Joshi, and Himabindu Lakkaraju. 2021. Towards robust and reliable algorithmic recourse. Advances in Neural Information Processing Systems, Vol. 34 (2021), 16926--16937.
[39]
Berk Ustun, Alexander Spangher, and Yang Liu. 2019. Actionable recourse in linear classification. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 10--19.
[40]
Arnaud Van Looveren and Janis Klaise. 2019. Interpretable counterfactual explanations guided by prototypes. arXiv preprint arXiv:1907.02584 (2019).
[41]
Sahil Verma, John Dickerson, and Keegan Hines. 2020. Counterfactual Explanations for Machine Learning: A Review. arXiv preprint arXiv:2010.10596 (2020).
[42]
Marco Virgolin and Saverio Fracaros. 2023. On the Robustness of Sparse Counterfactual Explanations to Adverse Perturbations. Artif. Intell., Vol. 316, C (feb 2023), 30 pages. https://doi.org/10.1016/j.artint.2022.103840
[43]
Julius von Kügelgen, Amir-Hossein Karimi, Umang Bhatt, Isabel Valera, Adrian Weller, and Bernhard Schölkopf. 2022. On the fairness of causal algorithmic recourse. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 9584--9594.
[44]
Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2017. Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR. arxiv: 1711.00399 [cs.AI]
[45]
Yongjie Wang, Qinxu Ding, Ke Wang, Yue Liu, Xingyu Wu, Jinglong Wang, Yong Liu, and Chunyan Miao. 2021. The Skyline of Counterfactual Explanations for Machine Learning Decision Models. Association for Computing Machinery, New York, NY, USA, 2030--2039. https://doi.org/10.1145/3459637.3482397

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management
October 2023
5508 pages
ISBN:9798400701245
DOI:10.1145/3583780
This work is licensed under a Creative Commons Attribution-ShareAlike International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 October 2023

Check for updates

Author Tags

  1. counterfactual explanations
  2. flexibility
  3. multiplicity
  4. normal ranges
  5. robustness

Qualifiers

  • Research-article

Funding Sources

  • Alibaba Innovative Research (AIR) Program
  • Alibaba-NTU Singapore Joint Research Institute (JRI)
  • NRF Investigatorship Programme

Conference

CIKM '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 278
    Total Downloads
  • Downloads (Last 12 months)254
  • Downloads (Last 6 weeks)32
Reflects downloads up to 09 Nov 2024

Other Metrics

Citations

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media