research-article

Free access

Counterfactual randomization: rescuing experimental studies from obscured confounding

AUTHORs:

Elias BareinboimAuthors Info & Claims

AAAI'19/IAAI'19/EAAI'19: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence

Article No.: 303, Pages 2454 - 2461

https://doi.org/10.1609/aaai.v33i01.33012454

Published: 27 January 2019 Publication History

PDF eReader Publisher Site

Abstract

Randomized clinical trials (RCTs) like those conducted by the FDA provide medical practitioners with average effects of treatments, and are generally more desirable than observational studies due to their control of unobserved confounders (UCs), viz., latent factors that influence both treatment and recovery. However, recent results from causal inference have shown that randomization results in a subsequent loss of information about the UCs, which may impede treatment efficacy if left uncontrolled in practice (Bareinboim, Forney, and Pearl 2015). Our paper presents a novel experimental design that can be noninvasively layered atop past and future RCTs to not only expose the presence of UCs in a system, but also reveal patient- and practitioner-specific treatment effects in order to improve decision-making. Applications are given to personalized medicine, second opinions in diagnosis, and employing offline results in online recommender systems.

References

[1]

Bareinboim, E.; Forney, A.; and Pearl, J. 2015. Bandits with unobserved confounders: A causal approach. In Advances in Neural Information Processing Systems, 1342-1350.

[2]

Brookhart, M. A.; Stürmer, T.; Glynn, R. J.; Rassen, J.; and Schneeweiss, S. 2010. Confounding control in healthcare database research: challenges and potential approaches. Medical care 48(6 0):S114-S120.

[3]

Cormack, D.; Harris, R.; Stanley, J.; Lacey, C.; Jones, R.; and Curtis, E. 2018. Ethnic bias amongst medical students in aotearoa/new Zealand: Findings from the bias and decision making in medicine (bdmm) study. PloS one 13(8):e0201168.

[4]

Fisher, R. 1951. The Design of Experiments. Edinburgh: Oliver and Boyd, 6th edition.

[5]

Forney, A.; Pearl, J.; and Bareinboim, E. 2017. Counter-factual data-fusion for online reinforcement learners. In International Conference on Machine Learning, 1156-1164.

[6]

Giffin, R. B.; Lebovitz, Y.; English, R. A.; et al. 2010. Transforming clinical research in the United States: challenges and opportunities: workshop summary. National Academies Press.

[7]

Haider, A. H.; Schneider, E. B.; Sriram, N.; Dossick, D. S.; Scott, V. K.; Swoboda, S. M.; Losonczy, L.; Haut, E. R.; Efron, D. T.; Pronovost, P. J.; et al. 2015. Unconscious race and social class bias among acute care surgical clinicians and clinical treatment decisions. JAMA surgery 150(5):457-464.

[8]

Hamburg, M. A., and Collins, F. S. 2010. The path to personalized medicine. New England Journal of Medicine 363(4):301-304.

[9]

Langford, J., and Zhang, T. 2008. The epoch-greedy algorithm for multi-armed bandits with side information. In Advances in neural information processing systems, 817-824.

[10]

Lyles, A. 2002. Direct Marketing of Pharmaceuticals to Consumers. Annual Review of Public Health 23(1):73-91.

[11]

Meyer, A. N.; Singh, H.; and Graber, M. L. 2015. Evaluation of Outcomes From a National Patient-initiated Second-opinion Program. The American Journal of Medicine 128(10):1138.e25-1138.e33.

[12]

Ortega, P. A., and Braun, D. A. 2014. Generalized thompson sampling for sequential decision-making and causal inference. Complex Adaptive Systems Modeling 2(1):2.

[13]

Papangelou, K.; Sechidis, K.; Weatherall, J.; and Brown, G. 2018. Toward an understanding of adversarial examples in clinical trials. Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases.

[14]

Pearl, J.; Glymour, M.; and Jewell, N. P. 2016. Causal Inference in Statistics: A Primer. Wiley.

[15]

Pearl, J. 2000. Causality: Models, Reasoning, and Inference. New York: Cambridge University Press. Second ed., 2009.

Digital Library

[16]

Pearl, J. 2017. Detecting latent heterogeneity. Sociological Methods & Research 46(3):370-389.

[17]

Ruberg, S. J., and Shen, L. 2015. Personalized medicine: four perspectives of tailored medicine. Statistics in Biophar-maceutical Research 7(3):214-229.

[18]

Shalit, U.; Johansson, F. D.; and Sontag, D. 2016. Estimating individual treatment effect: generalization bounds and algorithms. arXiv preprint arXiv: 1606.03976.

[19]

Stead, W. W.; Starmer, J. M.; and McClellan, M. 2008. Beyond expert based practice. In Evidence-Based Medicine and the Changing Nature of Healthcare: 2007 IOM Annual Meeting Summary. National Academies Press.

[20]

Van Ryn, M., and Burke, J. 2000. The effect of patient race and socio-economic status on physicians' perceptions of patients. Social science & medicine 50(6):813-828.

[21]

Vashitz, G.; Pliskin, J. S.; Parmet, Y.; Kosashvili, Y.; Ifergane, G.; Wientroub, S.; and Davidovitch, N. 2012. Do first opinions affect second opinions? Journal of general internal medicine 27(10): 1265-71.

[22]

Ventola, C. L. 2011. Direct-to-Consumer Pharmaceutical Advertising: Therapeutic or Toxic? P & T: a peer-reviewed journal for formulary management 36(10):669-84.

[23]

Wainer, H. 1989. Eelworms, bullet holes, and Geraldine Ferraro: Some problems with statistical adjustment and some solutions. Journal of Educational Statistics 14:121-140.

[24]

White, H. D. 2005. Adherence and outcomes: it's more than taking the pills. The Lancet 366(9502): 1989-1991.

Cited By

Wang XLiu YSong XWang BJiang SOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)CaMPProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3666819(15855-15868)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3666819
Browne AForney APelachaud CTaylor MFaliszewski PMascardi V(2022)Exploiting Causal Structure for Transportability in Online, Multi-Agent EnvironmentsProceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems10.5555/3535850.3535874(199-207)Online publication date: 9-May-2022
https://dl.acm.org/doi/10.5555/3535850.3535874
Agwan USpangher LArnold WSrivastava TPoolla KSpanos C(2021)Pricing in Prosumer Aggregations using Reinforcement LearningProceedings of the Twelfth ACM International Conference on Future Energy Systems10.1145/3447555.3464853(220-224)Online publication date: 22-Jun-2021
https://dl.acm.org/doi/10.1145/3447555.3464853

Index Terms

Counterfactual randomization: rescuing experimental studies from obscured confounding

Index terms have been assigned to the content through auto-classification.

Recommendations

Counterfactual Explanations for Prediction and Diagnosis in XAI
AIES '22: Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society

We compared two sorts of explanations for decisions made by an AI system: counterfactual explanations about how an outcome could have been different in the past, and prefactual explanations about how it could be different in the future. We examined the ...
Counterfactual explanations for misclassified images: How human and machine explanations differ
Abstract
Counterfactual explanations have emerged as a popular solution for the eXplainable AI (XAI) problem of elucidating the predictions of black-box deep-learning systems because people easily understand them, they apply across different problem ...
Highlights
- A novel user-centric methodology where users create counterfactual explanations.
- Human-generated counterfactuals differ significantly from machine-generated ones.
- A novel interpretation of this difference using the notion of “...
Model-agnostic counterfactual explanation: A feature weights-based comprehensive causal multi-objective counterfactual framework
Abstract
Counterfactual explanation, as a post-doc interpretable method, has garnered considerable attention. However, previous methods face hurdles in integrating multiple optimization objectives, posing challenges in balancing constraint weights during ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

AAAI'19/IAAI'19/EAAI'19: Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence

January 2019

10088 pages

ISBN:978-1-57735-809-1

Copyright © 2019 Association for the Advancement of Artificial Intelligence.

Sponsors

Association for the Advancement of Artificial Intelligence

Publisher

AAAI Press

Publication History

Published: 27 January 2019

Qualifiers

Research-article
Research
Refereed limited

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
20
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)2

Reflects downloads up to 19 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang XLiu YSong XWang BJiang SOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)CaMPProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3666819(15855-15868)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3666819
Browne AForney APelachaud CTaylor MFaliszewski PMascardi V(2022)Exploiting Causal Structure for Transportability in Online, Multi-Agent EnvironmentsProceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems10.5555/3535850.3535874(199-207)Online publication date: 9-May-2022
https://dl.acm.org/doi/10.5555/3535850.3535874
Agwan USpangher LArnold WSrivastava TPoolla KSpanos C(2021)Pricing in Prosumer Aggregations using Reinforcement LearningProceedings of the Twelfth ACM International Conference on Future Energy Systems10.1145/3447555.3464853(220-224)Online publication date: 22-Jun-2021
https://dl.acm.org/doi/10.1145/3447555.3464853

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten