Learning Entangled Interactions of Complex Causality via Self-Paced Contrastive Learning
Authors: 
Yunji Liang, Lei Liu, Luwen Huangfu, Sagar Samtani, Zhiwen Yu, Daniel D. ZengAuthors Info & Claims
Article No.: 56, Pages 1 - 24
Abstract
Learning causality from large-scale text corpora is an important task with numerous applications—for example, in finance, biology, medicine, and scientific discovery. Prior studies have focused mainly on simple causality, which only includes one cause-effect pair. However, causality is notoriously difficult to understand and analyze because of multiple cause spans and their entangled interactions. To detect complex causality, we propose a self-paced contrastive learning model, namely N2NCause, to learn entangled interactions between multiple spans. Specifically, N2NCause introduces data enhancement operations to convert implicit expressions into explicit expressions with the most rational causal connectives for the synthesis of positive samples and to invert the directed connection between a cause-effect pair for the synthesis of negative samples. To learn the semantic dependency and causal direction of positive and negative samples, self-paced contrastive learning is proposed to learn the entangled interactions among spans, including the interaction direction and interaction field. We evaluated the performance of N2NCause in three cause-effect detection tasks. The experimental results show that, with the least data annotation efforts, N2NCause demonstrates competitive performance in detecting simple cause-effect relations, and it is superior to existing solutions for the detection of complex causality.
References
[1]
Muhammad Abulaish, Mohd Fazil, and Mohammed J. Zaki. 2022. Domain-specific keyword extraction using joint modeling of local and global contextual semantics. ACM Transactions on Knowledge Discovery from Data 16, 4 (Jan. 2022), Article 70, 30 pages.
[2]
Abbas Akkasi and Mari-Francine Moens. 2021. Causal relationship extraction from biomedical text using deep neural models: A comprehensive survey. Journal of Biomedical Informatics 119 (2021), 103820.
[3]
Giannis Bekoulis, Johannes Deleu, Thomas Demeester, and Chris Develder. 2018. Joint entity recognition and relation extraction as a multi-head selection problem. Expert Systems with Applications 114 (2018), 34–45.
[4]
Lu Cheng, Ruocheng Guo, and Huan Liu. 2019. Robust cyberbullying detection with causal interpretation. In Companion Proceedings of the 2019 World Wide Web Conference. 169–175.
[5]
Zifeng Cheng, Zhiwei Jiang, Yafeng Yin, Na Li, and Qing Gu. 2021. A unified target-oriented sequence-to-sequence model for emotion-cause pair extraction. IEEE/ACM Transactions on Audio, Speech, and Language Processing 29 (2021), 2779–2791.
[6]
Fenia Christopoulou, Makoto Miwa, and Sophia Ananiadou. 2018. A walk-based model on entity graphs for relation extraction. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 81–88.
[7]
Tirthankar Dasgupta, Rupsa Saha, Lipika Dey, and Abir Naskar. 2018. Automatic extraction of causal relations from text using linguistically informed deep neural networks. In Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue. 306–316.
[8]
Son Doan, Elly W. Yang, Sameer Tilak, and Manabu Torii. 2018. Using natural language processing to extract health-related causality from Twitter messages. In Proceedings of the 2018 IEEE International Conference on Healthcare Informatics Workshop (ICHI-W ’18). 84–85.
[9]
Chris Donahue, Mina Lee, and Percy Liang. 2020. Enabling language models to fill in the blanks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2492–2501.
[10]
Jesse Dunietz, Lori Levin, and Jaime G. Carbonell. 2017. The BECauSE corpus 2.0: Annotating causality and overlapping relations. In Proceedings of the 11th Linguistic Annotation Workshop. 95–104.
[11]
Jianfeng Fu, Zongtian Liu, Wei Liu, and Wen Zhou. 2011. Event causal relation extraction based on cascaded conditional random fields. Pattern Recognition and Artificial Intelligence 24, 4 (2011), 567–573.
[12]
Tianyu Gao, Xu Han, Zhiyuan Liu, and Maosong Sun. 2019. Hybrid attention-based prototypical networks for noisy few-shot relation classification. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 6407–6414.
[13]
Lin Gui, Dongyin Wu, Ruifeng Xu, Qin Lu, and Yu Zhou. 2016. Event-driven emotion cause extraction with corpus construction. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 1639–1649.
[14]
Lin Gui, Ruifeng Xu, Dongyin Wu, Qin Lu, and Yu Zhou. 2018. Event-driven emotion cause extraction with corpus construction. In Social Media Content Analysis: Natural Language Processing and Beyond. World Scientific, 145–160.
[15]
Harsha Gurulingappa, Abdul Mateen Rajput, Angus Roberts, Juliane Fluck, Martin Hofmann-Apitius, and Luca Toldo. 2012. Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports. Journal of Biomedical Informatics 45, 5 (2012), 885–892.
[16]
Raia Hadsell, Sumit Chopra, and Yann LeCun. 2006. Dimensionality reduction by learning an invariant mapping. In Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR ’06), Vol. 2. IEEE, Los Alamitos, CA, 1735–1742.
[17]
Iris Hendrickx, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid Ó. Séaghdha, Sebastian Padó, Marco Pennacchiotti, Lorenza Romano, and Stan Szpakowicz. 2009. SemEval-2010 Task 8: Multi-way classification of semantic relations between pairs of nominals. In Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions (SEW ’09). 94–99.
[18]
Christopher Hidey and Kathleen McKeown. 2016. Identifying causal relations using parallel Wikipedia articles. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1424–1433.
[19]
Pedram Hosseini, David A. Broniatowski, and Mona Diab. 2021. Predicting directionality in causal relations in text. arXiv preprint arXiv:2103.13606 (2021).
[20]
Akshita Jha, Vineeth Rakesh, Jaideep Chandrashekar, Adithya Samavedhi, and Chandan K. Reddy. 2023. Supervised contrastive learning for interpretable long-form document matching. ACM Transactions on Knowledge Discovery from Data 17, 2 (2023), Article 27, 17 pages.
[21]
Lu Jiang, Deyu Meng, Shoou-I. Yu, Zhenzhong Lan, Shiguang Shan, and Alexander G. Hauptmann. 2014. Self-paced learning with diversity. In Proceedings of the 27th International Conference on Neural Information Processing Systems(NIPS ’14), Vol. 2. 2078–2086.
[22]
Zhijing Jin, Julius von Kügelgen, Jingwei Ni, Tejas Vaidhya, Ayush Kaushal, Mrinmaya Sachan, and Bernhard Schoelkopf. 2021. Causal direction of data collection matters: Implications of causal and anticausal learning for NLP. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 9499–9513.
[23]
Xiangyu Ke, Arijit Khan, and Francesco Bonchi. 2022. Multi-relation graph summarization. ACM Transactions on Knowledge Discovery from Data 16, 5, (March 2022), Article 82, 30 pages.
[24]
Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional Transformers for language understanding. In Proceedings of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT ’19), Vol. 1. 4171–4186.
[25]
Vivek Khetan, Md. Imbesat Rizvi, Jessica Huber, Paige Bartusiak, Bogdan Sacaleanu, and Andrew Fano. 2022. MIMICause: Representation and automatic extraction of causal relation types from clinical notes. In Findings of the Association for Computational Linguistics: ACL 2022, Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). Association for Computational Linguistics, 764–773.
[26]
Manolis Kyriakakis, Ion Androutsopoulos, Artur Saudabayev, and Joan Ginés i Ametllé. 2019. Transfer learning for causal sentence detection. In Proceedings of the 18th BioNLP Workshop and Shared Task. 292–297.
[27]
Siwei Lai, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Recurrent convolutional neural networks for text classification. In Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI ’15). 2267–2273.
[28]
Pengfei Li and Kezhi Mao. 2019. Knowledge-oriented convolutional neural network for causal relation extraction from natural language texts. Expert Systems with Applications 115 (2019), 512–523.
[29]
Qian Li, Xiangmeng Wang, Zhichao Wang, and Guandong Xu. 2023. Be causal: De-biasing social network confounding in recommendation. ACM Transactions on Knowledge Discovery from Data 17, 1 (2023), Article 14, 23 pages.
[30]
Yuequn Li, Wenji Mao, Daniel Zeng, Luwen Huangfu, and Chunyang Liu. 2012. Extracting opinion explanations from Chinese online reviews. In Proceedings of the IEEE International Conference on Intelligence and Security Informatics. 221–223.
[31]
Zhaoning Li, Qi Li, Xiaotian Zou, and Jiangtao Ren. 2021. Causality extraction based on self-attentive BiLSTM-CRF with transferred embeddings. Neurocomputing 423 (2021), 207–219.
[32]
Bin Liang, Xiang Li, Lin Gui, Yonghao Fu, Yulan He, Min Yang, and Ruifeng Xu. 2023. Few-shot aspect category sentiment analysis via meta-learning. ACM Transactions on Information Systems 41, 1 (Jan. 2023), Article 22, 31 pages.
[33]
Yunji Liang, Lei Liu, Yapeng Ji, Luwen Huangfu, and Daniel Dajun Zeng. 2023. Identifying emotional causes of mental disorders from social media for effective intervention. Information Processing & Management 60, 4 (2023), 103407.
[34]
Anan Liu, Ning Xu, and Haozhe Liu. 2021. Self-attention graph residual convolutional networks for event detection with dependency relations. In Findings of the Association for Computational Linguistics: EMNLP 2021. Association for Computational Linguistics, 302–311.
[35]
Jian Liu, Yubo Chen, and Jun Zhao. 2020. Knowledge enhanced event causality identification with mention masking generalizations. In Proceedings of the 29th International Joint Conference on Artificial Intelligence: Main Track (IJCAI ’20). 3608–3614.
[36]
Jingping Liu, Juntao Liu, Lihan Chen, Jiaqing Liang, Yanghua Xiao, Huimin Xu, Fubao Zhang, Zongyu Wang, and Rui Xie. 2023. Noun compound interpretation with relation classification and paraphrasing. IEEE Transactions on Knowledge and Data Engineering 35, 9 (2023), 8757–8769.
[37]
Junlong Liu, Xichen Shang, and Qianli Ma. 2022. Pair-based joint encoding with relational graph convolutional networks for emotion-cause pair extraction. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 5339–5351.
[38]
Jiaying Liu, Feng Xia, Jing Ren, Bo Xu, Guansong Pang, and Lianhua Chi. 2023. MIRROR: Mining implicit relationships via structure-enhanced graph convolutional networks. ACM Transactions on Knowledge Discovery from Data 17, 4 (Feb. 2023), Article 55, 24 pages.
[39]
Matthew Ridley, Gautam Rao, Frank Schilbach, and Vikram Patel. 2020. Poverty, depression, and anxiety: Causal evidence and mechanisms. Science 370, 6522 (2020), eaay0214.
[40]
Jakob Runge, Peer Nowack, Marlene Kretschmer, Seth Flaxman, and Dino Sejdinovic. 2019. Detecting and quantifying causal associations in large nonlinear time series datasets. Science Advances 5, 11 (2019), eaau4996.
[41]
Saurav Sahay, Sougata Mukherjea, Eugene Agichtein, Ernest V. Garcia, Shamkant B. Navathe, and Ashwin Ram. 2008. Discovering semantic biomedical relations utilizing the web. ACM Transactions on Knowledge Discovery from Data 2, 1 (April 2008), Article 3, 15 pages.
[42]
Seyed Amjad Seyedi, S. Siamak Ghodsi, Fardin Akhlaghian, Mahdi Jalili, and Parham Moradi. 2019. Self-paced multi-label learning with diversity. In Proceedings of the Eleventh Asian Conference on Machine Learning, Wee Sun Lee and Taiji Suzuki (Eds.). Proceedings of Machine Learning Research, Vol. 1010. PMLR, 790–805.
[43]
Xinxin Su, Zhen Huang, Yunxiang Zhao, Yifan Chen, Yong Dou, and Hengyue Pan. 2023. Recent trends in deep learning based textual emotion cause extraction. IEEE/ACM Transactions on Audio, Speech, and Language Processing 31 (2023), 2765–2786.
[44]
Peter Turchin, Harvey Whitehouse, Sergey Gavrilets, Daniel Hoyer, Pieter François, James S. Bennett, Kevin C. Feeney, Peter Peregrine, Gary Feinman, Andrey Korotayev, Nikolay Kradin, Jill Levine, Jenny Reddish, Enrico Cioni, Romain Wacziarg, Gavin Mendel-Gleason, and Majid Benam. 2022. Disentangling the evolutionary drivers of social complexity: A comprehensive test of hypotheses. Science Advances 8, 25 (2022), eabn3517.
[45]
Haoyu Wang, Ming Tan, Mo Yu, Shiyu Chang, Dakuo Wang, Kun Xu, Xiaoxiao Guo, and Saloni Potdar. 2019. Extracting multiple-relations in one-pass with pre-trained transformers. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 1371–1377.
[46]
Xiaozhi Wang, Ziqi Wang, Xu Han, Wangyi Jiang, Rong Han, Zhiyuan Liu, Juanzi Li, Peng Li, Yankai Lin, and Jie Zhou. 2020. MAVEN: A massive general domain event detection dataset. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP ’20). 1652–1671.
[47]
Yashen Wang and Huanhuan Zhang. 2021. HARP: A novel hierarchical attention model for relation prediction. ACM Transactions on Knowledge Discovery from Data 15, 2 (Jan. 2021), Article 17, 22 pages.
[48]
Penghui Wei, Jiahao Zhao, and Wenji Mao. 2020. Effective inter-clause modeling for end-to-end emotion-cause pair extraction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 3171–3181.
[49]
Rui Xia and Zixiang Ding. 2019. Emotion-cause pair extraction: A new task to emotion analysis in texts. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 1003–1012.
[50]
Rui Xia, Mengran Zhang, and Zixiang Ding. 2019. RTHN: A RNN-transformer hierarchical network for emotion cause extraction. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI ’19). 5285–5291.
[51]
Jinghang Xu, Wanli Zuo, Shining Liang, and Xianglin Zuo. 2020. A review of dataset and labeling methods for causality extraction. In Proceedings of the 28th International Conference on Computational Linguistics. 1519–1531.
[52]
Zhaodong Yan, Serena Jeblee, and Graeme Hirst. 2019. Can character embeddings improve cause-of-death classification for verbal autopsy narratives? In Proceedings of the 18th BioNLP Workshop and Shared Task. 234–239.
[53]
Jie Yang, Soyeon Caren Han, and Josiah Poon. 2022. A survey on extraction of causal relations from natural language text. Knowledge and Information Systems 64, 5 (May 2022), 1161–1186.
[54]
Liuyi Yao, Zhixuan Chu, Sheng Li, Yaliang Li, Jing Gao, and Aidong Zhang. 2021. A survey on causal inference. ACM Transactions on Knowledge Discovery from Data 15, 5 (May 2021), Article 74, 46 pages.
[55]
Yating Zhang, Adam Jatowt, and Katsumi Tanaka. 2016. Causal relationship detection in archival collections of product reviews for understanding technology evolution. ACM Transactions on Information Systems 35, 1 (Aug. 2016), Article 3, 41 pages.
[56]
Shan Zhao, Minghao Hu, Zhiping Cai, and Fang Liu. 2021. Modeling dense cross-modal interactions for joint entity-relation extraction. In Proceedings of the 29th International Joint Conference on Artificial Intelligence. 4032–4038.
[57]
Sendong Zhao, Ting Liu, Sicheng Zhao, Yiheng Chen, and Jian-Yun Nie. 2016. Event causality extraction based on connectives analysis. Neurocomputing 173 (2016), 1943–1950.
[58]
Peng Zhou, Xinwang Liu, Liang Du, and Xuejun Li. 2023. Self-paced adaptive bipartite graph learning for consensus clustering. ACM Transactions on Knowledge Discovery from Data 17, 5 (Sept. 2023), Article 62, 35 pages.
[59]
Peng Zhou, Wei Shi, Jun Tian, Zhenyu Qi, Bingchen Li, Hongwei Hao, and Bo Xu. 2016. Attention-based bidirectional long short-term memory networks for relation classification. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 207–212.
[60]
Peng Zhou, Bicheng Sun, Xinwang Liu, Liang Du, and Xuejun Li. 2023. Active clustering ensemble with self-paced learning. IEEE Transactions on Neural Networks and Learning Systems. Early Access.
Index Terms
- Learning Entangled Interactions of Complex Causality via Self-Paced Contrastive Learning
Recommendations
A multi-level neural network for implicit causality detection in web texts
AbstractMining causality from text is a complex and crucial natural language understanding task corresponding to human cognition. Existing studies on this subject can be divided into two categories: feature engineering-based and neural model-...
Complexity results for structure-based causality
We give a precise picture of the computational complexity of causal relationships in Pearl's structural models, where we focus on causality between variables, event causality, and probabilistic causality. As for causality between variables, we consider ...
Learning perceptual causality from video
AAAIWS'13-12: Proceedings of the 12th AAAI Conference on Learning Rich Representations from Low-Level SensorsComputer vision and artificial intelligence research has long danced around the subject of causality: vision researchers use causal relationships to aid action detection, and AI researchers propose methods for causal induction independent of video ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
April 2024
663 pages
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 09 December 2023
Online AM: 09 November 2023
Accepted: 27 October 2023
Revised: 25 August 2023
Received: 06 November 2022
Published in TKDD Volume 18, Issue 3
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Natural Science Foundation of China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 283Total Downloads
- Downloads (Last 12 months)283
- Downloads (Last 6 weeks)22
Reflects downloads up to 13 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in