A Unified Framework for Analyzing Textual Context and Intent in Social Media
Article No.: 118, Pages 1 - 25
Abstract
In the realm of natural language processing, tasks like emotion recognition, irony detection, hate speech detection, offensive language identification, and stance detection are pivotal for understanding user-generated content. While several task-specific and multitask learning models have been proposed, there remains a need for a unified framework that can effectively address these tasks simultaneously. This research introduces a novel unified framework designed to tackle multiple NLP tasks concurrently, aiming to outperform existing task-specific and multitask models in terms of accuracy, F1-score, and AUC-ROC. We compared our proposed framework against several baseline models, including task-specific models like SVM, RF, LSTM, CNN, and BERT, as well as multitask learning frameworks such as Hard Parameter Sharing, Soft Parameter Sharing, Cross-stitch Networks, MMoE, and T5. The performance was evaluated across various tasks, and statistical significance was assessed using the Wilcoxon signed-rank test. Additionally, an ablation study was conducted to determine the contribution of individual components within our proposed method. The proposed framework consistently outperformed other models across all tasks. For instance, in emotion recognition, our model achieved an accuracy of 0.899, F1-score of 0.883, and AUC-ROC of 0.971, surpassing all baseline models. The Wilcoxon signed-rank test further confirmed the statistical superiority of our model over the baselines across all datasets.
References
[1]
Tariq Abdullah and Ahmed Ahmet. 2022. Deep learning in sentiment analysis: Recent architectures. Comput. Surveys 55, 8 (2022), 1–37.
[2]
Francesco Barbieri, Jose Camacho-Collados, Leonardo Neves, and Luis Espinosa-Anke. 2020. Tweeteval: Unified benchmark and comparative evaluation for tweet classification. arXiv:2010.12421. Retrieved from https://arxiv.org/abs/2010.12421
[3]
Rong Cao, Xiangyang Luo, Yaoyi Xi, and Yaqiong Qiao. 2022. Stance detection for online public opinion awareness: An overview. International Journal of Intelligent Systems 37, 12 (2022), 11944–11965.
[4]
Halit ÇETİNER. 2022. Multi-label text analysis with a CNN and LSTM based hybrid deep learning model. Adi̇yaman Üniversitesi Mühendislik Bilimleri Dergisi 9, 17 (2022), 447–457.
[5]
Fang Chen, Zhongliang Yang, and Yongfeng Huang. 2022. A multi-task learning framework for end-to-end aspect sentiment triplet extraction. Neurocomputing 479 (2022), 12–21.
[6]
Gloria del Valle-Cano, Lara Quijano-Sánchez, Federico Liberatore, and Jesús Gómez. 2023. SocialHaterBERT: A dichotomous approach for automatically detecting hate speech on Twitter through textual analysis and user profiles. Expert Systems with Applications 216 (2023), 119446.
[7]
Subhabrata Dutta, Samiya Caur, Soumen Chakrabarti, and Tanmoy Chakraborty. 2022. Semi-supervised stance detection of tweets via distant network supervision. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, 241–251.
[8]
Ankita Gandhi, Kinjal Adhvaryu, Soujanya Poria, Erik Cambria, and Amir Hussain. 2022. Multimodal sentiment analysis: A systematic review of history, datasets, multimodal fusion methods, applications, challenges and future directions. Information Fusion.
[9]
Soumitra Ghosh, Amit Priyankar, Asif Ekbal, and Pushpak Bhattacharyya. 2023. Multitasking of sentiment detection and emotion recognition in code-mixed Hinglish data. Knowledge-Based Systems 260 (2023), 110182.
[10]
Zhiyu Hou and Danping Cao. 2022. Estimating elastic parameters from digital rock images based on multi-task learning with multi-gate mixture-of-experts. Journal of Petroleum Science and Engineering 213 (2022), 110310.
[11]
Myeong-Ha Hwang, Jikang Shin, Hojin Seo, Jeong-Seon Im, Hee Cho, and Chun-Kwon Lee. 2023. Ensemble-NQG-T5: Ensemble neural question generation model based on text-to-text transfer transformer. Applied Sciences 13, 2 (2023), 903.
[12]
V Jothi Prakash and NK Karthikeyan. 2021. Enhanced evolutionary feature selection and ensemble method for cardiovascular disease prediction. Interdisciplinary Sciences: Computational Life Sciences 13, 3 (2021), 389–412.
[13]
Abdalsamad Keramatfar, Hossein Amirkhani, and Amir Jalaly Bidgoly. 2023. Multi-thread hierarchical deep model for context-aware sentiment analysis. Journal of Information Science 49, 1 (2023), 133–144.
[14]
Natt Leelawat, Sirawit Jariyapongpaiboon, Arnon Promjun, Samit Boonyarak, Kumpol Saengtabtim, Ampan Laosunthara, Alfan Kurnia Yudha, and Jing Tang. 2022. Twitter data sentiment analysis of tourism in Thailand during the COVID-19 pandemic using machine learning. Heliyon 8, 10 (2022).
[15]
Jiaqi Ma, Zhe Zhao, Jilin Chen, Ang Li, Lichan Hong, and Ed H Chi. 2019. Snr: Sub-network routing for flexible parameter sharing in multi-task learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, 216–223.
[16]
Sreenivasulu Madichetty and Sreekanth Madisetty. 2023. A RoBERTa based model for identifying the multi-modal informative tweets during disaster. Multimedia Tools and Applications (2023), 1–19.
[17]
Suman Mann, Jyoti Arora, Mudita Bhatia, Ritika Sharma, and Rewangi Taragi. 2023. Twitter sentiment analysis using enhanced BERT. In Intelligent Systems and Applications: Select Proceedings of ICISA 2022. Springer, 263–271.
[18]
Kiran Babu Nelatoori and Hima Bindu Kommanti. 2023. Multi-task learning for toxic comment classification and rationale extraction. Journal of Intelligent Information Systems 60, 2 (2023), 495–519.
[19]
Endang Wahyu Pamungkas, Valerio Basile, and Viviana Patti. 2023. Investigating the role of swear words in abusive language detection tasks. Language Resources and Evaluation 57, 1 (2023), 155–188.
[20]
V Jothi Prakash and NK Karthikeyan. 2022. Dual-layer deep ensemble techniques for classifying heart disease. Information Technology and Control 51, 1 (2022), 158–179.
[21]
Diaz Tiyasya Putra, Erwin Budi Setiawan, et al. 2023. Sentiment Analysis on Social Media with Glove Using Combination CNN and RoBERTa. Journal RESTI (Rekayasa Sistem dan Teknologi Informasi) 7, 3 (2023), 457–563.
[22]
Rukhma Qasim, Waqas Haider Bangyal, Mohammed A Alqarni, and Abdulwahab Ali Almazroi. 2022. A fine-tuned BERT-based transfer learning approach for text classification. Journal of Healthcare Engineering.
[23]
Aneri Rana and Sonali Jha. 2022. Emotion based hate speech detection using multimodal learning. arXiv:2202.06218. Retrieved from https://arxiv.org/abs/2202.06218
[24]
Antonio Reyes, Paolo Rosso, and Davide Buscaldi. 2012. From humor recognition to irony detection: The figurative language of social media. Data & Knowledge Engineering 74 (2012), 1–12.
[25]
Koyyalagunta Krishna Sampath and M Supriya. 2023. Traffic Prediction in Indian Cities from Twitter Data Using Deep Learning and Word Embedding Models. In International Conference on Multi-disciplinary Trends in Artificial Intelligence. Springer, 671–682.
[26]
Anita Saroj and Sukomal Pal. 2023. Ensemble-based domain adaptation on social media posts for irony detection. Multimedia Tools and Applications (2023), 1–20.
[27]
Jose Ramon Saura, Domingo Ribeiro-Soriano, and Pablo Zegarra Saldana. 2022. Exploring the challenges of remote work on Twitter users’ sentiments: From digital technology development to a post-pandemic era. Journal of Business Research 142 (2022), 242–254.
[28]
Arul Antran Vijay Subramanian and Jothi Prakash Venugopal. 2023. A deep ensemble network model for classifying and predicting breast cancer. Computational Intelligence 39, 2 (2023), 258–282.
[29]
Yik Yang Tan, Chee-Onn Chow, Jeevan Kanesan, Joon Huang Chuah, and YongLiang Lim. 2023. Sentiment analysis and sarcasm detection using deep multi-task learning. Wireless Personal Communications 129, 3 (2023), 2213–2237.
[30]
Apoorva Upadhyaya, Marco Fisichella, and Wolfgang Nejdl. 2023. A multi-task model for emotion and offensive aided stance detection of climate change tweets. In Proceedings of the ACM Web Conference 2023, 3948–3958.
[31]
Anil Utku, Umit Can, and Serpil Aslan. 2023. Detection of hateful twitter users with graph convolutional network model. Earth Science Informatics 16, 1 (2023), 329–343.
[32]
Abhishek Velankar, Hrushikesh Patil, Amol Gore, Shubham Salunke, and Raviraj Joshi. 2022. L3cube-mahahate: A tweet-based Marathi hate speech detection dataset and bert models. arXiv:2203.13778. Retrieved from https://arxiv.org/abs/2203.13778
[33]
Lingzhi Wang, Jing Li, Xingshan Zeng, and Kam-Fai Wong. 2022. Successful new-entry prediction for multi-party online conversations via latent topics and discourse modeling. In Proceedings of the ACM Web Conference, 1663–1672.
[34]
Sinan Wang, Yumeng Li, Hongyan Li, Tanchao Zhu, Zhao Li, and Wenwu Ou. 2022a. Multi-task learning with calibrated mixture of insightful experts. In Proceedings of the 2022 IEEE 38th International Conference on Data Engineering (ICDE). IEEE, 3307–3319.
[35]
Shweta Yadav, Jainish Chauhan, Joy Prakash Sain, Krishnaprasad Thirunarayan, Amit Sheth, and Jeremiah Schumm. 2020. Identifying depressive symptoms from tweets: Figurative language enabled multitask learning framework. arXiv:2011.06149. Retrieved from https://arxiv.org/abs/2011.06149
[36]
Tianhua Zhang, Hongyin Luo, Yung-Sung Chuang, Wei Fang, Luc Gaitskell, Thomas Hartvigsen, Xixin Wu, Danny Fox, Helen Meng, and James Glass. 2023. Interpretable unified language checking. arXiv:2304.03728. Retrieved from https://arxiv.org/abs/2304.03728
Index Terms
- A Unified Framework for Analyzing Textual Context and Intent in Social Media
Recommendations
Irony Detection in a Multilingual Context
Advances in Information RetrievalAbstractThis paper proposes the first multilingual (French, English and Arabic) and multicultural (Indo-European languages vs. less culturally close languages) irony detection system. We employ both feature-based models and neural architectures using ...
Cross-lingual Capsule Network for Hate Speech Detection in Social Media
HT '21: Proceedings of the 32nd ACM Conference on Hypertext and Social MediaMost hate speech detection research focuses on a single language, generally English, which limits their generalisability to other languages. In this paper we investigate the cross-lingual hate speech detection task, tackling the problem by adapting the ...
mBERT-GRU multilingual deep learning framework for hate speech detection in social media
One major issue plaguing online social media is hate speech, a complex phenomenon whose identification and target categorization have been studied by the natural language processing community. In recent years, notable studies have been made towards hate ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 19 November 2024
Online AM: 29 July 2024
Accepted: 23 July 2024
Revised: 10 April 2024
Received: 21 September 2023
Published in TIST Volume 15, Issue 6
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 145Total Downloads
- Downloads (Last 12 months)145
- Downloads (Last 6 weeks)61
Reflects downloads up to 21 Nov 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in