Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3314183.3324983acmconferencesArticle/Chapter ViewAbstractPublication PagesumapConference Proceedingsconference-collections
research-article

A Comparison of Word-Embeddings in Emotion Detection from Text using BiLSTM, CNN and Self-Attention

Published: 06 June 2019 Publication History

Abstract

User profiling is becoming increasingly holistic by including aspects of the user that until a few years ago seemed irrelevant. The content that users produce on the Internet and social networks is an essential source of information about their habits, preferences, and behaviors in many situations. One factor that has proved to be very important for obtaining a complete user profile that includes her psychological traits are the emotions experienced. Therefore, it is of great interest to the research community to develop approaches for identifying emotions from the text that are accurate and robust in situations of everyday writing. In this work, we propose a classification approach based on deep neural networks, Bi-LSTM, CNN, and self-attention demonstrating its effectiveness on different datasets. Moreover, we compare three pre-trained word-embeddings for words encoding. The encouraging results obtained on state-of-the-art datasets allow us to confirm the validity of the model and to discuss what are the best word embeddings to adopt for the task of emotion detection. As a consequence of the great importance of deep learning in the research community, we promote our model as a starting point for further investigations in the domain.

References

[1]
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).
[2]
Alexandra Balahur, Jesus M Hermida, and Andres Montoyo. 2012. Building and exploiting emotinet, a knowledge base for emotion detection based on the appraisal theory model. IEEE Transactions on Affective Computing, Vol. 3, 1 (2012), 88--101.
[3]
Pierpaolo Basile, Valerio Basile, Danilo Croce, and Marco Polignano. 2018. Overview of the EVALITA 2018 Aspect-based Sentiment Analysis task (ABSITA). Proceedings of the 6th evaluation campaign of Natural Language Processing and Speech tools for Italian (EVALITA'18), Turin, Italy. CEUR. org (2018).
[4]
Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. Journal of machine learning research, Vol. 3, Feb (2003), 1137--1155.
[5]
Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, Vol. 5 (2017), 135--146.
[6]
Ankush Chatterjee, Kedhar Nath Narahari, Meghana Joshi, and Puneet Agrawal. 2019. SemEval-2019 Task 3: EmoContext: Contextual Emotion Detection in Text. In Proceedings of The 13th International Workshop on Semantic Evaluation (SemEval-2019) . Minneapolis, Minnesota.
[7]
Jianpeng Cheng, Li Dong, and Mirella Lapata. 2016. Long short-term memory-networks for machine reading. arXiv preprint arXiv:1601.06733 (2016).
[8]
Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014).
[9]
Franccois Chollet et almbox. 2015. Keras. https://github.com/fchollet/keras .
[10]
Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning. ACM, 160--167.
[11]
Cicero Dos Santos and Maira Gatti. 2014. Deep convolutional neural networks for sentiment analysis of short texts. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers. 69--78.
[12]
Paul Ekman and Dacher Keltner. 1997. Universal facial expressions of emotion. Segerstrale U, P. Molnar P, eds. Nonverbal communication: Where nature meets culture (1997), 27--46.
[13]
Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. 2016. Deep learning. Vol. 1. MIT press Cambridge.
[14]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780.
[15]
Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. 2014. A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188 (2014).
[16]
Edward Chao-Chun Kao, Chun-Chieh Liu, Ting-Hao Yang, Chang-Tai Hsieh, and Von-Wun Soo. 2009. Towards text-based emotion detection a survey and possible improvements. In Information Management and Engineering, 2009. ICIME'09. International Conference on. IEEE, 70--74.
[17]
Yann LeCun et almbox. 1989. Generalization and network design strategies. Connectionism in perspective (1989), 143--155.
[18]
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. 3111--3119.
[19]
Saif M. Mohammad, Felipe Bravo-Marquez, Mohammad Salameh, and Svetlana Kiritchenko. 2018. SemEval-2018 Task 1: Affect in Tweets. In Proceedings of International Workshop on Semantic Evaluation (SemEval-2018). New Orleans, LA, USA.
[20]
Saif M Mohammad and Svetlana Kiritchenko. 2015. Using hashtags to capture fine emotion categories from tweets. Computational Intelligence, Vol. 31, 2 (2015), 301--326.
[21]
Preslav Nakov, Alan Ritter, Sara Rosenthal, Fabrizio Sebastiani, and Veselin Stoyanov. 2016. SemEval-2016 task 4: Sentiment analysis in Twitter. In Proceedings of the 10th international workshop on semantic evaluation (semeval-2016). 1--18.
[22]
Bo Pang, Lillian Lee, et almbox. 2008. Opinion mining and sentiment analysis. Foundations and Trends® in Information Retrieval, Vol. 2, 1--2 (2008), 1--135.
[23]
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) . 1532--1543.
[24]
Marco Polignano. 2015. The Infuence of User's Emotions in Recommender Systems for Decision Making Processes. In DC@ CHItaly. 58--66.
[25]
Mohammed Abdel Razek and Claude Frasson. 2017. Text-Based Intelligent Learning Emotion System. Journal of Intelligent Learning Systems and Applications, Vol. 9, 01 (2017), 17.
[26]
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. 1985. Learning internal representations by error propagation . Technical Report. California Univ San Diego La Jolla Inst for Cognitive Science.
[27]
James A Russell. 1980. A circumplex model of affect. Journal of personality and social psychology, Vol. 39, 6 (1980), 1161.
[28]
Kashfia Sailunaz, Manmeet Dhaliwal, Jon Rokne, and Reda Alhajj. 2018. Emotion detection from text and speech: a survey. Social Network Analysis and Mining, Vol. 8, 1 (2018), 28.
[29]
Klaus R Scherer and Harald G Wallbott. 1994. Evidence for universality and cultural variation of differential emotion response patterning. Journal of personality and social psychology, Vol. 66, 2 (1994), 310.
[30]
Marko Tkalcic, Andrej Kosir, and Jurij Tasic. 2011. Affective recommender systems: the role of emotions in recommender systems. In Proc. The RecSys 2011 Workshop on Human Decision Making in Recommender Systems. Citeseer, 9--13.
[31]
Vincent Van Asch. 2013. Macro-and micro-averaged evaluation measures {{basic draft}}. Belgium: CLiPS (2013).
[32]
Han Zhang, Ian Goodfellow, Dimitris Metaxas, and Augustus Odena. 2018a. Self-attention generative adversarial networks. arXiv preprint arXiv:1805.08318 (2018).
[33]
Lei Zhang, Shuai Wang, and Bing Liu. 2018c. Deep learning for sentiment analysis: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, Vol. 8, 4 (2018), e1253.
[34]
Ziqi Zhang, David Robinson, and Jonathan Tepper. 2018b. Detecting Hate Speech on Twitter Using a Convolution-GRU Based Deep Neural Network. In European Semantic Web Conference. Springer, 745--760.
[35]
Xu Zhe and AC Boucouvalas. 2002. Text-to-emotion engine for real time internet communication. In Proceedings of International Symposium on Communication Systems, Networks and DSPs. Citeseer, 164--168.
[36]
Guineng Zheng, Subhabrata Mukherjee, Xin Luna Dong, and Feifei Li. 2018. OpenTag: Open attribute value extraction from product profiles. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 1049--1058.

Cited By

View all
  • (2024)Meta-Learning in Textual Sentiment and Emotion Analysis: A Comprehensive Review2024 MIT Art, Design and Technology School of Computing International Conference (MITADTSoCiCon)10.1109/MITADTSoCiCon60330.2024.10574942(1-8)Online publication date: 25-Apr-2024
  • (2024)Evaluating Public Opinion on the 2024 Indonesian Presidential Election Candidate: An IndoBERT Approach to Twitter Sentiment Analysis2024 10th International Conference on Smart Computing and Communication (ICSCC)10.1109/ICSCC62041.2024.10690796(88-94)Online publication date: 25-Jul-2024
  • (2024)Medication Recommendation System for Skin Diseases using NLP2024 International Conference on Advancements in Smart, Secure and Intelligent Computing (ASSIC)10.1109/ASSIC60049.2024.10507908(1-7)Online publication date: 27-Jan-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
UMAP'19 Adjunct: Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization
June 2019
455 pages
ISBN:9781450367110
DOI:10.1145/3314183
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 June 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. deep learning
  2. emotion detection
  3. natural language processing
  4. sentiment analysis
  5. text analysis
  6. word embeddings

Qualifiers

  • Research-article

Funding Sources

  • H2020 Marie Sk?odowska-Curie Actions

Conference

UMAP '19
Sponsor:

Acceptance Rates

UMAP'19 Adjunct Paper Acceptance Rate 30 of 122 submissions, 25%;
Overall Acceptance Rate 162 of 633 submissions, 26%

Upcoming Conference

UMAP '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)74
  • Downloads (Last 6 weeks)12
Reflects downloads up to 20 Nov 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Meta-Learning in Textual Sentiment and Emotion Analysis: A Comprehensive Review2024 MIT Art, Design and Technology School of Computing International Conference (MITADTSoCiCon)10.1109/MITADTSoCiCon60330.2024.10574942(1-8)Online publication date: 25-Apr-2024
  • (2024)Evaluating Public Opinion on the 2024 Indonesian Presidential Election Candidate: An IndoBERT Approach to Twitter Sentiment Analysis2024 10th International Conference on Smart Computing and Communication (ICSCC)10.1109/ICSCC62041.2024.10690796(88-94)Online publication date: 25-Jul-2024
  • (2024)Medication Recommendation System for Skin Diseases using NLP2024 International Conference on Advancements in Smart, Secure and Intelligent Computing (ASSIC)10.1109/ASSIC60049.2024.10507908(1-7)Online publication date: 27-Jan-2024
  • (2024)Challenges and Opportunities of Text-Based Emotion Detection: A SurveyIEEE Access10.1109/ACCESS.2024.335635712(18416-18450)Online publication date: 2024
  • (2024)A Transfer-Based Deep Learning Model for Persian Emotion ClassificationMultimedia Tools and Applications10.1007/s11042-024-19668-wOnline publication date: 4-Jul-2024
  • (2024)Leveraging distant supervision and deep learning for twitter sentiment and emotion classificationJournal of Intelligent Information Systems10.1007/s10844-024-00845-062:4(1045-1070)Online publication date: 22-Mar-2024
  • (2024)A review on emotion detection by using deep learning techniquesArtificial Intelligence Review10.1007/s10462-024-10831-157:8Online publication date: 11-Jul-2024
  • (2024)A Review on Emotion Detection from Text: Opportunities and ChallengesProceedings of Trends in Electronics and Health Informatics10.1007/978-981-97-3937-0_2(17-31)Online publication date: 17-Oct-2024
  • (2023)Survey of Accuracy Prediction on the PlantVillage Dataset using different ML techniquesEAI Endorsed Transactions on Internet of Things10.4108/eetiot.457810Online publication date: 12-Dec-2023
  • (2023)Emotion Recognition From Text Using Multi-Head Attention-Based Bidirectional Long Short-Term Memory Architecture Using Multi-Level ClassificationAdvanced Applications of NLP and Deep Learning in Social Media Data10.4018/978-1-6684-6909-5.ch005(92-113)Online publication date: 9-Jun-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media