research-article

Do all facial emojis communicate emotion? The impact of facial emojis on perceived sender emotion and text processing

Authors:

Valeria A. Pfeifer,

Emma L. Armstrong,

Vicky Tzuyin LaiAuthors Info & Claims

Volume 126, Issue C

https://doi.org/10.1016/j.chb.2021.107016

Published: 01 January 2022 Publication History

Abstract

Facial emojis can express a variety of positive and negative emotions, and are commonly used in digital, written communication. However, little is known about how emojis impact text processing and how different emoji-text combinations give rise to a sender's mental state. In this study, we investigated how facial emojis with positive valence (= happy emojis) and facial emojis with negative valence (= upset emojis) embedded in emotionally ambiguous/neutral text affect the perceived mental state of the sender using ratings (Experiment 1) and the processing of the text messages using Event-Related Potentials (Experiment 2). We predicted that (1) the same text message with happy and upset emojis would convey different sender mental states, and (2) emoji valence would affect the processing of subsequent text in valence-specific ways. Our Experiment 1 results showed that while texts with upset emojis convey specific sender mental states, texts with happy emojis convey positive emotion more generally, with no further differentiation between emojis. In ERPs (Experiment 2), we found that emojis affect subsequent text processing at N400, and emoji valence affects processing downstream at the second word. We concluded that all facial-emojis impact text processing, but happy and upset emojis carry differential social-emotional salience and impact text processing differently when content becomes available.

Highlights

•

We investigated how facial emojis impact text comprehension using behavioral ratings and EEG.

•

In ratings, negative emojis communicate specific mental states, while positive emojis communicate general positive emotion.

•

In ERPs, the first words preceded by emojis elicited a larger N400 compared to the first words preceded by commas.

•

In ERPs, the second words after emojis showed valence-specific word processing, as reflected by LPCs.

•

Positive emojis maintain conversation flow, negative emojis impact text processing by providing emotional frame for text.

References

[1]

N. Aldunate, R. González-Ibáñez, An integrated review of emoticons in computer-mediated communication, Frontiers in Psychology 7 (2017) 2061.

[2]

Q. Bai, Q. Dan, Z. Mu, M. Yang, A systematic review of emoji: Current research and future perspectives, Frontiers in Psychology 10 (2019) 2221.

[3]

L.F. Barrett, R. Adolphs, S. Marsella, A.M. Martinez, S.D. Pollak, Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements, Psychological Science in the Public Interest 20 (1) (2019) 1–68.

[4]

M. Bayer, W. Sommer, A. Schacht, Reading emotional words within sentences: The impact of arousal and valence on event-related potentials, International Journal of Psychophysiology 78 (3) (2010) 299–307.

[5]

I. Boutet, M. LeBlanc, J.A. Chamberland, C.A. Collin, Emojis influence emotional communication, social attributions, and information processing, Computers in Human Behavior 119 (2021) 106722.

[6]

C. Brown, P. Hagoort, The processing nature of the N400: Evidence from masked priming, Journal of Cognitive Neuroscience 5 (1) (1993) 34–44.

[7]

F.M. Citron, Neural correlates of written emotion word processing: A review of recent electrophysiological and hemodynamic neuroimaging studies, Brain and Language 122 (3) (2012) 211–226.

[8]

N. Delaney-Busch, G. Wilkie, G. Kuperberg, Vivid: How valence and arousal influence word processing under different task demands, Cognitive, Affective, & Behavioral Neuroscience 16 (3) (2016) 415–432.

[9]

D. Derks, A.E. Bos, J. Von Grumbkow, Emoticons and social interaction on the internet: The importance of social context, Computers in Human Behavior 23 (1) (2007) 842–849.

[10]

G. Egidi, H.C. Nusbaum, Emotional language processing: How mood affects integration processes during discourse comprehension, Brain and Language 122 (3) (2012) 199–210.

[11]

P. Ekman, W.V. Friesen, Constants across cultures in the face and emotion, Journal of Personality and Social Psychology 17 (2) (1971) 124.

[12]

Emojipedia. https://emojipedia.org/.

[13]

Emojitracker. https://emojitracker.com/.

[14]

C. Gantiva, M. Sotaquirá, A. Araujo, P. Cuervo, Cortical processing of human and emoji faces: An ERP analysis, Behaviour & Information Technology 39 (8) (2020) 935–943.

[15]

J.A. Hinojosa, E.M. Moreno, P. Ferré, Affective neurolinguistics: Towards a framework for reconciling language and emotion, Language, Cognition and Neuroscience 35 (7) (2020) 813–839.

[16]

D.J. Holt, S.K. Lynn, G.R. Kuperberg, Neurophysiological correlates of comprehending emotional meaning in context, Journal of Cognitive Neuroscience 21 (11) (2009) 2245–2262.

[17]

H.E. Howman, R. Filik, The role of emoticons in sarcasm comprehension in younger and older adults: Evidence from an eye-tracking experiment, Quarterly Journal of Experimental Psychology 73 (11) (2020) 1729–1744.

[18]

A.H. Huang, D.C. Yen, X. Zhang, Exploring the potential effects of emoticons, Information & Management 45 (7) (2008) 466–473.

Digital Library

[19]

Kaye, L.K.; Rodríguez-Cuadrado, S.; Jones, B.R.; Malone, S.A.; Wall, H.J. (2020): Do emoji enhance emotional processing of written language?. Preprint https://doi.org/10.31219/osf.io/p7vnq.

[20]

R. Kelly, L. Watts, Characterising the inventive appropriation of emoji as relationally meaningful in mediated close personal relationships, Vol. 2, Experiences of technology appropriation: unanticipated users, usage, circumstances, and design, 2015.

[21]

S. Kelly, C. Kravitz, M. Hopkins, Neural correlates of bimodal speech and gesture comprehension, Brain and Language 89 (1) (2004) 253–260,.

[22]

J. Kissler, C. Herbert, I. Winkler, M. Junghofer, Emotion and attention in visual word processing—an ERP study, Biological Psychology 80 (1) (2009) 75–83.

[23]

M. Kutas, K.D. Federmeier, Thirty years and counting: Finding meaning in the N400 component of the event-related brain potential (ERP), Annual Review of Psychology 62 (2011) 621–647.

[24]

E.F. Lau, C. Phillips, D. Poeppel, A cortical network for semantics:(de) constructing the N400, Nature Reviews Neuroscience 9 (12) (2008) 920–933.

[25]

S.K. Lo, The nonverbal communication functions of emoticons in computer-mediated communication, CyberPsychology and Behavior 11 (5) (2008) 595–597.

[26]

S.J. Luck, N. Gaspelin, How to get statistically significant effects in any ERP experiment (and why you shouldn't), Psychophysiology 54 (1) (2017) 146–157,.

[27]

P.K. Novak, J. Smailović, B. Sluban, I. Mozetič, Sentiment of emojis, PloS One 10 (12) (2015).

[28]

V. Pfeifer, V.T. Lai, The comprehension of irony in high and low emotional contexts, Canadian Journal of Experimental Psychology/Revue canadienne de psychologie expérimentale. 75 (2) (2021) 120–125,.

[29]

M.A. Riordan, The communicative role of non-face emojis: Affect and disambiguation, Computers in Human Behavior 76 (2017) 75–86.

[30]

C.M. Robus, C.J. Hand, R. Filik, M. Pitchford, Investigating effects of emoji on neutral narrative text: Evidence from eye movements and perceived emotional valence, Computers in Human Behavior (2020) 106361.

[31]

D. Rodrigues, M. Prada, R. Gaspar, M.V. Garrido, D. Lopes, Lisbon Emoji and Emoticon Database (LEED): Norms for emoji and emoticons in seven evaluative dimensions, Behavior Research Methods 50 (1) (2018) 392–405.

[32]

A.S. Rotaru, G. Vigliocco, Constructing semantic models from words, images, and emojis, Cognitive Science 44 (4) (2020).

[33]

D. Thompson, I.G. Mackenzie, H. Leuthold, R. Filik, Emotional responses to irony and emoticons in written language: Evidence from EDA and facial EMG, Psychophysiology 53 (7) (2016) 1054–1062.

[34]

J.J. Van Berkum, B. Holleman, M. Nieuwland, M. Otten, J. Murre, Right or wrong? The brain's fast response to morally objectionable statements, Psychological Science 20 (9) (2009) 1092–1099.

[35]

D. Van den Brink, J.J. Van Berkum, M.C. Bastiaansen, C.M. Tesink, M. Kos, J.K. Buitelaar, P. Hagoort, Empathy matters: ERP evidence for inter-individual differences in social language processing, Social Cognitive and Affective Neuroscience 7 (2) (2012) 173–183.

[36]

M.W. Verhees, D.J. Chwilla, J. Tromp, C.T. Vissers, Contributions of emotional state and attention to the processing of syntactic agreement errors: Evidence from P600, Frontiers in Psychology 6 (2015) 388.

[37]

C.T.W. Vissers, D. Virgillito, D.A. Fitzgerald, A.E. Speckens, I. Tendolkar, I. van Oostrom, D.J. Chwilla, The influence of mood on the processing of syntactic anomalies: Evidence from P600, Neuropsychologia 48 (12) (2010) 3521–3531.

[38]

B. Weissman, D. Tanner, A strong wink between verbal and emoji-based irony: How the brain processes ironic emojis during language comprehension, PloS One 13 (8) (2018).

[39]

J. Yang, Y. Yang, L. Xiu, G. Yu, Effect of emoji prime on the understanding of emotional words–evidence from ERPs, Behaviour & Information Technology (2021) 1–10.

Cited By

Glikson ERiordan M(2024)Beneficial outcomes of (appropriate) nonverbal displays of negative affect in virtual teamsComputers in Human Behavior10.1016/j.chb.2024.108165154:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.chb.2024.108165
Liegl SFurtner M(2024)Emotional leader communication in the digital ageComputers in Human Behavior10.1016/j.chb.2024.108148154:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.chb.2024.108148
Li YZhang DLiu XGuo D(2024)Processing facial emojis as social informationComputers in Human Behavior10.1016/j.chb.2023.108106153:COnline publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1016/j.chb.2023.108106
Show More Cited By

Index Terms

Do all facial emojis communicate emotion? The impact of facial emojis on perceived sender emotion and text processing

Index terms have been assigned to the content through auto-classification.

Recommendations

Emojis influence emotional communication, social attributions, and information processing
Abstract
Many emojis symbolize nonverbal cues that are used during face-to-face communication. Despite their popularity, few studies have examined how emojis influence digital interactions. The present study addresses this gap by measuring the ...
Highlights
- Emojis convey information about the sender's affect.
- Senders that use positive ...
The communicative role of non-face emojis

Emojis have evolved from imitations of facial expressions meant to communicate affect into pictures of objects, food, and places that are not directly linked to affect. While emojis that resemble facial expressions are well-researched, emojis that ...
Exploring Emojis for Emotion Recognition in Portuguese Text
Progress in Artificial Intelligence
Abstract
New forms of communication, like emojis, are frequent today in social media. Having in mind their strong connection with expressed emotions, we exploit emojis towards the creation of a model for emotion recognition in Portuguese. We gather short ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Computers in Human Behavior

Computers in Human Behavior Volume 126, Issue C

Jan 2022

510 pages

ISSN:0747-5632

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 January 2022

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Glikson ERiordan M(2024)Beneficial outcomes of (appropriate) nonverbal displays of negative affect in virtual teamsComputers in Human Behavior10.1016/j.chb.2024.108165154:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.chb.2024.108165
Liegl SFurtner M(2024)Emotional leader communication in the digital ageComputers in Human Behavior10.1016/j.chb.2024.108148154:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.chb.2024.108148
Li YZhang DLiu XGuo D(2024)Processing facial emojis as social informationComputers in Human Behavior10.1016/j.chb.2023.108106153:COnline publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1016/j.chb.2023.108106
Teh PRayson PPak IPiao SHo JMoore ACheah Y(2022)Textual variations affect human judgements of sentiment valuesElectronic Commerce Research and Applications10.1016/j.elerap.2022.10114953:COnline publication date: 1-May-2022
https://dl.acm.org/doi/10.1016/j.elerap.2022.101149

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents