abstract

Negative Emotions, Positive Experience: What Are We Doing Wrong When Evaluating the UX?

Authors:

Walter T. Nakamura,

Elaine H. T. de Oliveira,

Tayana ConteAuthors Info & Claims

CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

Paper No.: LBW0281, Pages 1 - 6

https://doi.org/10.1145/3290607.3313000

Published: 02 May 2019 Publication History

Abstract

Since its idealization more than 20 years ago, much research has been carried out on the User eXperience (UX) field, with several evaluation methods being proposed. However, studies have been pointing out conflicting results when evaluating the UX. Users frequently evaluate their UX as positive, even when experiencing many negative emotions when interacting with a product. Moreover, variables such as the peak-end effect and the memory-experience gap may also have been influencing the results, leading to misinterpretations about the product's quality. In this context, this paper presents our work-in-progress research on the following research question: "What are we doing wrong when evaluating the UX?". We discuss about different variables that may have been influencing users' perceptions about their experiences in previous studies and highlight research opportunities. With this work, we expect to shed light on and bring reflection to current practices on UX evaluations in order to progress the research in the UX field.

Supplementary Material

MP4 File (lbw0281p.mp4)

Preview video

Download
1.58 MB

References

[1]

ISO 9241--210. 2010. Ergonomics of Human-system Interaction: Part 210: Human-centred Design for Interactive Systems.

[2]

Javier A Bargas-Avila and Kasper Hornb. 2011. Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 10.

Digital Library

[3]

Anders Bruun and Simon Ahm. 2015. Mind the Gap! Comparing Retrospective and Concurrent Ratings of Emotion in User Experience Evaluation. In Human-Computer Interaction -- INTERACT 2015, Julio Abascal, Simone Barbosa, Mirko Fetter, Tom Gross, Philippe Palanque, and Marco Winckler (Eds.). Vol. 9296. Springer International Publishing, Cham, 237--254.

[4]

Margaret Burnett, Simone Stumpf, Jamie Macbeth, Stephann Makri, Laura Beckwith, Irwin Kwan, Anicia Peters, and William Jernigan. 2016. GenderMag: A Method for Evaluating Software's Gender Inclusiveness. Interacting with Computers 28, 6 (Nov. 2016), 760--787.

[5]

Andy Cockburn, Philip Quinn, and Carl Gutwin. 2015. Examining the Peak-End Effects of Subjective Experience. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI '15. ACM Press, Seoul, Republic of Korea, 357--366.

Digital Library

[6]

Giulia de Andrade Cardieri and Luciana Martinez Zaina. 2018. Analyzing User Experience in Mobile Web, Native and Progressive Web Applications: A User and HCI Specialist Perspectives. In Proceedings of the 17th Brazilian Symposium on Human Factors in Computing Systems - IHC 2018. ACM Press, Belém, Brazil, 1--11.

Digital Library

[7]

Marc Hassenzahl. 2018. The Thing and I (Summer of '17 Remix). In Funology 2, Mark Blythe and Andrew Monk (Eds.). Springer International Publishing, Cham, 17--31.

[8]

Sari Kujala and Talya Miron-Shatz. 2015. The evolving role of expectations in long-term user experience. In Proceedings of the 19th International Academic Mindtrek Conference on - AcademicMindTrek '15. ACM Press, Tampere, Finland, 167--174.

Digital Library

[9]

Carine Lallemand, Guillaume Gronier, and Vincent Koenig. 2015. User experience: A concept without consensus? Exploring practitioners' perspectives through an international survey. Computers in Human Behavior 43 (Feb. 2015), 35--48.

Digital Library

[10]

Effie Lai-Chong Law, Virpi Roto, Marc Hassenzahl, Arnold POS Vermeeren, and Joke Kort. 2009. Understanding, scoping and defining user experience: a survey approach. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 719--728.

Digital Library

[11]

Effie Lai-Chong Law, Arnold POS Vermeeren, Marc Hassenzahl, and Mark Blythe. 2007. Towards a UX manifesto. In Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI... but not as we know it-Volume 2. BCS Learning & Development Ltd., 205--206.

Digital Library

[12]

Craig M. MacDonald and Michael E. Atwood. 2014. What does it mean for a system to be useful?: an exploratory study of usefulness. In Proceedings of the 2014 conference on Designing interactive systems - DIS '14. ACM Press, Vancouver, BC, Canada, 885--894.

Digital Library

[13]

Leonardo Marques, Walter Nakamura, Natasha Valentim, Luis Rivero, and Tayana Conte. 2018. Do Scale Type Techniques Identify Problems that Affect User eXperience? User Experience Evaluation of a Mobile Application (S). In The 30th International Conference on Software Engineering and Knowledge Engineering. 451--501.

[14]

Oussama Metatla, Nuno N. Correia, Fiore Martin, Nick Bryan-Kinns, and Tony Stockman. 2016. Tap the ShapeTones: Exploring the Effects of Crossmodal Congruence in an Audio-Visual Interface. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI '16. ACM Press, Santa Clara, California, USA, 1055--1066.

Digital Library

[15]

Talya Miron-Shatz, Arthur Stone, and Daniel Kahneman. 2009. Memories of yesterday's emotions: Does the valence of experience affect the memory-experience gap? Emotion 9, 6 (2009), 885--891.

[16]

Walter Nakamura, Leonardo Marques, Luis Rivero, Elaine Oliveira, and Tayana Conte. 2017. Are Generic UX Evaluation Techniques Enough? A study on the UX Evaluation of the Edmodo Learning Management System. In Brazilian Symposium on Computers in Education. Recife, Pernambuco, Brasil, 1007.

[17]

Ingrid Pettersson, Florian Lachner, Anna-Katharina Frison, Andreas Riener, and Andreas Butz. 2018. A Bermuda Triangle?. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI '18. ACM Press, Montreal QC, Canada, 1--16.

Digital Library

[18]

Samaneh Soleimani and Effie Lai-Chong Law. 2017. What Can Self-Reports and Acoustic Data Analyses on Emotions Tell Us?. In Proceedings of the 2017 Conference on Designing Interactive Systems - DIS '17. ACM Press, Edinburgh, United Kingdom, 489--501.

Digital Library

[19]

Arnold POS Vermeeren, Effie Lai-Chong Law, Virpi Roto, Marianna Obrist, Jettie Hoonhout, and Kaisa Väänänen-VainioMattila. 2010. User experience evaluation methods: current state and development needs. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries. ACM, 521--530.

Digital Library

Cited By

Nakamura WMarques LRedmiles Dde Oliveira EConte T(2022)Investigating the Influence of Different Factors on the UX Evaluation of a Mobile ApplicationInternational Journal of Human–Computer Interaction10.1080/10447318.2022.210865839:20(3948-3968)Online publication date: 16-Aug-2022
https://doi.org/10.1080/10447318.2022.2108658
Nakamura Wde Oliveira Ede Oliveira ERedmiles DConte T(2022)What factors affect the UX in mobile apps? A systematic mapping study on the analysis of app store reviewsJournal of Systems and Software10.1016/j.jss.2022.111462193:COnline publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1016/j.jss.2022.111462
Nakamura WAhmed IRedmiles DOliveira EFernandes Dde Oliveira EConte T(2021)Are UX Evaluation Methods Providing the Same Big Picture?Sensors10.3390/s2110348021:10(3480)Online publication date: 17-May-2021
https://doi.org/10.3390/s21103480
Show More Cited By

Index Terms

Negative Emotions, Positive Experience: What Are We Doing Wrong When Evaluating the UX?
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

Negative Emotions, Positive Outcomes? Exploring the Communication of Negativity in Serious Data Stories
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

Recent work has highlighted that emotion is key to the user experience with data stories. However, limited attention has been paid to negative emotions specifically. This work investigates the outcomes of negative emotions in the context of serious data ...
Negative Emotion, Positive Experience?: Emotionally Moving Moments in Digital Games
CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

Emotions are key to the player experience (PX) and interest in the potential of games to provide unique emotional, sometimes uncomfortable experiences is growing. Yet there has been little empirical investigation of what game experiences players ...
The experience of interactive storytelling: comparing “fahrenheit” with “façade”
ICEC'11: Proceedings of the 10th international conference on Entertainment Computing

At the intersection of multimedia, artificial intelligence, and gaming technology, new visions of future entertainment media arise that approximate the “Holodeck” ® idea of interactive storytelling. We report exploratory experiments on the user ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

May 2019

3673 pages

ISBN:9781450359719

DOI:10.1145/3290607

General Chairs:
Stephen Brewster
University of Glasgow, Scotland, UK
,
Geraldine Fitzpatrick
TU Wien, Austria
,
Program Chairs:
Anna Cox
University College London, UK
,
Vassilis Kostakos
University of Melbourne, Australia

Copyright © 2019 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Check for updates

Author Tags

Qualifiers

Abstract

Funding Sources

Conference

CHI '19

Sponsor:

SIGCHI

CHI '19: CHI Conference on Human Factors in Computing Systems

May 4 - 9, 2019

Glasgow, Scotland Uk

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
469
Total Downloads

Downloads (Last 12 months)50
Downloads (Last 6 weeks)5

Reflects downloads up to 22 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Nakamura WMarques LRedmiles Dde Oliveira EConte T(2022)Investigating the Influence of Different Factors on the UX Evaluation of a Mobile ApplicationInternational Journal of Human–Computer Interaction10.1080/10447318.2022.210865839:20(3948-3968)Online publication date: 16-Aug-2022
https://doi.org/10.1080/10447318.2022.2108658
Nakamura Wde Oliveira Ede Oliveira ERedmiles DConte T(2022)What factors affect the UX in mobile apps? A systematic mapping study on the analysis of app store reviewsJournal of Systems and Software10.1016/j.jss.2022.111462193:COnline publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1016/j.jss.2022.111462
Nakamura WAhmed IRedmiles DOliveira EFernandes Dde Oliveira EConte T(2021)Are UX Evaluation Methods Providing the Same Big Picture?Sensors10.3390/s2110348021:10(3480)Online publication date: 17-May-2021
https://doi.org/10.3390/s21103480
Coelho BAndrade RDarin T(2021)Not the Same Everywhere: Comparing the Scope and Definition of User Experience between the Brazilian and International CommunitiesInternational Journal of Human–Computer Interaction10.1080/10447318.2021.196072738:7(595-613)Online publication date: 10-Aug-2021
https://doi.org/10.1080/10447318.2021.1960727
Makama Nvan Staden CBotha A(2021)User Experience Framework for Teachers Using Mobile Technologies in Resource-Constrained EnvironmentsProceedings of Sixth International Congress on Information and Communication Technology10.1007/978-981-16-2377-6_24(239-252)Online publication date: 24-Sep-2021
https://doi.org/10.1007/978-981-16-2377-6_24
Chromik MLachner FButz ALamas DSarapuu HŠmorgun IBerget G(2020)ML for UX? - An Inventory and Predictions on the Use of Machine Learning Techniques for UX ResearchProceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society10.1145/3419249.3420163(1-11)Online publication date: 25-Oct-2020
https://dl.acm.org/doi/10.1145/3419249.3420163
Melo BDarin TGuimaraes RSaibel Santos Cde Santana VKronbauer A(2019)Scope and definition of user experience in BrazilProceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems10.1145/3357155.3358444(1-11)Online publication date: 22-Oct-2019
https://dl.acm.org/doi/10.1145/3357155.3358444

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents