research-article

Usefulness of long-term user experience evaluation to product development: practitioners' views from three case studies

Authors:

Jari Varsaluoma,

Farrukh SaharAuthors Info & Claims

NordiCHI '14: Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational

Pages 79 - 88

https://doi.org/10.1145/2639189.2641218

Published: 26 October 2014 Publication History

Abstract

Understanding the temporal aspects of user experience (UX) has received increasing attention in the HCI community. However, little empirical evidence is available on how practitioners in product development companies evaluate the usefulness or actually use long-term UX evaluation data in their work. In this study, we explore how practitioners (e.g., managers, designers and UX specialists) evaluate the usefulness of long-term UX evaluation results to their own work. Three case studies were conducted with longitudinal and retrospective methods in a company developing interactive sports products. Our findings suggest that long-term UX evaluation provides results that are perceived as interesting, relevant and useful by practitioners. Potential uses for the results were e.g., verifying practitioners' expectations, planning future work, understanding changes in UX, the development of future products, and updating current software products. Future research should focus on how to provide long-term UX evaluation results in more efficient manner to benefit product development.

References

[1]

Bark, I., Følstad, A., and Gulliksen, J. Use and usefulness of HCI methods: results from an exploratory study among Nordic HCI practitioners. People and Computers XIX --- The Bigger Picture, (2006), 201--217.

[2]

Csikszentmihalyi, M., and Larson, R. Validity and Reliability of the Experience-Sampling Method. Journal of Nervous and Mental Disease, 175 (1987), 526--537

[3]

Hassenzahl, M., Burmester, M., and Koller F. AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität In: J. Ziegler & G. Szwillus (Hrsg.), Mensch & Computer 2003. Interaktion in Bewegung, (2003) 187--196.

[4]

Jain, J., Rosenbaum, S., and Courage, C. Best practices in longitudinal research. Ext. Abstracts CHI '10, (2010), 3167--3170.

Digital Library

[5]

Kahneman, D., Krueger, A. B., Schkade, D. A., Schwarz, N., and Stone, A. A. A Survey Method for Characterizing Daily Life Experience: The Day Reconstruction Method. Science 306, 5702 (2004), 1776--1780.

[6]

Karapanos, E., Martens, J.-B., and Hassenzahl, M. Reconstructing experiences with iScale. Int. J. Human-Computer Studies 70, 11 (2012), 849--865.

Digital Library

[7]

Karapanos, E., Jain, J., and Hassenzahl, M. Theories, methods and case studies of longitudinal HCI research. Ext. Abstracts CHI'12, (2012), 2727--2730.

Digital Library

[8]

Kujala, S. and Miron-Shatz, T. Emotions, experiences and usability in real-life mobile phone use. Proc. CHI '13, (2013), 1061--1070.

Digital Library

[9]

Kujala, S., Roto, V., Väänänen-Vainio-Mattila, K., Karapanos, E., and Sinnelä, A. UX Curve: A method for evaluating long-term user experience. Interacting with Computers 23, 5 (2011), 473--483.

Digital Library

[10]

Kujala, S., Vogel, M., Pohlmeyer, A., and Obrist, M. Lost in Time: The Meaning of Temporal Aspects in User Experience. Ext. Abstracts CHI '13, (2013), 559--564.

Digital Library

[11]

Mendoza, V. and Novick, D. Usability over time. Proc. SIGDOC '05, (2005), 151--158.

Digital Library

[12]

Norman, D. A. Memory is More Important than Actuality. Interactions, March + April (2009), 24--26.

[13]

Novick, D. and Santaella, B. Short-term methodology for long-term usability. Proc. SIGDOC '12, (2012), 205--211.

Digital Library

[14]

Ployhart, R. E., and Ward, A.-K. The "Quick Start Guide" for Conducting and Publishing Longitudinal Research. J. Bus. Psychol. 26, 4 (2011), 413--422.

[15]

Vaughan, M. and Courage, C. SIG: capturing longitudinal usability: what really affects user performance over time? Ext. Abstracts CHI '07, (2007), 2149--2152.

Digital Library

[16]

Vaughan, M., Courage, C., and Rosenbaum, S. Longitudinal usability data collection: art versus science? Ext. Abstracts CHI '08, (2008), 2261--2264.

Digital Library

[17]

von Wilamowitz-Moellendorff, M., Hassenzahl, M., and Platz, A. (2006). Dynamics of user experience: How the perceived quality of mobile phones changes over time. In User Experience -- Towards a unified view, Workshop at the 4th Nordic Conference on Human-Computer Interaction, 74--78.

Cited By

Bianchini LZaina L(2024)Which UX Research method or technique should I take? Assisting software startup professionals in making their choicesJournal on Interactive Systems10.5753/jis.2024.402815:1(762-789)Online publication date: 21-Jul-2024
https://doi.org/10.5753/jis.2024.4028
Martinelli Sda Silva TZaina L(2024)Challenges of UX research and long-term UX: A multiple case study with software startupsProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685381(1-11)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685381
López Moreno MVivas LRevollo Sarmiento EVivas J(2024)A Tailor-Made Web Lab for Older Adults: User-Centered Redesign of a Cognitive Stimulation Platform Based on User ExperienceInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2434768(1-12)Online publication date: 4-Dec-2024
https://doi.org/10.1080/10447318.2024.2434768
Show More Cited By

Index Terms

Usefulness of long-term user experience evaluation to product development: practitioners' views from three case studies
1. Human-centered computing

Recommendations

Identifying hedonic factors in long-term user experience
DPPI '11: Proceedings of the 2011 Conference on Designing Pleasurable Products and Interfaces

User experience (UX) arises from the user's interaction with a product and its pragmatic and hedonic (pleasure) qualities. Until recently, UX evaluation has focused mainly on examining short-term experiences. However, as the user-product relationship ...
User Experience Evaluation Methods in Product Development (UXEM'09)
INTERACT '09: Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part II

High quality user experience (UX) has become a central competitive factor of product development in mature consumer markets [1]. Although the term UX originated from industry and is a widely used term also in academia, the tools for managing UX in ...
Approaches to manage the user experience process in Agile software development: A systematic literature review
Abstract Context:
Software development companies use Agile methods to develop their products or services efficiently and in a goal-oriented way. But this alone is not enough to satisfy user demands today. It is much more important ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

NordiCHI '14: Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational

October 2014

361 pages

ISBN:9781450325424

DOI:10.1145/2639189

General Chairs:
Virpi Roto,
Jonna Häkkilä,
Program Chairs:
Kaisa Väänänen-Vainio-Mattila,
Oskar Juhlin,
Publications Chairs:
Thomas Olsson,
Ebba Hvannberg

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 October 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

NordiCHI '14

NordiCHI '14: The 8th Nordic Conference on Human-Computer Interaction

October 26 - 30, 2014

Helsinki, Finland

Acceptance Rates

NordiCHI '14 Paper Acceptance Rate 89 of 361 submissions, 25%;

Overall Acceptance Rate 379 of 1,572 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
355
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bianchini LZaina L(2024)Which UX Research method or technique should I take? Assisting software startup professionals in making their choicesJournal on Interactive Systems10.5753/jis.2024.402815:1(762-789)Online publication date: 21-Jul-2024
https://doi.org/10.5753/jis.2024.4028
Martinelli Sda Silva TZaina L(2024)Challenges of UX research and long-term UX: A multiple case study with software startupsProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685381(1-11)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685381
López Moreno MVivas LRevollo Sarmiento EVivas J(2024)A Tailor-Made Web Lab for Older Adults: User-Centered Redesign of a Cognitive Stimulation Platform Based on User ExperienceInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2434768(1-12)Online publication date: 4-Dec-2024
https://doi.org/10.1080/10447318.2024.2434768
Martinelli SLopes LZaina L(2024)UX Research practices related to Long-Term UXInformation and Software Technology10.1016/j.infsof.2024.107431170:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.infsof.2024.107431
Guerino GMartinelli SChoma JLeal GBalancieri RZaina L(2023)Perceptions about Usefulness and Attitudes toward UX work: a survey with software startup Brazilian professionalsProceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems10.1145/3638067.3638069(1-12)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3638067.3638069
Qiu HLieb AChou JCarneal MMok JAmspoker EVasilescu BDabbish L(2023)Climate Coach: A Dashboard for Open-Source Maintainers to Overview Community DynamicsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581317(1-18)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581317
Nielsen SOrdoñez RSkov MJochum E(2023)Strategies for strengthening UX competencies and cultivating corporate UX in a large organisation developing robotsBehaviour & Information Technology10.1080/0144929X.2023.222728443:9(1769-1797)Online publication date: 28-Jun-2023
https://doi.org/10.1080/0144929X.2023.2227284
Baskan AGoncu-Berk G(2022)User Experience of Wearable Technologies: A Comparative Analysis of Textile-Based and Accessory-Based Wearable ProductsApplied Sciences10.3390/app12211115412:21(11154)Online publication date: 3-Nov-2022
https://doi.org/10.3390/app122111154
Martinelli SLopes LZaina LSantos CBento Villela MRios da Hora Rodrigues Kde Gois Ribeiro Darin T(2022)UX research in the software industryProceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems10.1145/3554364.3559126(1-13)Online publication date: 17-Oct-2022
https://dl.acm.org/doi/10.1145/3554364.3559126
Doi TDoi SYamaoka T(2022)The peak–end rule in evaluating product user experience: The chronological evaluation of past impressive episodes on overall satisfactionHuman Factors and Ergonomics in Manufacturing & Service Industries10.1002/hfm.2095132:3(256-267)Online publication date: 10-Mar-2022
https://doi.org/10.1002/hfm.20951
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten