research-article

Quantified UX: Towards a Common Organizational Understanding of User Experience

Authors:

Florian Lachner,

Philipp Naegelein,

Robert Kowalski,

Andreas ButzAuthors Info & Claims

NordiCHI '16: Proceedings of the 9th Nordic Conference on Human-Computer Interaction

Article No.: 56, Pages 1 - 10

https://doi.org/10.1145/2971485.2971501

Published: 23 October 2016 Publication History

Abstract

User Experience (UX) is increasingly being recognized as an important factor for the commercial success of digital products. In fact, it has become a buzzword, which is interpreted differently by different parties. This lack of common understanding inevitably leads to misunderstandings and inefficiency in industrial practice. We therefore propose a quantifiable way of describing User Experience (QUX). Based on the analysis of 84 UX evaluation methods, a sample of UX characteristics from literature, and 24 interviews with experts from academia and practice, we propose a formalism and a corresponding tool to measure, visualize, and communicate a product's UX within organizations. We showcase the benefits of our approach by integrating it into the product development processes of companies from three different industries.

References

[1]

All About UX. Information for user experience professionals, 2015. http://www.allaboutux.org/.

[2]

Alves, R., Valente, P., and Nunes, N. J. The State of User Experience Evaluation Practice. In Proc. NordiCHI 2014, ACM Press (2014), 93--102.

Digital Library

[3]

Bakker, S., Markopoulos, P., and de Kort, Y. OPOS: an Observation Scheme for Evaluating Head-Up Play. In Proc. NordiCHI 2008, ACM Press (2008), 18--22.

Digital Library

[4]

Bargas-Avila, J. A., and Brühlmann, F. Measuring user rated language quality: Development and validation of the user interface language quality survey (lqs). International Journal of Human-Computer Studies 86 (2016), 1--10.

Digital Library

[5]

Bargas-Avila, J. A., and Hornbæk, K. Old Wine in New Bottles or Novel Challenges? A Critical Analysis of Empirical Studies of User Experience. In Proc. CHI 2011, ACM Press (2011), 2689--2698.

Digital Library

[6]

Battarbee, K., and Koskinen, I. Co-experience: user experience as interaction. CoDesign 1, 1 (2005), 5--18.

[7]

Bruner, G. C., Hensel, P. J., and James, K. E. Marketing scales handbook, vol. 6. American Marketing Association (2012).

[8]

Burmester, M., Mast, M., Jäger, K., and Homans, H. Valence Method for Formative Evaluation of User Experience. In Proc. DIS 2010 (2010), 364--367.

Digital Library

[9]

Cheng, S. "Third Eye": Designing Eye Gaze Visualizations For Online Shopping Social Recommendations. In Proc. CSCW 2013, ACM Press (2013), 125--128.

Digital Library

[10]

Cohen, J. A coefficient of agreement for nominal scales. Educational and psychological measurement 20, 1 (1960), 37--46.

[11]

Cooper, A., Reimann, R., Cronin, D., and Noessel, C. About Face. The Essentials of Interaction Design, 4th ed. John Wiley & Sons, 2014.

Digital Library

[12]

Cronbach, L. J. Coefficient alpha and the internal structure of tests. Psychometrika 16, 3 (1951), 297--334.

[13]

Den Uyl, M. J., and van Kuilenburg, H. The FaceReader: Online facial expression recognition. In Proc. Measuring Behavior 2005 (2005), 589--590.

[14]

Desmet, P. Measuring Emotion: Development and Application of an Instrument to Measure Emotional Responses to Products. In Human-Computer Interaction Series Volume 3. Kluwer Academic, 2004, ch. 9, 111--123.

Digital Library

[15]

Desmet, P., Overbeeke, K., and Tax, S. Designing products with added emotional value: development and application of an approach for research through design. The Design Journal 4, 1 (2001), 32--47.

[16]

Frandsen-Thorlacius, O., Hornbæk, K., Hertzum, M., and Clemmensen, T. Non-Universal Usability? A Survey of How Usability is Understood by Chinese and Danish Users. In Proc. CHI 2009, ACM (2009), 41--50.

Digital Library

[17]

Glaser, B. G., and Strauss, A. L. The Discovery of Grounded Theory: Strategies for Qualitative Research, vol. 36. Aldine Transaction, 1971.

[18]

Gray, C. M. Evolution of Design Competence in UX Practice. In Proc. CHI 2014, ACM Press (2014), 1645--1654.

Digital Library

[19]

Hart, J., Sutcliffe, a. G., and De Angeli, A. Love it or Hate it! Interactivity and User Types. In Ext. Abstracts CHI 2013, ACM Press (2013), 2059--2068.

Digital Library

[20]

Hartmann, J., Sutcliffe, A., and Angeli, A. D. Towards a theory of user judgment of aesthetics and user interface quality. ACM Trans. Comput.-Hum. Interact. 15, 4 (2008), 1--30.

Digital Library

[21]

Hassenzahl, M. User experience (UX): Towards an experiential perspective on product quality. In Proc. IHM 2008, ACM Press (2008), 11--15.

Digital Library

[22]

Hassenzahl, M., Burmester, M., and Koller, F. AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In Mensch & Computer 2003. Interaktion in Bewegung, J. Ziegler and G. Szwillus, Eds. Teubner, 2003, 187--196.

[23]

Hassenzahl, M., and Tractinsky, N. User experience-a research agenda. Behaviour & Information Technology 25, 2 (2006), 91--97.

[24]

Helander, M. G., and Zhang, L. Field studies of comfort and discomfort in sitting. Ergonomics 40, 9 (1997), 895--915.

[25]

Hellman, M., and Rönkkö, K. Is User Experience Supported Effectively in Existing Software Development Processes? In Proc. VUUM 2008, Institute of Research in Informatics of Toulouse (2008), 32--37.

[26]

Hornbæk, K., and Law, E. L.-C. Meta-Analysis of Correlations Among Usability Measures. In Ext. Abstracts CHI 2007, ACM Press (2007), 617--626.

Digital Library

[27]

Iachello, G., Truong, K. N., Abowd, G. D., Hayes, G. R., and Stevens, M. Prototyping and Sampling Experience to Evaluate Ubiquitous Computing Privacy in the Real World. In Proc. CHI 2006, ACM Press (2006), 1009--1018.

Digital Library

[28]

Isabella, L. A. Evolving Interpretations as a Change Unfolds: How Managers Construe Key Organizational Events. Academy of Management Journal 33, 1 (1990), 7--41.

[29]

Jordan, P. W. Human factors for pleasure in product use. Applied Ergonomics 29, 1 (1998), 25--33.

[30]

Ju, W., Lee, B. A., and Klemmer, S. R. Range: Exploring Implicit Interaction through Electronic Whiteboard Design. In Proc. CSCW 2008, ACM Press (2008), 17--26.

Digital Library

[31]

Jung, Y., Anttila, A., and Blom, J. Designing for the Evolution of Mobile Contacts Application. In Proc. MobileHCI 2008, ACM Press (2008), 449--452.

Digital Library

[32]

Karapanos, E., Martens, J.-B., and Hassenzahl, M. On the Retrospective Assessment of Users' Experiences Over Time: Memory or Actuality? In Ext. Abstracts CHI 2010, ACM Press (2010), 4075--4080.

Digital Library

[33]

Karapanos, E., Zimmerman, J., Forlizzi, J., and Martens, J.-b. User Experience Over Time: An Initial Framework. In Proc. CHI 2009, ACM Press (2009), 729--738.

Digital Library

[34]

Karran, A. J., Fairclough, S. H., and Gilleade, K. A Framework for Psychophysiological Classification within a Cultural Heritage Context Using Interest. ACM Trans. Comput.-Hum. Interact. 21, 6 (2015).

Digital Library

[35]

Kujala, S., and Miron-Shatz, T. Emotions, experiences and usability in real-life mobile phone use. In Proc. CHI 2013, ACM Press (2013), 1061--1070.

Digital Library

[36]

Kujala, S., Roto, V., Väänänen-Vainio-Mattila, K., Karapanos, E., and Sinnelä, A. UX Curve: A method for evaluating long-term user experience. Interacting with Computers 23, 5 (2011), 473--483.

Digital Library

[37]

Lavie, T., and Tractinsky, N. Assessing dimensions of perceived visual aesthetics of web sites. International Journal of Human Computer Studies 60, 3 (2004), 269--298.

Digital Library

[38]

Law, E. L.-C., Roto, V., Hassenzahl, M., Vermeeren, A. P., and Kort, J. Understanding, scoping and defining user experience. In Proc. CHI 2009, ACM Press (2009), 719--728.

Digital Library

[39]

Lee, J. M., Jeong, S. Y., and Ju, D. Y. Emotional Interaction and Notification of Flexible Handheld Devices. In Ext. Abstracts CHI 2015 (2015), 2025--2030.

Digital Library

[40]

Liu, N., Liu, Y., and Wang, X. Data Logging plus E-diary: Towards an Online Evaluation Approach of Mobile Service Field Trial. In Proc. MobileHCI 2010, ACM Press (2010), 287--290.

Digital Library

[41]

Löchtefeld, M., Lautemann, N., Gehring, S., and Krüger, A. ambiPad Enriching Mobile Digital Media With Ambient Feedback. In Proc. MobileHCI 2014, ACM Press (2014), 295--298.

Digital Library

[42]

Magnus Revang, The User Experience Wheel, 2007. http://userexperienceproject.blogspot.de/.

[43]

Markova, M., Aula, A., Vainio, T., Wigelius, H., and Kulju, M. MoBiS-Q: a tool for evaluating the success of mobile business services. In Proc. MobileHCI 2007, ACM (2007), 238--245.

Digital Library

[44]

Mayring, P., and Fenzl, T. Qualitative Content Analysis. Springer, 2014.

[45]

Meschtscherjakov, A., Wilfinger, D., and Tscheligi, M. Mobile Attachment - Causes and Consequences for Emotional Bonding with Mobile Phones. In Proc. CHI 2014, ACM Press (2014), 2317--2326.

Digital Library

[46]

Milic-Frayling, N., Hicks, M., Jones, R., and Costello, J. On the Design and Evaluation of Web Augmented Mobile Applications. In Proc. MobileHCI 2007 (2007), 226--233.

Digital Library

[47]

Mueller, F. F., Vetere, F., Gibbs, M., Edge, D., Agamanolis, S., and Sheridan, J. Jogging over a Distance between Europe and Australia. In Proc. UIST 2014, ACM Press (2010), 189--198.

Digital Library

[48]

Mugge, R., Schifferstein, H. N. J., and Schoormans, J. P. L. A Longitudinal Study of Product Attachment and its Determinants. European Advances in Consumer Research 7 (2006), 641--647.

[49]

Nielsen, J. Enhancing the Explanatory Power of Usability Heuristics. In Proc. CHI 1994, ACM Press (1994), 152--158.

Digital Library

[50]

Norman, D. A. The Design of Everyday Things. Basic Books, 2013.

[51]

Obrist, M., Beck, E., Wurhofer, D., and Tscheligi, M. Experience Characters: A Design Tool for Communicating Mobile Phone Experiences to Designers. In Proc. MobileHCI 2011 (2011), 385--394.

Digital Library

[52]

Oliveira, R. D., Cherubini, M., and Oliver, N. Influence of Personality on Satisfaction with Mobile Phone Services. ACM Trans. Comput.-Hum. Interact. 20, 2 (2013), 1--23.

Digital Library

[53]

Olsson, T. Layers of User Expectations of Future Technologies: an Early Framework. In Ext. Abstracts CHI 2014, ACM Press (2014), 1957--1962.

Digital Library

[54]

Olsson, T., and Salo, M. Narratives of satisfying and unsatisfying experiences of current mobile augmented reality applications. In Ext. Abstracts CHI 2012, ACM Press (2012), 2779.

Digital Library

[55]

Pine, J., and Gilmore, J. H. Welcome to the Experience Economy. Harvard Business Review 76, 4 (1998), 97--105.

[56]

Rodden, K., Hutchinson, H., and Fu, X. Measuring the User Experience on a Large Scale: User-Centered Metrics for Web Applications. In Proc. CHI 2010, ACM Press (2010), 2395--2398.

Digital Library

[57]

Sauro, J., and Lewis, J. R. Correlations among Prototypical Usability Metrics: Evidence for the Construct of Usability. In Proc. CHI 2009, ACM Press (2009), 1609--1618.

Digital Library

[58]

Schulze, K., and Kroemker, H. Extracting User Experience Centered Product Requirements for Mobile Social Media Applications. In Proc. MobileHCI 2012, ACM Press (2012), 143--148.

Digital Library

[59]

Scollon, C. N., Kim-Prieto, C., and Diener, E. Experience Sampling: Promises and Pitfalls, Strengths and Weaknesses. Journal of Happiness Studies 4, 1 (2003), 5--34.

[60]

Sheldon, K. M., Elliot, A. J., Kim, Y., and Kasser, T. What Is Satisfying About Satisfying Events? Journal of Personality and Social Psychology 80, 2 (2001), 325--339.

[61]

Sonderegger, A., Uebelbacher, A., Pugliese, M., and Sauer, J. The influence of aesthetics in usability testing. In Proc. CHI 2014, ACM (2014), 21--30.

Digital Library

[62]

Srinivasan, V., Moghaddam, S., Mukherji, A., Rachuri, K. K., Xu, C., and Tapia, E. M. MobileMiner: Mining Your Frequent Patterns on Your Phone. In Proc. UbiComp 2014, ACM Press (2014), 389--400.

Digital Library

[63]

Tuch, A. N., and Hornbæk, K. Does Herzbergs Notion of Hygienes and Motivators Apply to User Experience? ACM Trans. Comput.-Hum. Interact. 22, 4 (2015).

Digital Library

[64]

Turunen, M., Melto, A., Hella, J., Heimonen, T., Hakulinen, J., Mäkinen, E., Laivo, T., and Soronen, H. User Expectations and User Experience with Different Modalities in a Mobile Phone Controlled Home Entertainment System. In Proc. MobileHCI 2009, ACM Press (2009), 1--4.

Digital Library

[65]

Väänänen-Vainio-Mattila, K., Roto, V., and Hassenzahl, M. Now Let's Do It in Practice: User Experience Evaluation Methods in Product Development. In Ext. Abstracts CHI 2008, ACM Press (2008), 3961--3964.

Digital Library

[66]

Väänänen-Vainio-Mattila, K., Roto, V., and Hassenzahl, M. Towards Practical User Experience Evaluation Methods. In Proc. VUUM 2008, Institute of Research in Informatics of Toulouse (2008), 19--22.

[67]

van Schaik, P., Hassenzahl, M., and Ling, J. User-Experience from an Inference Perspective. ACM Trans. Comput.-Hum. Interact. 19, 2 (2012), 1--25.

Digital Library

[68]

Vandenbroucke, K., Ferreira, D., Goncalves, J., Kostakos, V., and De Moor, K. Mobile Cloud Storage: A Contextual Experience. In Proc. MobileHCI 2014, ACM Press (Toronto, ON, Canada, 2014), 101--110.

Digital Library

[69]

Vermeeren, A. P., Law, E. L.-C., Roto, V., Obrist, M., Hoonhout, J., and Väänänen-Vainio-Mattila, K. User experience evaluation methods: current state and development needs. In Proc. NordiCHI 2010, ACM Press (2010), 521--530.

Digital Library

[70]

Vescovi, M., Bruno, L., Perentis, C., Moiso, C., and Leonardi, C. My Data Store: Toward User Awareness and Control on Personal Data. In Proc. UbiComp 2014, ACM Press (2014), 179--182.

Digital Library

[71]

Vuolle, M., Tiainen, M., Kallio, T., Vainio, T., Kulju, M., and Wigelius, H. Developing a Questionnaire for Measuring Mobile Business Service Experience. In Proc. MobileHCI 2008, ACM Press (2008), 53--62.

Digital Library

Cited By

Deters HDroste JHess AKlös VSchneider KSpeith TVogelsang A(2024)The X Factor: On the Relationship between User eXperience and eXplainabilityProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685352(1-12)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685352
Macedo MZaina L(2023)UX Data Visualization: Supporting Software Professionals in Exploring Users’ Interaction DataEngineering Interactive Computer Systems. EICS 2023 International Workshops and Doctoral Consortium10.1007/978-3-031-59235-5_17(207-222)Online publication date: 27-Jun-2023
https://dl.acm.org/doi/10.1007/978-3-031-59235-5_17
Coronado EKiyokawa TRicardez GRamirez-Alpizar IVenture GYamanobe N(2022)Evaluating quality in human-robot interaction: A systematic search and classification of performance and human-centered factors, measures and metrics towards an industry 5.0Journal of Manufacturing Systems10.1016/j.jmsy.2022.04.00763(392-410)Online publication date: Apr-2022
https://doi.org/10.1016/j.jmsy.2022.04.007
Show More Cited By

Index Terms

Quantified UX: Towards a Common Organizational Understanding of User Experience
1. Human-centered computing

Recommendations

UX trek: a post-interaction journey from immersive experience logs
IHC '20: Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems

User Experience (UX) is a holistic attribute focused on the quality of user-product interaction. However, the technology evolution in different contexts has not imposed limits on user-product interactions. For instance, in immersive technologies, the ...
Triangulation in UX Studies: Learning from Experience
DIS '17 Companion: Proceedings of the 2017 ACM Conference Companion Publication on Designing Interactive Systems

While the consideration of User Experience (UX) has become embedded in research and design processes, UX evaluation remains a challenging and strongly discussed area for both researchers in academia and practitioners in industry. A variety of different ...
User Research and Design for Live TV UX in China
TVX '17 Adjunct: Adjunct Publication of the 2017 ACM International Conference on Interactive Experiences for TV and Online Video

In current Chinese TV market, Live TV programs are buffeted by internet contents from smart TV/over-the-top (OTT) box and other screens like mobile phone and pad. The user experience of Live TV watching can't satisfy user needs. We defined Chinese ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

NordiCHI '16: Proceedings of the 9th Nordic Conference on Human-Computer Interaction

October 2016

1045 pages

ISBN:9781450347631

DOI:10.1145/2971485

General Chairs:
Staffan Björk
University of Gothenburg
,
Eva Eriksson
Chalmers University of Technology
,
Program Chairs:
Morten Fjeld
Chalmers University of Technology
,
Susanne Bødker
University of Aarhus
,
Publications Chairs:
Wolmet Barendregt
University of Gothenburg
,
Mohammad Obaid
Chalmers University of Technology

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

Chalmers University of Technology
SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

German Federal Ministry for Education and Research

Conference

NordiCHI '16

NordiCHI '16: 9th Nordic Conference on Human-Computer Interaction

October 23 - 27, 2016

Gothenburg, Sweden

Acceptance Rates

NordiCHI '16 Paper Acceptance Rate 58 of 231 submissions, 25%;

Overall Acceptance Rate 379 of 1,572 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
1,062
Total Downloads

Downloads (Last 12 months)164
Downloads (Last 6 weeks)8

Reflects downloads up to 01 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Deters HDroste JHess AKlös VSchneider KSpeith TVogelsang A(2024)The X Factor: On the Relationship between User eXperience and eXplainabilityProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685352(1-12)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685352
Macedo MZaina L(2023)UX Data Visualization: Supporting Software Professionals in Exploring Users’ Interaction DataEngineering Interactive Computer Systems. EICS 2023 International Workshops and Doctoral Consortium10.1007/978-3-031-59235-5_17(207-222)Online publication date: 27-Jun-2023
https://dl.acm.org/doi/10.1007/978-3-031-59235-5_17
Coronado EKiyokawa TRicardez GRamirez-Alpizar IVenture GYamanobe N(2022)Evaluating quality in human-robot interaction: A systematic search and classification of performance and human-centered factors, measures and metrics towards an industry 5.0Journal of Manufacturing Systems10.1016/j.jmsy.2022.04.00763(392-410)Online publication date: Apr-2022
https://doi.org/10.1016/j.jmsy.2022.04.007
Clemmensen TIivari NRajanen DSivaji A(2022)“Organized UX Professionalism” – An Empirical Study and Conceptual Tool for Scrutinizing UX Work of the FutureHuman Work Interaction Design. Artificial Intelligence and Designing for a Positive Work Experience in a Low Desire Society10.1007/978-3-031-02904-2_2(34-65)Online publication date: 1-Apr-2022
https://doi.org/10.1007/978-3-031-02904-2_2
Ntoa SMargetis GAntona MStephanidis C(2021)User Experience Evaluation in Intelligent Environments: A Comprehensive FrameworkTechnologies10.3390/technologies90200419:2(41)Online publication date: 25-May-2021
https://doi.org/10.3390/technologies9020041
Lanius CWeber RRobinson J(2021)User Experience Methods in Research and PracticeJournal of Technical Writing and Communication10.1177/0047281621104449951:4(350-379)Online publication date: 16-Oct-2021
https://doi.org/10.1177/00472816211044499
Coelho BAndrade RDarin T(2021)Not the Same Everywhere: Comparing the Scope and Definition of User Experience between the Brazilian and International CommunitiesInternational Journal of Human–Computer Interaction10.1080/10447318.2021.196072738:7(595-613)Online publication date: 10-Aug-2021
https://doi.org/10.1080/10447318.2021.1960727
Clemmensen T(2021)Dropping a Bomb or Providing a Gentle Loving Touch? Towards a Relation Artefact Theory of Pilot ImplementationSense, Feel, Design10.1007/978-3-030-98388-8_39(429-439)Online publication date: 30-Aug-2021
https://dl.acm.org/doi/10.1007/978-3-030-98388-8_39
Alexandre RFaria AProvidência B(2021)Design Attributes for the Development of Digital Solutions: An Exploratory Study from the Perspective of Professionals Connected to Digital TransformationAdvances in Design and Digital Communication II10.1007/978-3-030-89735-2_24(287-295)Online publication date: 22-Oct-2021
https://doi.org/10.1007/978-3-030-89735-2_24
Clemmensen TClemmensen T(2021)HWID ConsultancyHuman Work Interaction Design10.1007/978-3-030-71796-4_8(173-236)Online publication date: 24-Sep-2021
https://doi.org/10.1007/978-3-030-71796-4_8
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents