research-article

Evolving Heuristic Evaluation for multiple contexts and audiences: Perspectives from a mapping study

Authors:

André de Lima Salgado,

Sandra Souza Rodrigues,

Renata Pontin M. FortesAuthors Info & Claims

SIGDOC '16: Proceedings of the 34th ACM International Conference on the Design of Communication

Article No.: 19, Pages 1 - 8

https://doi.org/10.1145/2987592.2987617

Published: 23 September 2016 Publication History

Abstract

This paper presents the results concerning an update of a previous mapping study about usability heuristics applied in Heuristic Evaluation of mobile usability. Heuristic Evaluation is a popular method of usability evaluation, and its applicability to inspect the mobile usability may be improved by the development of context and audience specific heuristics. Additionally to a previous mapping, we present analysis of the recent usage of context and audience specific heuristics, and the application of Heuristic Evaluation to evaluate multiple contexts and audiences. The results showed the use of specific heuristics for the Healthy context and Malaysian audience. In addition, the results turned out the evolving of Heuristic Evaluation for multiple contexts and audiences, such as: Health Sector, Games, Learning, Elderly and Accessibility.

References

[1]

M. S. Al-Razgan, H. S. Al-Khalifa, and M. D. Al-Shahrani. Design, User Experience, and Usability. Theories, Methods, and Tools for Designing the User Experience: Third Int. Conf., DUXU 2014, Held as Part of HCI Int. 2014, Heraklion, Crete, Greece, June 22-27, 2014, Proc., Part I, chapter Heuristics for Evaluating the Usability of Mobile Launchers for Elderly People, pages 415--424. Springer Int. Publishing, Cham, 2014.

[2]

S. Alhumoud, L. Alabdulkarim, N. Almobarak, and A. Al-Wabil. Human-Computer Interaction: Users and Contexts: 17th Int. Conf., HCI Int. 2015, Los Angeles, CA, USA, August 2-7, 2015, Proc., Part III, chapter Socio-Cultural Aspects in the Design of Multilingual Banking Interfaces in the Arab Region, pages 269--280. Springer Int. Publishing, Cham, 2015.

[3]

S. A. Ariffin and L. E. Dyson. Cross-Cultural Design Applications in Mobile Interaction, Education, Health, Transport and Cultural Heritage: 7th Int. Conf., CCD 2015, Held as Part of HCI Int. 2015, Los Angeles, CA, USA, August 2-7, 2015, Proc., Part II, chapter Culturally Appropriate Design of Mobile Learning Applications in the Malaysian Context, pages 3--14. Springer Int. Publishing, Cham, 2015.

[4]

B. R. Barricelli, Y. Devis, J. Abdelnour-Nocera, J. Wilson, and J. Moore. Mantra: Mobile anticoagulant therapy management. In Proc. of the 7th Int. Conf. on Pervasive Comput. Technologies for Healthcare, PervasiveHealth '13, pages 278--281, ICST, Brussels, Belgium, Belgium, 2013. ICST (Institute for Computer Sci., Social-Informatics and Telecommunications Engineering).

Digital Library

[5]

E. Bertini, S. Gabrielli, and S. Kimani. Appropriating and assessing heuristics for mobile comput. In Proc. of the Working Conf. on Advanced Visual Interfaces, AVI '06, pages 119--126, New York, NY, USA, 2006. ACM.

Digital Library

[6]

M. Billi, L. Burzagli, T. Catarci, G. Santucci, E. Bertini, F. Gabbanini, and E. Palchetti. A unified methodology for the evaluation of accessibility and usability of mobile applications. Universal Access in the Inform. Society, 9(4):337--356, 2010.

Digital Library

[7]

R. Budiu. The state of mobile user experience. Nielsen Norman Group: Evidence-Based User Experience Research, Training, and Consulting. Url: https://www.nngroup.com/articles/mobile-usability-update/1, 2015.

[8]

A. Conde, K. L. Ipiña, M. Larrañaga, J. A. Elorriaga, J. M. López, E. Irigoyen, N. Garay-Vitoria, A. Ezeiza, and J. Rubio. Trends in Practical Applications of Agents and Multiagent Syst.: 8th Int. Conf. on Practical Applications of Agents and Multiagent Syst., chapter An Intelligent Tutoring System Oriented to the Integration of People with Intellectual Disabilities, pages 639--647. Springer Berlin Heidelberg, Berlin, Heidelberg, 2010.

[9]

A. de Lima Salgado and A. P. Freire. Human-Computer Interaction. Applications and Services: 16th Int. Conf., HCI Int. 2014, Heraklion, Crete, Greece, June 22-27, 2014, Proceedings, Part III, chapter Heuristic Evaluation of Mobile Usability: A Mapping Study, pages 178--188. Springer Int. Publishing, Cham, 2014.

[10]

A. Eidaroos and A. Alkraiji. New Contributions in Inform. Syst. and Technologies: Volume 1, chapter Evaluating the Usability of Library Websites Using an Heuristic Analysis Approach on Smart Mobile Phones: Preliminary Findings of a Study in Saudi Universities, pages 1141--1152. Springer Int. Publishing, Cham, 2015.

[11]

M. Fetaji and B. Fetaji. Comparing developed mluat (mobile learning usability attribute testing) methodology with qualitative user testing method and heuristics evaluation. In Proc. of the 12th Int. Conf. on Computer Syst. and Technologies, CompSysTech '11, pages 516--523, New York, NY, USA, 2011. ACM.

Digital Library

[12]

C. Gielkens and R. Wetzel. Entertainment Comput. -ICEC 2012: 11th Int. Conf., ICEC 2012, Bremen, Germany, September 26-29, 2012. Proc., chapter A Framework for Usability Evaluation of Mobile Mixed Reality Games, pages 401--404. Springer Berlin Heidelberg, Berlin, Heidelberg, 2012.

Digital Library

[13]

C.-H. Huang and C.-M. Wang. Design, User Experience, and Usability. Theory, Methods, Tools and Practice: First Int. Conf., DUXU 2011, Held as Part of HCI Int. 2011, Orlando, FL, USA, July 9-14, 2011, Proc., Part I, chapter Usability Analysis in Gesture Operation of Interactive E-Books on Mobile Devices, pages 573--582. Springer Berlin Heidelberg, Berlin, Heidelberg, 2011.

[14]

R. Inostroza and C. Rusu. Mapping usability heuristics and design principles for touchscreen-based mobile devices. In Proc. of the 7th Euro American Conf. on Telematics and Inform. Syst., EATIS '14, pages 27:1--27:4, New York, NY, USA, 2014. ACM.

Digital Library

[15]

R. Inostroza, C. Rusu, S. Roncagliolo, C. Jimenez, and V. Rusu. Usability heuristics for touchscreen-based mobile devices. In Inform. Technology: New Generations (ITNG), 2012 Ninth Int. Conf. on, pages 662--667, April 2012.

Digital Library

[16]

R. Inostroza, C. Rusu, S. Roncagliolo, and V. Rusu. Usability heuristics for touchscreen-based mobile devices: Update. In Proc. of the 2013 Chilean Conf. on Human - Computer Interaction, ChileCHI '13, pages 24--29, New York, NY, USA, 2013. ACM.

Digital Library

[17]

R. Inostroza, C. Rusu, S. Roncagliolo, V. Rusu, and C. A. Collazos. Developing smash: A set of smartphone's usability heuristics. Computer Standards & Interfaces, 43:40--52, 2016.

Digital Library

[18]

G. Joyce, M. Lilley, T. Barker, and A. Jefferies. Design, User Experience, and Usability: Users and Interactions: 4th Int. Conf., DUXU 2015, Held as Part of HCI Int. 2015, Los Angeles, CA, USA, August 2-7, 2015, Proc., Part II, chapter Smartphone Application Usability Evaluation: The Applicability of Traditional Heuristics, pages 541--550. Springer Int. Publishing, Cham, 2015.

Digital Library

[19]

A. Karahoca, E. Bayraktar, E. Tatoglu, and D. Karahoca. Inform. system design for a hospital emergency department: A usability analysis of software prototypes. J. of Biomedical Informatics, 43(2):224--232, 2010.

Digital Library

[20]

P. Karampelas, D. Akoumianakis, and C. Stephanidis. Universal Access in Health Telematics: A Design Code of Practice, chapter Chapter 16 Usability Inspection of the WardInHand Prototype, pages 197--208. Springer Berlin Heidelberg, Berlin, Heidelberg, 2005.

[21]

B. Kitchenham. Procedures for performing systematic reviews. Keele, UK, Keele University, 33(2004):1--26, 2004.

[22]

A. C. T. Klock and I. Gasparini. HCI Int. 2015-Posters' Extended Abstracts: Int. Conf., HCI Int. 2015, Los Angeles, CA, USA, August 2-7, 2015. Proc., Part II, chapter A Usability Evaluation of Fitness-Tracking Apps for Initial Users, pages 457--462. Springer Int. Publishing, Cham, 2015.

[23]

R. G. Longoria, M. McGee, and E. Nash. Designing Software for the Mobile Context: A Practitioner's Guide, chapter Heuristics for Designing Mobile Applications, pages 109--134. Springer London, London, 2004.

[24]

O. Machado Neto and M. d. G. Pimentel. Heuristics for the assessment of interfaces of mobile devices. In Proc. of the 19th Brazilian Symposium on Multimedia and the Web, WebMedia '13, pages 93--96, New York, NY, USA, 2013. ACM.

Digital Library

[25]

A. Marcus, S. Abromowitz, and M. F. Abulkhair. Design, User Experience, and Usability. User Experience in Novel Technological Environments: Second Int. Conf., DUXU 2013, Held as Part of HCI Int. 2013, Las Vegas, NV, USA, July 21-26, 2013, Proc., Part III, chapter Heuristic Evaluation of iCalamityGuide Application, pages 130--139. Springer Berlin Heidelberg, Berlin, Heidelberg, 2013.

[26]

M. Masood and M. Thigambaram. The usability of mobile applications for pre-schoolers. Procedia - Social and Behavioral Sci., 197:1818--1826, 2015. 7th World Conf. on Educational Sci.

[27]

N. Mi, L. A. Cavuoto, K. Benson, T. Smith-Jackson, and M. A. Nussbaum. A heuristic checklist for an accessible smartphone interface design. Universal Access in the Inform. Society, 13(4):351--365, 2013.

Digital Library

[28]

S. A. Morris and B. Van der Veer Martens. Mapping research specialties. Annual Review of Inform. Sci. and Technology, 42(1):213--295, 2008.

Digital Library

[29]

C. Mullins. Responsive, mobile app, mobile first: Untangling the ux design web in practical experience. In Proc. of the 33rd Annual Int. Conf. on the Design of Communication, SIGDOC '15, pages 22:1--22:6, New York, NY, USA, 2015. ACM.

Digital Library

[30]

J. Nielsen. Enhancing the explanatory power of usability heuristics. In Proc. of the SIGCHI Conf. on Human Factors in Comput. Syst., CHI '94, pages 152--158, New York, NY, USA, 1994. ACM.

Digital Library

[31]

J. Nielsen. Enhancing the Explanatory Power of Usability Heuristics. In Proc. of the SIGCHI Conf. on Human Factors in Comput. Syst., CHI '94, pages 152--158, New York, NY, USA, 1994. ACM.

Digital Library

[32]

J. Nielsen. Usability inspection methods. chapter Heuristic Evaluation, pages 25--62. John Wiley & Sons, Inc., New York, NY, USA, 1994.

Digital Library

[33]

J. Nielsen and R. Molich. Heuristic evaluation of user interfaces. In Proc. of the SIGCHI Conf. on Human Factors in Comput. Syst., CHI '90, pages 249--256, 1990.

Digital Library

[34]

G. Orlandini, G. A. Castadelli, and L. M. Presumido Braccialli. Design, User Experience, and Usability. User Experience Design for Diverse Interaction Platforms and Environments: Third Int. Conf., DUXU 2014, Held as Part of HCI Int. 2014, Heraklion, Crete, Greece, June 22-27, 2014, Proc., Part II, chapter Ergonomics and Usability in Sound Dimension: Evaluation of a Haptic and Acoustic Interface Application for Mobile Devices, pages 193--202. Springer Int. Publishing, Cham, 2014.

Digital Library

[35]

H. Petrie, A. Savva, and C. Power. Towards a unified definition of web accessibility. In Proc. of the 12th Web for All Conf., W4A '15, pages 35:1--35:13, New York, NY, USA, 2015. ACM.

Digital Library

[36]

S. Po, S. Howard, F. Vetere, and M. B. Skov. Mobile Human-Computer Interaction - MobileHCI 2004: 6th Int. Symposium, MobileHCI, Glasgow, UK, September 13-16, 2004. Proc., chapter Heuristic Evaluation and Mobile Usability: Bridging the Realism Gap, pages 49--60. Springer Berlin Heidelberg, Berlin, Heidelberg, 2004.

[37]

R. Racadio, E. Rose, and S. Boyd. Designing and evaluating the mobile experience through iterative field studies. In Proc. of the 30th ACM Int. Conf. on Design of Communication, SIGDOC '12, pages 191--196, New York, NY, USA, 2012. ACM.

Digital Library

[38]

G. Reynaga, S. Chiasson, and P. C. van Oorschot. Heuristics for the evaluation of captchas on smartphones. In Proc. of the 2015 British HCI Conf., British HCI 15, pages 126--135, New York, NY, USA, 2015. ACM.

Digital Library

[39]

L. H. A. Salazar, T. Lacerda, J. V. Nunes, and C. Gresse von Wangenheim. A Systematic Literature Review on Usability Heuristics for Mobile Phones:. Int. J. of Mobile Human Computer Interaction, 5(2):50--61, 2013.

[40]

L. H. A. Salazar, T. Lacerda, C. G. von Wangenheim, and R. A. Barbalho. Customizando heurÍsticas de usabilidade para celulares. In Companion Proc. of the 11th Brazilian Symposium on Human Factors in Comput. Syst., IHC '12, pages 37--38, Porto Alegre, Brazil, Brazil, 2012. Brazilian Computer Society.

Digital Library

[41]

R. Schoenfelder and D. Schmalstieg. Augmented reality for industrial building acceptance. In 2008 IEEE Virtual Reality Conf., pages 83--90, March 2008.

[42]

S. Soomro, W. F. Wan Ahmad, and S. Sulaiman. Advances in Visual Informatics: Third Int. Visual Informatics Conf., IVIC 2013, Selangor, Malaysia, November 13-15, 2013. Proc., chapter Evaluation of Mobile Games Using Playability Heuristics, pages 264--274. Springer Int. Publishing, Cham, 2013.

Digital Library

[43]

J. Varsaluoma. Human Centered Design: First Int. Conf., HCD 2009, Held as Part of HCI Int. 2009, San Diego, CA, USA, July 19-24, 2009 Proc., chapter Scenarios in the Heuristic Evaluation of Mobile Devices: Emphasizing the Context of Use, pages 332--341. Springer Berlin Heidelberg, Berlin, Heidelberg, 2009.

Digital Library

[44]

N. A. Wahab, A. Osman, and M. H. Ismail. Engaging children to sci. subject: A heuristic evaluation of mobile learning prototype. In Computer Engineering and Applications (ICCEA), 2010 Second Int. Conf. on, volume 2, pages 513--516, March 2010.

Digital Library

[45]

J. Xu, X. Ding, K. Huang, and G. Chen. A pilot study of an inspection framework for automated usability guideline reviews of mobile health applications. In Proc. of the Wireless Health 2014 on National Institutes of Health, WH '14, pages 7:1--7:8, New York, NY, USA, 2014. ACM.

Digital Library

[46]

S. B. Zaibon and N. Shiratuddin. Heuristics evaluation strategy for mobile game-based learning. In Wireless, Mobile and Ubiquitous Technologies in Education (WMUTE), 2010 6th IEEE Int. Conf. on, pages 127--131, April 2010.

Digital Library

Cited By

Galavi ZNorouzi SKhajouei R(2024)Heuristics used for evaluating the usability of mobile health applications: A systematic literature reviewDIGITAL HEALTH10.1177/2055207624125353910Online publication date: 15-May-2024
https://doi.org/10.1177/20552076241253539
Oswal SPalmer Z(2024)Compliance, Commitment, and Web Accessibility: Findings of an Organizational Study Engaging Accessibility Experts at Twenty U.S. UniversitiesProceedings of the 42nd ACM International Conference on Design of Communication10.1145/3641237.3691647(17-25)Online publication date: 20-Oct-2024
https://dl.acm.org/doi/10.1145/3641237.3691647
Duan PWarner JLi YHartmann B(2024)Generating Automatic Feedback on UI Mockups with Large Language ModelsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642782(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642782
Show More Cited By

Recommendations

Heuristic evaluation of ambient displays
CHI '03: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

We present a technique for evaluating the usability and effectiveness of ambient displays. Ambient displays are abstract and aesthetic peripheral displays portraying non-critical information on the periphery of a user's attention. Although many ...
Heuristic evaluation of Geoweb: case study
ECC'11: Proceedings of the 5th European conference on European computing conference

Usability is a part of quality, and this article focuses on the one of testing methods that can be used in usability testing. There are plenty of methods to verify usability, like usability testing, heuristic evaluation, heuristic estimation, cognitive ...
Evaluating a Methodology to Establish Usability Heuristics
SCCC '12: Proceedings of the 2012 31st International Conference of the Chilean Computer Science Society

Assessing usability in any software product may be a key factor for predicting its success or fail. Heuristic evaluation is the most commonly used usability evaluation method. It uses a set of recognized usability design principles (heuristics). Until ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGDOC '16: Proceedings of the 34th ACM International Conference on the Design of Communication

September 2016

304 pages

ISBN:9781450344951

DOI:10.1145/2987592

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGDOC: ACM Special Interest Group for Design of Communications

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 September 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

São Paulo Research Foundation (FAPESP)
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior

Conference

SIGDOC '16

Sponsor:

SIGDOC

SIGDOC '16: The 34th ACM International Conference on the Design of Communication

September 23 - 24, 2016

MD, Silver Spring, USA

Acceptance Rates

SIGDOC '16 Paper Acceptance Rate 36 of 102 submissions, 35%;

Overall Acceptance Rate 355 of 582 submissions, 61%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
361
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)3

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Galavi ZNorouzi SKhajouei R(2024)Heuristics used for evaluating the usability of mobile health applications: A systematic literature reviewDIGITAL HEALTH10.1177/2055207624125353910Online publication date: 15-May-2024
https://doi.org/10.1177/20552076241253539
Oswal SPalmer Z(2024)Compliance, Commitment, and Web Accessibility: Findings of an Organizational Study Engaging Accessibility Experts at Twenty U.S. UniversitiesProceedings of the 42nd ACM International Conference on Design of Communication10.1145/3641237.3691647(17-25)Online publication date: 20-Oct-2024
https://dl.acm.org/doi/10.1145/3641237.3691647
Duan PWarner JLi YHartmann B(2024)Generating Automatic Feedback on UI Mockups with Large Language ModelsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642782(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642782
Hassan L(2023)Accessibility of games and game-based applications: A systematic literature review and mapping of future directionsNew Media & Society10.1177/1461444823120402026:4(2336-2384)Online publication date: 9-Nov-2023
https://doi.org/10.1177/14614448231204020
Salgado AHung PFortes R(2023)Six usable privacy heuristicsProceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems10.1145/3638067.3638111(1-11)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3638067.3638111
Fernandes SLucena MPegurin LBlanco JLucrédio D(2022)OfflineManager: A Lightweight Approach for Managing Offline Status in Mobile ApplicationsProceedings of the 16th Brazilian Symposium on Software Components, Architectures, and Reuse10.1145/3559712.3559717(30-39)Online publication date: 3-Oct-2022
https://dl.acm.org/doi/10.1145/3559712.3559717
Hossain MAzhar UTo YChoi JZaman L(2021)Automated Escalation and Incident Management in Healthcare During Mass Casualties and Pandemic EventsDigital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. AI, Product and Service10.1007/978-3-030-77820-0_6(68-85)Online publication date: 3-Jul-2021
https://doi.org/10.1007/978-3-030-77820-0_6
Gupta S(2020)An Analysis of UI/UX Designing With Software Prototyping ToolsCrowdsourcing and Probabilistic Decision-Making in Software Engineering10.4018/978-1-5225-9659-2.ch008(134-145)Online publication date: 2020
https://doi.org/10.4018/978-1-5225-9659-2.ch008
Carneiro NMelo BCavalcante LCastro RAndrade RDarin T(2020)Mobili: Development and Use of a Usability Checklist for Mobile Games and ApplicationsProceedings of the XIX Brazilian Symposium on Software Quality10.1145/3439961.3439994(1-9)Online publication date: 1-Dec-2020
https://dl.acm.org/doi/10.1145/3439961.3439994
Oswal SStaggers JRichards DAmidon TPflugfelder E(2019)Breaking the exclusionary boundary between user experience and accessProceedings of the 37th ACM International Conference on the Design of Communication10.1145/3328020.3353957(1-8)Online publication date: 4-Oct-2019
https://dl.acm.org/doi/10.1145/3328020.3353957
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents