research-article

The complementary role of two evaluation methods in the usability and accessibility evaluation of a non-standard system

Authors:

Funmi Adebesin,

Helene GelderblomAuthors Info & Claims

SAICSIT '10: Proceedings of the 2010 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists

Pages 1 - 11

https://doi.org/10.1145/1899503.1899504

Published: 11 October 2010 Publication History

Abstract

Usability, which is generally defined in terms of application effectiveness, efficiency and user satisfaction, is one of the focus areas of human-computer interaction (HCI). Accessibility is the design of systems that can be perceived, understood and used by people with varying abilities. Although accessibility concerns are aimed at making systems usable for people with disabilities, support for direct accessibility, the built-in redundancies in an application that enable as many people as possible to utilize it without system modifications, is beneficial to people with or without disabilities. Different usability evaluation methods (UEMs) are available. Selecting between the various methods can be influenced by the type of system being evaluated. The Digital Doorway (DD), a non-standard computer system deployed to promote computer literacy amongst underprivileged communities in South Africa, was evaluated using the heuristic evaluation method and a field usability study. The heuristic evaluation method revealed a large number of usability and direct accessibility-related problems, some of which could be classified as low-severity problems. The field study showed additional problems that affected the successful completion of user tasks. Since a number of these were a direct consequence of the context of use, they were not recognized as problems by expert evaluators. The study showed that the heuristic evaluation method can be optimized by complementing it with another method that involves user participation and is, preferably, carried out in the intended context of use.

References

[1]

Adebesin, T. F. 2010. Report on the Usability and Accessibility Evaluation of the Digital Doorway. Available from http://hufee.meraka.org.za/Hufeesite/links/files/Technical%20Report%20-%20Digital%20Doorway%20Evaluation-Protected.pdf/view

[2]

Alessi, S. M., & Trollip, S. R. 2001. Multimedia for Learning: Methods and Development (3^rd ed.). Massachusetts: Allyn & Bacon.

Digital Library

[3]

Barnum, C. M. 2002. Usability Testing and Research. Massachusetts: Allyn & Bacon.

Digital Library

[4]

Cockton, G., Woolrych, A., & Lavery, D. 2008. Inspection-Based Evaluations. In A. Sears & J. A. Jacko (Eds.), The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications (2^nd ed.). New York: Lawrence Erlbaum Associates.

[5]

Davis, F. D. 1989. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly, 13(3), 319--340.

Digital Library

[6]

Dix, A., Finlay, J., Abowd, G. D., & Beale, R. 2004. Human-Computer Interaction (3^rd ed.): Pearson Education Ltd.

Digital Library

[7]

Dumas, J. S. 2003. User-Based Evaluation. In J. A. Jacko & A. Sears (Eds.), The Human-Computer Interaction Handbook. Mahwah: Lawrence Erlbaum Associates.

Digital Library

[8]

Gardner-Bonneau, D. 2010. Is Technology Becoming More Usable -or Less - and With What Consequences. Journal of Usability Studies, 5(2), 46--49.

Digital Library

[9]

Gelderblom, J. H. 2008. Designing Technology for Young Children: Guidelines Grounded in a Literature Investigation on Child Development and Children's Technology. PhD Thesis, UNISA. Retrieved from http://etd.unisa.ac.za/ETD-db/theses/available/etd-09172008-132111/unrestricted/thesis.pdf

Digital Library

[10]

Gray, W. D., & Salzman, M. C. 1998. Damaged Merchandise? A Review of Experiments That Compare Usability Evaluation Methods. Human-Computer Interaction, 13, 203--261.

Digital Library

[11]

Gush, K., Cambridge, G., & Smith, R. 2004. The Digital Doorway - minimally invasive education in Africa. Paper presented at the Proceedings of the ICT in Education Conference, Cape Town.

[12]

Gush, K., De Villiers, M. R., Smith, R., & Cambridge, G. In Press. Digital Doorways. In J. Steyn, J. Belle & E. M. Villeneuva (Eds.), Development Informatics and Regional Information Technologies: Theory, Practice and the Digital Divide (Vol. 2). Pennsylvania: IGI.

[13]

Henry, S. L. 2002. Understanding Web Accessibility. In J. Thatcher, P. Bohman, M. R. Burks, S. L. Henry, B. Regan, S. Swierenga, M. D. Urban & C. D. Waddell (Eds.), Constructing Accessibility Web Sites. Birmingham: Glasshaus.

[14]

Henry, S. L. 2007. Just Ask: Integrating Accessibility Throughout Design. Retrieved from http://www.uiaccess.com/JustAsk/

[15]

Hertzum, M., & Jacobsen, N. E. 2003. The Evaluator Effect: A Chilling Fact About Usability Evaluation Methods. International Journal of Human-Computer Interaction, 15(1), 183--204.

[16]

Hornbaek, K., & Frokjaer, E. 2008. A Study of the Evaluator Effect in Usability Testing. Human Computer Interaction, 23(3), 251--277.

[17]

IBM. 2009. IBM Software Accessibility Checklist Retrieved 24 November 2009, from http://www-03.ibm.com/able/guidelines/software/accesssoftware.html

[18]

International Organization for Standardization. 1998. ISO 9241-11 Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs) - Part 11: Guidance on Usability.

[19]

Iwarsson, S., & Stahl, A. 2003. Accessibility, Usability and Universal Design - Positioning and Definition of Concepts Describing Person-Environment Relationships. Disability & Rehabilitation, 25(2), 57--66.

[20]

Jeffries, R., Miller, J. R., Wharton, C., & Uyeda, K. 1991. User Interface Evaluation in the Real World: A Comparison of Four Techniques. Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.

Digital Library

[21]

Lindgaard, G. 1994. Usability Testing and System Evaluation: A Guide for Designing Useful Computer Systems. London: Chapman & Hall Computing.

[22]

Malone, T. W. 1980. What Makes Things Fun to Learn? Heuristics for Designing Instructional Computer Games. Paper presented at the 3rd ACM SIGSMALL Symposium and 1st SIGPC Symposium on Small Systems.

Digital Library

[23]

Malone, T. W. 1981. Toward a Theory of Intrinsically Motivating Instruction. Cognitive Science, 5(4), 333--369.

[24]

Mayhew, D. J. 1992. Principles and Guidelines in Software User Interface Design. New Jersey: Prentice Hall, Inc.

Digital Library

[25]

Molich, R., & Dumas, J. S. 2008. Comparative Usability Evaluation (CUE-4). Behaviour & Information Technology, 27(3), 263--281.

Digital Library

[26]

Nielsen, J. 1993. Usability Engineering. Boston: Academic Press, Inc.

Digital Library

[27]

Nielsen, J. 1994. Heuristic Evaluation. In J. Nielsen & R. L. Mack (Eds.), Usability Inspection Methods. New York: John Wiley & Sons, Inc.

Digital Library

[28]

Nielsen, J. 2003. Usability 101: Introduction to Usability Retrieved 20 July 2009, from http://www.useit.com/alertbox/20030825.html

[29]

Norman, D. A. 2001. The Design of Everyday Things. London: MIT Press.

Digital Library

[30]

Preece, J., Rogers, Y., & Sharp, H. 2007. Interaction Design: Beyond Human-Computer Interaction (2^nd ed.). Chichester: John Wiley & Sons Ltd.

Digital Library

[31]

Pühretmair, F., & Miesenberger. 2005. Making sense of Accessibility in IT Design - Usable Accessibility vs. Accessible Usability. Paper presented at the Database and Expert Systems Applications (DEXA'05).

[32]

Rogoff, R. 2001, 24--27 Oct 2001. Making Electronic Information Accessible to Everyone. Paper presented at the Professional Communication Conference, (IPCC 2001).

[33]

Rubin, J. 1994. Handbook of Usability Testing: How to plan, design, and conduct effective tests. New York: John Wiley & Sons.

Digital Library

[34]

Sears, A. 1997. Heuristic Walkthroughs: Finding the Problems Without the Noise. International Journal of Human-Computer Interaction, 9(3), 213--234.

[35]

Shelley, B. 2001. Guidelines for Developing Successful Games Retrieved 10 March 2010, from http://jnoodle.com/careertech/files/GuidelinesDevelopingSuccessfulGames.pdf

[36]

Shneiderman, B. 1998. Designing the User Interface: strategies for Effective Human-Computer Interaction (3^rd ed.). New York: Addison Wesley.

Digital Library

[37]

Story, M. F., Mueller, J. L., & Mace, R. L. 1998. The Universal Design File: Designing for People of All Ages and Abilities Retrieved 17 November 2009, from http://eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/19/ac/11.pdf

[38]

United States Access Board. 2000. Electronic and Information Technology Accessibility Standards (Section 508) Retrieved 09 November 2009, from http://www.access-board.gov/sec508/standards.htm

[39]

Vanderheiden, G. C. 1994. Application Software Design Guidelines: Increasing the Accessibility of Application Software to People with Disabilities and Older Users Retrieved 04 November 2009, from http://trace.wisc.edu/docs/software_guidelines/software.htm

[40]

Wharton, C., Braffort, J., Jeffries, R., & Franzke, M. 1992. Applying Cognitive Walkthroughs to More Complex User Interfaces: Experiences, Issues, and Recommendations. Paper presented at the SIGCHI Conference on Human factors in Computing Systems.

Digital Library

[41]

World Wide Web Consortium. 1999. Web Content Accessibility Guidelines 1.0 Retrieved 7 August 2009, 2009, from http://www.w3.org/TR/WAI-WEBCONTENT/

Cited By

Drzyzga G(2024)Incorporating Artificial Intelligence into Design Criteria ConsiderationsArtificial Intelligence in HCI10.1007/978-3-031-60611-3_10(133-151)Online publication date: 29-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-60611-3_10
(2018)SMARTPOWERCHAIR:TOBOLDLYGOWHEREA POWERCHAIRHASNOTGONEBEFOREContemporary Ergonomics and Human Factors 201510.1201/b18293-37(249-258)Online publication date: 8-Jun-2018
https://doi.org/10.1201/b18293-37
Whittington PDogan H(2017)SmartPowerchair: Characterization and Usability of a Pervasive System of SystemsIEEE Transactions on Human-Machine Systems10.1109/THMS.2016.261628847:4(500-510)Online publication date: Aug-2017
https://doi.org/10.1109/THMS.2016.2616288
Show More Cited By

Index Terms

The complementary role of two evaluation methods in the usability and accessibility evaluation of a non-standard system
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
  2. Interaction design
    1. Interaction design process and methods
      1. User centered design

Recommendations

The relationship between accessibility and usability of websites
CHI '07: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Accessibility and usability are well established concepts for user interfaces and websites. Usability is precisely defined, but there are different approaches to accessibility. In addition, different possible relationships could exist between problems ...
Accessibility and usability evaluation of university websites in Afghanistan: a comparison between public and private universities
Abstract
The world wide web as a service offered on top of the internet has become the information repository through which we access the web pages of each individual or organization. To make the content accessible for users with disabilities, the World ...
A4U - an approach to evaluation considering accessibility and usability guidelines
WebMedia '18: Proceedings of the 24th Brazilian Symposium on Multimedia and the Web

The Web presents an extensive information content available to diverse people, with different abilities and needs. For this reason, the evaluation for the universal accessibility and the usability are difficult tasks, even though having automatic tools ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SAICSIT '10: Proceedings of the 2010 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists

October 2010

447 pages

ISBN:9781605589503

DOI:10.1145/1899503

Conference Chair:
Paula Kotzé
CSIR Meraka Institute, Pretoria, South Africa
,
Program Chairs:
Alta van der Merwe
CSIR Meraka Institute, Pretoria, South Africa
,
Aurona Gerber
CSIR Meraka Institute, Pretoria, South Africa

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

AAIICT: African Advanced Institute for Information & Communications Technology
NRF
ACM: Association for Computing Machinery
CSIR: CSIR
Microsoft: Microsoft
ASFKP-ICT: South Africa-Finland Knowledge Partnership on ICT
SAICSIT: So. African Inst. Of Computer Scientists & Info Tecnologists
Telkom: Telkom
IFIP

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 October 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SAICSIT '10

Sponsor:

AAIICT
ACM
CSIR
Microsoft
ASFKP-ICT
SAICSIT
Telkom

SAICSIT '10: 2010 Annual Conference of the South African Institute of Computer Scientists and Information Technologists

October 11 - 13, 2010

Bela Bela, South Africa

Acceptance Rates

Overall Acceptance Rate 187 of 439 submissions, 43%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
551
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)5

Reflects downloads up to 17 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Drzyzga G(2024)Incorporating Artificial Intelligence into Design Criteria ConsiderationsArtificial Intelligence in HCI10.1007/978-3-031-60611-3_10(133-151)Online publication date: 29-Jun-2024
https://dl.acm.org/doi/10.1007/978-3-031-60611-3_10
(2018)SMARTPOWERCHAIR:TOBOLDLYGOWHEREA POWERCHAIRHASNOTGONEBEFOREContemporary Ergonomics and Human Factors 201510.1201/b18293-37(249-258)Online publication date: 8-Jun-2018
https://doi.org/10.1201/b18293-37
Whittington PDogan H(2017)SmartPowerchair: Characterization and Usability of a Pervasive System of SystemsIEEE Transactions on Human-Machine Systems10.1109/THMS.2016.261628847:4(500-510)Online publication date: Aug-2017
https://doi.org/10.1109/THMS.2016.2616288
Casare Ada Silva CMartins PMoraes ROssowski S(2016)Usability heuristics and accessibility guidelinesProceedings of the 31st Annual ACM Symposium on Applied Computing10.1145/2851613.2851913(213-215)Online publication date: 4-Apr-2016
https://dl.acm.org/doi/10.1145/2851613.2851913
Whittington PDogan H(2015)SmartPowerchair: A pervasive system of systems2015 10th System of Systems Engineering Conference (SoSE)10.1109/SYSOSE.2015.7151932(244-249)Online publication date: May-2015
https://doi.org/10.1109/SYSOSE.2015.7151932
Assila ASfax TBouhlel M(2013)A web questionnaire generating tool to aid for interactive systems quality subjective assessment2013 International Conference on Control, Decision and Information Technologies (CoDIT)10.1109/CoDIT.2013.6689648(815-821)Online publication date: May-2013
https://doi.org/10.1109/CoDIT.2013.6689648
Carneiro Mde Queiroz JFechine J(2013)Including uncertainty treatment on the accessibility assessment of DOSVOX systemProceedings of the 7th international conference on Universal Access in Human-Computer Interaction: design methods, tools, and interaction techniques for eInclusion - Volume Part I10.1007/978-3-642-39188-0_50(464-473)Online publication date: 21-Jul-2013
https://dl.acm.org/doi/10.1007/978-3-642-39188-0_50

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents