research-article

Usability evaluation of ambient assisted living systems using a multi-method approach

Authors:

Ana Isabel Martins,

Alexandra Queirós,

Anabela G. Silva,

Nelson Pacheco RochaAuthors Info & Claims

DSAI '16: Proceedings of the 7th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion

Pages 261 - 268

https://doi.org/10.1145/3019943.3019981

Published: 01 December 2016 Publication History

Abstract

Background -- The assessment of the usability of Ambient Assisted Living (AAL) products and services is a demanding process due to their complexity. Objective -- To verify the feasibility of using a multi-method approach for data collection when evaluating the usability of AAL products and services. Methods -- An experimental study was conducted using two questionnaires, the Post-Study System Usability Questionnaire (PSSUQ) and the International Classification of Functionality, Disability and Health based Usability Scale (ICF-US), together with the registration of quantitative data related to the performance of the users carrying out specific tasks, and the registration of critical incidents. Results -- The experimental results suggest that the PSSUQ (to determine self-perceived usability) and the ICF-US (to determine the opinion of the evaluator on the users' performance) promote an accurate evaluation of the interaction mechanisms. Conclusion -- Multiple data collection methods seems to be an efficient approach for usability evaluation of AAL products and services.

References

[1]

Afonso, A. P., Lima, J. and Cota, M. P. 2013. Usability Assessment of Web Interfaces: User Testing. In Conferência Ibérica de Sistemas e Tecnologias de Informacão, 1, 1--7.

[2]

Aviles-Lopez, E., Villanueva-Miranda, I., Garcia-Macias, J. A. and Palafox-Maestre, L. E. 2009. Taking Care of Our Elders Through Augmented Spaces. In Latin American Web Congress, 16--21.

Digital Library

[3]

Broek, G., Cavallo, F. Oddetti, L. and Wehrmann, C. 2009. Ambient Assisted Living Roadmap. VDI/VDE-IT AALIANCE Office, Berlin.

[4]

Camarinha-Matos, L. and Vieira W. 1999. Intelligent mobile agents in elderly care. Robotics and Autonomous Systems 27, 1-2, 59--75.

[5]

Cook D. and Das, S. 2007. How Smart are our Environments? An Updated Look at the State of the Art. Pervasive and Mobile Computing 3, 2, 53--73.

Digital Library

[6]

Costa, R., Carneiro, D., Novais, P., Lima, L., Machado, J., Marques, A. and Neves, J. 2009. Ambient assisted living. In 3rd Symposium of Ubiquitous Computing and Ambient Intelligence 2008, 86--94. Springer Berlin Heidelberg.

[7]

Costa, R. 2009. Collaborative networks in ambient assisted living. Universidade do Minho.

[8]

De Carolis B. and Cozzolongo, G. 2007. Planning the Behaviour of a Social Robot Acting as a Majordomo in Public Environments. In Proceedings of the 10th Congress of the Italian Association for Artificial Intelligence 2007: Artificial Intelligence and Human-Oriented Computing, 805--12.

Digital Library

[9]

De Paola, A., Farruggia, A., Gaglio, S., Lo Re, G. and Ortolani, M. Exploiting the Human Factor in a WSN-Based System for Ambient Intelligence. 2009. In 2009 International Conference on Complex, Intelligent and Software Intensive Systems, 748--53.

[10]

Figueiredo, C. P., Becher, K., Hoffmann, K.-P. and Mendes P. M. 2010. Low power wireless acquisition module for wearable health monitoring systems. In 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, 704--7.

[11]

Georgsson, M. and Staggers, N. 2016. An evaluation of patients' experienced usability of a diabetes mHealth system using a multi-method approach. Journal of biomedical informatics 59, 115--129.

Digital Library

[12]

Hanington B. and Martin, B. 2012. Universal Methods of Design: 100 Ways to Research Complex Problems, Develop Innovative Ideas, and Design Effective Solutions. Beverly, MA: Rockport Publishers.

[13]

Hoppe, A. 2011. Technological Stress: Mental Strain of Younger and Older Users if Technology Fails. In Ambient Asssisted Living. Germany: Springer, 2011, pp. 17--29.

[14]

ISTAG - Information Society Technologies Advisory Group. 2009. ISTAG's Report on Orientations for Work Programme 2011--2013. Brussels.

[15]

Islam M. and Logeeswaran, V. 2010. Nanoscale materials and devices for future communication networks. IEEE Communications Magazine, 48, 6, 112--20.

Digital Library

[16]

Koch, J., Jung, H. Wettach, J., Nemeth, G. and Berns, K. 2008. Dynamic Speech Interaction for Robotic Agents. In Recent Progress in Robotics: Viable Robotic Service to Human, 303--15. Berlin, Heidelberg: Springer.

[17]

Lewis, J. R. 2002. Psychometric Evaluation of the PSSUQ Using Data from Five Years of Usability Studies. International Journal of Human--Computer Interaction 14, 3-4 463--88.

[18]

Mahatody, T., Kolski, C. and Sagar, M. 2009. CWE: Assistance Environment for the Evaluation Operating a Set of Variations of the Cognitive Walkthrough Ergonomic Inspection Method. In International Conference on Engineering Psychology and Cognitive Ergonomics, 52--61. Springer Berlin Heidelberg.

Digital Library

[19]

Martins, A. I., Queirós, A., Rocha, N. P. and Santos, B. S. 2013. Usability evaluation: A systematic literature review {Avaliação de usabilidade: Uma revisão sistemática da literatura}. RISTI - Revista Iberica de Sistemas e Tecnologias de Informacao, 11, 1, 31--44.

[20]

Martins, A. I., Rosa, A. F., Queirós, A., Silva, A. and Rocha, N. P. 2015. Definition and Validation of the ICF --Usability Scale. Procedia Computer Science, 67, 132--139.

[21]

Mitchell, P. A. 2007. Step-by-step Guide to Usability Testing. USA: iUniverse.

Digital Library

[22]

Nielsen, J. 1993. Usability Engineering. Boston: Academic Press.

Digital Library

[23]

Nielsen, J., and Landauer, T. K. 1993. A mathematical model of the finding of usability problems. In Proceedings of the INTERACT'93 and CHI'93 conference on Human factors in computing systems, 206--213.

Digital Library

[24]

Nielsen, J. 1995. How to Conduct a Heuristic Evaluation. 1995. {Online}. Available: http://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/.

[25]

Rosa, F., Martins, A. I., Queirós, A., Silva, A. and Rocha, N. P. 2105. European Portuguese validation of the Post-Study System Usability Questionnaire (PSSUQ) {Validação para Português Europeu do Post-Study System Usability Questionnaire (PSSUQ)}. In 2015 10th Iberian Conference on Information Systems and Technologies, CISTI 2015, art. no. 7170431, 2015.

[26]

Schneidert, M., Hurst, R. Miller, J. and Ustun, B. 2003. The role of environment in the International Classification of Functioning, Disability and Health (ICF). Disability and Rehabilitation 25, 11-12, 588--95.

[27]

Simes-Marques M. and Nunes, I. L. 2012. Usability of Interfaces. In Ergonomics -- A Systems Approach. InTech, 155--170.

[28]

WHO. 2001. International Classification of Functioning, Disability and Health (ICF). Geneve.

[29]

WHO. 2002. Active Ageing: A Policy Framework. A contribution of the World Health Organization to the second United Nations World Assembly on Aging. Geneve.

[30]

Wilkinson, S. 2003. Focus Groups in Qualitative Psychology -- A Practical Guide to Research Methods. London: Sage Publications.

[31]

Queirós, A., Carvalho, S., Pavão, J. and Da Rocha, N.P. 2013. AAL information based services and care integration. In HEALTHINF 2013 - Proceedings of the International Conference on Health Informatics, 403--406.

[32]

Queirós, A., Silva, A.G., Alvarelhão, J., Teixeira, A. and da Rocha, N.P. 2013. A conceptual framework for the design and development of AAL services. In Handbook of Research on ICTs for Human-Centered Healthcare and Social Care Services, 568--586.

[33]

Queirós, A., Silva, A., Alvarelhão, J., Rocha, N. P. and Teixeira, A. 2015. Usability, accessibility and ambient-assisted living: a systematic literature review. Universal Access in the Information Society 14, 5, 57--66.

Digital Library

Cited By

Ntoa S(2024)Usability and User Experience Evaluation in Intelligent Environments: A Review and ReappraisalInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2394724(1-30)Online publication date: 12-Sep-2024
https://doi.org/10.1080/10447318.2024.2394724
Colantonio SJovanovic MZdravevski ELameski PTellioglu HKampel MFlorez-Revuelta FMakedon F(2022)Are Active and Assisted Living applications addressing the main acceptance concerns of their beneficiaries? Preliminary insights from a scoping reviewProceedings of the 15th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3529190.3534753(414-421)Online publication date: 29-Jun-2022
https://dl.acm.org/doi/10.1145/3529190.3534753
Lima AVieira MMartins ARocha NMendes JFerreira R(2021)Impact of Evaluating the Usability of Assisted Technology Oriented by ProtocolResearch Anthology on Digital Transformation, Organizational Change, and the Impact of Remote Work10.4018/978-1-7998-7297-9.ch026(501-522)Online publication date: 2021
https://doi.org/10.4018/978-1-7998-7297-9.ch026
Show More Cited By

Recommendations

An ambient assisted living nighttime wandering system for elderly
iiWAS '16: Proceedings of the 18th International Conference on Information Integration and Web-based Applications and Services

The Assistive living technologies provide good results for the support of specific activities transforming a home into a smart home. In this paper, we present a personalized ambient support system for elderly suffering from Alzheimer's dementia and ...
Bootstrapping activity modeling for ambient assisted living
ICSH'13: Proceedings of the 2013 international conference on Smart Health

In many societies, the age profile of the population is increasing, posing many challenges for societies, health services and carers. One response to this unfolding situation has been to direct research effort towards Ambient Assisted Living (AAL), ...
Promises and Challenges of Ambient Assisted Living Systems
ITNG '09: Proceedings of the 2009 Sixth International Conference on Information Technology: New Generations

The population of elderly people keeps increasing rapidly, which becomes a predominant aspect of our societies. As such, solutions both efficacious and cost-effective need to be sought. Ambient Assisted Living (AAL) is a new approach which promises to ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

DSAI '16: Proceedings of the 7th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion

December 2016

440 pages

ISBN:9781450347488

DOI:10.1145/3019943

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 December 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

DSAI 2016

DSAI 2016: 7th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion

December 1 - 3, 2016

Vila Real, Portugal

Acceptance Rates

Overall Acceptance Rate 17 of 23 submissions, 74%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
142
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ntoa S(2024)Usability and User Experience Evaluation in Intelligent Environments: A Review and ReappraisalInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2394724(1-30)Online publication date: 12-Sep-2024
https://doi.org/10.1080/10447318.2024.2394724
Colantonio SJovanovic MZdravevski ELameski PTellioglu HKampel MFlorez-Revuelta FMakedon F(2022)Are Active and Assisted Living applications addressing the main acceptance concerns of their beneficiaries? Preliminary insights from a scoping reviewProceedings of the 15th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3529190.3534753(414-421)Online publication date: 29-Jun-2022
https://dl.acm.org/doi/10.1145/3529190.3534753
Lima AVieira MMartins ARocha NMendes JFerreira R(2021)Impact of Evaluating the Usability of Assisted Technology Oriented by ProtocolResearch Anthology on Digital Transformation, Organizational Change, and the Impact of Remote Work10.4018/978-1-7998-7297-9.ch026(501-522)Online publication date: 2021
https://doi.org/10.4018/978-1-7998-7297-9.ch026
Jegundo ADantas CQuintas JDutra JAlmeida ACaravau HRosa AMartins APacheco Rocha N(2020)Perceived Usefulness, Satisfaction, Ease of Use and Potential of a Virtual Companion to Support the Care Provision for Older AdultsTechnologies10.3390/technologies80300428:3(42)Online publication date: 25-Jul-2020
https://doi.org/10.3390/technologies8030042
Lima AVieira MMartins ARocha NMendes JFerreira R(2019)Impact of Evaluating the Usability of Assisted Technology Oriented by ProtocolHandbook of Research on Human-Computer Interfaces and New Modes of Interactivity10.4018/978-1-5225-9069-9.ch016(275-296)Online publication date: 2019
https://doi.org/10.4018/978-1-5225-9069-9.ch016
Martins ACerqueira M(2019)Ambient Assisted Living – A Multi-method Data Collection Approach to Evaluate the Usability of AAL SolutionsUsability, Accessibility and Ambient Assisted Living10.1007/978-3-319-91226-4_5(65-74)Online publication date: 22-Jan-2019
https://doi.org/10.1007/978-3-319-91226-4_5
Jegundo ADantas CQuintas JDutra JAlmeida ACaravau HRosa AMartins AQueirós ARocha N(2019)Usability Evaluation of a Virtual Assistive CompanionNew Knowledge in Information Systems and Technologies10.1007/978-3-030-16184-2_67(706-715)Online publication date: 30-Mar-2019
https://doi.org/10.1007/978-3-030-16184-2_67
Queirós ADias ASilva ARocha N(2017)Ambient Assisted Living and Health-Related Outcomes—A Systematic Literature ReviewInformatics10.3390/informatics40300194:3(19)Online publication date: 10-Jul-2017
https://doi.org/10.3390/informatics4030019

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents