research-article

Do Android app users care about accessibility?: an analysis of user reviews on the Google play store

Authors:

Marcelo Medeiros Eler,

Leandro Orlandin,

Alberto Dumont Alves OliveiraAuthors Info & Claims

IHC '19: Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems

Article No.: 23, Pages 1 - 11

https://doi.org/10.1145/3357155.3358477

Published: 22 October 2019 Publication History

Abstract

Mobile applications (or apps) have become a popular platform to provide users with information, digital content and services. Unfortunately, many studies have shown that even popular mobile applications still have many accessibility issues. A particular characteristic of mobile development is that developers tend to take into account comments published on official app stores to plan for new releases of their apps. Some researchers have classified user reviews into different categories, including user interface issues, but none of them have investigated whether accessibility is a topic addressed by user comments. Therefore, the purpose of this paper is to present an investigation we carried out to find out whether and how accessible is addressed by user reviews published on app stores. In our investigation, we analysed 701 Android apps available on the Google Play Store and found out that reviews associated with accessibility are scarce (1.2% of all reviews); user reviews usually refer to a small subset of accessibility issues; and users give reasonable good scores to mobile apps even when they report accessibility issues, with the exception of very specific cases. It seems our results show that user reviews have not been used to suggest developers to improve mobile apps accessibility.

References

[1]

BBC. 2017. The BBC Standards and Guidelines for Mobile Accessibility. https://www.bbc.co.uk/guidelines/futuremedia/accessibility/mobile

[2]

Lucas Pedroso Carvalho, Bruno Piovesan Melchiori Peruzza, Flávia Santos, Lucas Pereira Ferreira, and André Pimenta Freire. 2016. Accessible Smart Cities?: Inspecting the Accessibility of Brazilian Municipalities' Mobile Applications. In Proceedings of the 15th Brazilian Symposium on Human Factors in Computer Systems (IHC '16). ACM, New York, NY, USA, Article 17, 10 pages.

Digital Library

[3]

A. Ciurumelea, A. Schaufelbühl, S. Panichella, and H. C. Gall. 2017. Analyzing reviews and code of mobile apps for better release planning. In 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER). 91--102.

[4]

A. Di Sorbo, S. Panichella, C. V. Alexandra, C. A. Visaggio, and G. Canfora. 2017. SURF: Summarizer of User Reviews Feedback. In 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C). 55--58.

Digital Library

[5]

M. M. Eler, J. M. Rojas, Y. Ge, and G. Fraser. 2018. Automated Accessibility Testing of Mobile Apps. In 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST). 116--126.

[6]

Google. 2014. Material Design. https://material.io/design/

[7]

Brazilian Federal Government. 2014. eMAG - Modelo de Acessibilidade em Governo Eletrônico. http://emag.governoeletronico.gov.br/

[8]

J. E. Hannay, H. C. Benestad, and K. Strand. 2017. Benefit Points: The Best Part of the Story. IEEE Software 34, 3 (May 2017), 73--85.

Digital Library

[9]

C. Iacob and R. Harrison. 2013. Retrieving and analyzing mobile apps feature requests from online reviews. In 2013 10th Working Conference on Mining Software Repositories (MSR). 41--44.

[10]

Claudia Iacob, Rachel Harrison, and Shamal Faily. 2014. Online Reviews as First Class Artifacts in Mobile App Development. In Mobile Computing, Applications, and Services, Gérard Memmi and Ulf Blanke (Eds.). Springer International Publishing, Cham, 47--53.

[11]

Xiaozhou Li, Zheying Zhang, and Kostas Stefanidis. 2018. Mobile App Evolution Analysis Based on User Reviews. In SoMeT.

[12]

Stuart Mcilroy, Nasir Ali, Hammad Khalid, and Ahmed E. Hassan. 2016. Analyzing and Automatically Labelling the Types of User Issues That Are Raised in Mobile App Reviews. Empirical Softw. Engg. 21, 3 (June 2016), 1067--1106.

Digital Library

[13]

M. Nayebi, B. Adams, and G. Ruhe. 2016. Release Practices for Mobile Apps - What do Users and Developers Think?. In 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), Vol. 1. 552--562.

[14]

A. D. A. Oliveira and M. M. Eler. 2017. Strategies and Challenges on the Accessibility and Interoperability of e-Government Web Portals: A Case Study on Brazilian Federal Universities. In 2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC), Vol. 1. 737--742.

[15]

Adriana Emitza Guzman Ortega. 2015. Mining User Reviews from Mobile Applications for Software Evolution. Ph.D. Dissertation. Technical University of Munich.

[16]

D. Pagano and W. Maalej. 2013. User feedback in the appstore: An empirical study. In 2013 21st IEEE International Requirements Engineering Conference (RE). 125--134.

[17]

F. Palomba, M. Linares-Vásquez, G. Bavota, R. Oliveto, M. Di Penta, D. Poshyvanyk, and A. De Lucia. 2015. User reviews matter! Tracking crowdsourced reviews to support evolution of successful apps. In 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME). 291--300.

Digital Library

[18]

Fabio Palomba, Mario Linares-Vásquez, Gabriele Bavota, Rocco Oliveto, Massimiliano Di Penta, Denys Poshyvanyk, and Andrea De Lucia. 2018. Crowdsourcing user reviews to support the evolution of mobile apps. Journal of Systems and Software 137 (2018), 143 -- 162.

[19]

S. Panichella, A. Di Sorbo, E. Guzman, C. A. Visaggio, G. Canfora, and H. C. Gall. 2015. How can i improve my app? Classifying user reviews for software maintenance and evolution. In 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME). 281--290.

Digital Library

[20]

L. Pelloni, G. Grano, A. Ciurumelea, S. Panichella, F. Palomba, and H. C. Gall. 2018. BECLoMA: Augmenting stack traces with user review information. In 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER). 522--526.

[21]

Fiamma Eva Mendoza Quispe and Marcelo Medeiros Eler. 2018. Accessibility Recommendations for Mobile Applications: A Contribution to the Brazilian Digital Government Standards. In Proceedings of the XIV Brazilian Symposium on Information Systems (SBSI'18). ACM, New York, NY, USA, Article 67, 8 pages.

Digital Library

[22]

Leandro Coelho Serra, Lucas Pedroso Carvalho, Lucas Pereira Ferreira, Jorge Belimar Silva Vaz, and André Pimenta Freire. 2015. Accessibility Evaluation of E-Government Mobile Applications in Brazil. Procedia Computer Science - Proceedings of the 6th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion 67 (2015), 348 -- 357.

[23]

Camila Silva, Marcelo Medeiros Eler, and Gordon Fraser. 2018. A Survey on the Tool Support for the Automatic Evaluation of Mobile Accessibility. In Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion (DSAI 2018). ACM, New York, NY, USA, 286--293.

Digital Library

[24]

W3C. 2018. Web Content Accessibility Guidelines (WCAG) 2.1. https://www.w3.org/TR/WCAG21/

[25]

W3C-WAI. 2015. Accessibility Fundamentals - Introduction to Web Accessibility. https://www.w3.org/standards/webdesign/accessibility

[26]

Shunguo Yan and P. G. Ramachandran. 2019. The Current Status of Accessibility in Mobile Apps. ACM Trans. Access. Comput. 12, 1, Article 3 (Feb. 2019), 31 pages.

Digital Library

Cited By

Oliveira ASantos PAljedaani WEler M(2024)Exploring the Influence of Software Evolution on Mobile App Accessibility: Insights from User ReviewsJournal of the Brazilian Computer Society10.5753/jbcs.2024.432130:1(584-607)Online publication date: 2-Nov-2024
https://doi.org/10.5753/jbcs.2024.4321
Yang THasan RAyday EVaidya J(2024)Discovering Privacy Harms from Education Technology by Analyzing User ReviewsProceedings of the 23rd Workshop on Privacy in the Electronic Society10.1145/3689943.3695050(186-192)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3689943.3695050
Alghamdi AAljedaani WEler MLudi S(2024)Accessibility Guidelines and Standards: Analyzing Stack Overflow PostsProceedings of the 21st International Web for All Conference10.1145/3677846.3677857(118-122)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3677846.3677857
Show More Cited By

Index Terms

Do Android app users care about accessibility?: an analysis of user reviews on the Google play store
1. Human-centered computing
  1. Accessibility
    1. Empirical studies in accessibility
  2. Ubiquitous and mobile computing
    1. Ubiquitous and mobile devices
2. Software and its engineering
  1. Software creation and management
    1. Software post-development issues
      1. Software evolution

Recommendations

Mining User Reviews for Mobile App Comparisons

As the number of mobile apps keeps increasing, users often need to compare many apps, in order to choose one that best fits their needs. Fortunately, as there are so many users sharing an app market, it is likely that some other users with the same ...
Learn Android App Development
Android App Development For Dummies

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

IHC '19: Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems

October 2019

679 pages

ISBN:9781450369718

DOI:10.1145/3357155

General Chairs:
Rodrigo Laiola Guimaraes
Universidade Federal do Espirito Santo (UFES)
,
Celso Alberto Saibel Santos
Universidade Federal do Espirito Santo (UFES)
,
Program Chairs:
Vagner Figueredo de Santana
IBM Research
,
Artur Kronbauer
Universidade Salvador (UNIFACS)

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SBC: Brazilian Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 October 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Fundação de Amparo à Pesquisa do Estado de São Paulo

Conference

IHC '19

Sponsor:

SBC

IHC '19: XVIII Brazilian Symposium on Human Factors in Computing Systems

October 22 - 25, 2019

Espírito Santo, Vitória, Brazil

Acceptance Rates

IHC '19 Paper Acceptance Rate 56 of 165 submissions, 34%;

Overall Acceptance Rate 331 of 973 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
580
Total Downloads

Downloads (Last 12 months)63
Downloads (Last 6 weeks)7

Reflects downloads up to 01 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Oliveira ASantos PAljedaani WEler M(2024)Exploring the Influence of Software Evolution on Mobile App Accessibility: Insights from User ReviewsJournal of the Brazilian Computer Society10.5753/jbcs.2024.432130:1(584-607)Online publication date: 2-Nov-2024
https://doi.org/10.5753/jbcs.2024.4321
Yang THasan RAyday EVaidya J(2024)Discovering Privacy Harms from Education Technology by Analyzing User ReviewsProceedings of the 23rd Workshop on Privacy in the Electronic Society10.1145/3689943.3695050(186-192)Online publication date: 20-Nov-2024
https://dl.acm.org/doi/10.1145/3689943.3695050
Alghamdi AAljedaani WEler MLudi S(2024)Accessibility Guidelines and Standards: Analyzing Stack Overflow PostsProceedings of the 21st International Web for All Conference10.1145/3677846.3677857(118-122)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3677846.3677857
Aljedaani WMkaouer MEler MKessentini M(2024)Empirical Investigation of Accessibility Bug Reports in Mobile Platforms: A Chromium Case StudyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642508(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642508
Wang XZhang TTan YShang WLi Y(2024)How to effectively mine app reviews concerning software ecosystem? A survey of review characteristicsJournal of Systems and Software10.1016/j.jss.2024.112040213(112040)Online publication date: Jul-2024
https://doi.org/10.1016/j.jss.2024.112040
Alrizq MAlghamdi A(2024)Customer satisfaction analysis with Saudi Arabia mobile banking apps: a hybrid approach using text mining and predictive learning techniquesNeural Computing and Applications10.1007/s00521-023-09400-436:11(6005-6023)Online publication date: 17-Jan-2024
https://doi.org/10.1007/s00521-023-09400-4
Dos Santos POliveira ADe Jesus TAljedaani WEler M(2023)Evolution may come with a price: analyzing user reviews to understand the impact of updates on mobile apps accessibilityProceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems10.1145/3638067.3638081(1-11)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3638067.3638081
Da Costa Nunes ERibeiro GMonteiro IGonçalves E(2023)Digital Accessibility at the Brazilian Symposium on Human Factors in Computing Systems (IHC): An Updated Systematic Literature ReviewProceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems10.1145/3638067.3638075(1-15)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3638067.3638075
O'Hagan JMathis FMcGill M(2023)User Reviews as a Reporting Mechanism for Emergent Issues Within Social VR CommunitiesProceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia10.1145/3626705.3627780(236-243)Online publication date: 3-Dec-2023
https://dl.acm.org/doi/10.1145/3626705.3627780
Aljedaani WAlkahtani MLudi SMkaouer MEler MKessentini MOuni A(2023)The State of Accessibility in Blackboard: Survey and User Reviews Case StudyProceedings of the 20th International Web for All Conference10.1145/3587281.3587291(84-95)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3587281.3587291
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents