research-article

Empirical Investigation of Accessibility Bug Reports in Mobile Platforms: A Chromium Case Study

Authors:

Wajdi Aljedaani,

Mohamed Wiem Mkaouer,

Marcelo Medeiros Eler,

Marouane KessentiniAuthors Info & Claims

CHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems

Article No.: 967, Pages 1 - 17

https://doi.org/10.1145/3613904.3642508

Published: 11 May 2024 Publication History

Abstract

Accessibility is an important quality factor of mobile applications. Many studies have shown that, despite the availability of many resources to guide the development of accessible software, most apps and web applications contain many accessibility issues. Some researchers surveyed professionals and organizations to understand the lack of accessibility during software development, but few studies have investigated how developers and organizations respond to accessibility bug reports. Therefore, this paper analyzes accessibility bug reports posted in the Chromium repository to understand how developers and organizations handle them. More specifically, we want to determine the frequency of accessibility bug reports over time, the time-to-fix compared to traditional bug reports (e.g., functional bugs), and the types of accessibility barriers reported. Results show that the frequency of accessibility reports has increased over the years, and accessibility bugs take longer to be fixed, as they tend to be given low priority.

Supplemental Material

MP4 File - Video Presentation

Video Presentation

Transcript for: Video Presentation

References

[1]

IBM Accessibility. 2017. IBM Accessibility Checklist for 7.0. https://www.ibm.com/able/guidelines/ci162/accessibility_checklist.html.

[2]

IBM Accessibility. 2021. Design, build, and test like an accessibility expert. https://www.ibm.com/able/.

[3]

Wajdi Aljedaani, Mohammed Alkahtani, Stephanie Ludi, Mohamed Wiem Mkaouer, Marcelo M Eler, Marouane Kessentini, and Ali Ouni. 2023. The state of accessibility in blackboard: Survey and user reviews case study. In Proceedings of the 20th International Web for All Conference. 84–95.

Digital Library

[4]

Wajdi Aljedaani and Yasir Javed. 2018. Bug Reports Evolution in Open Source Systems. In 5th International Symposium on Data Mining Applications. Springer, 63–73.

[5]

Wajdi Aljedaani, Yasir Javed, and Mamdouh Alenezi. 2020. LDA categorization of security bug reports in chromium projects. In Proceedings of the 2020 European symposium on software engineering. 154–161.

Digital Library

[6]

Wajdi Aljedaani, Yasir Javed, and Mamdouh Alenezi. 2020. Open source systems bug reports: Meta-analysis. In Proceedings of the 2020 The 3rd International Conference on Big Data and Education. 43–49.

Digital Library

[7]

Wajdi Aljedaani, Mohamed Wiem Mkaouer, Stephanie Ludi, and Yasir Javed. 2022. Automatic classification of accessibility user reviews in android apps. In 2022 7th international conference on data science and machine learning applications (CDMA). IEEE, 133–138.

[8]

Wajdi Aljedaani, Mohamed Wiem Mkaouer, Stephanie Ludi, Ali Ouni, and Ilyes Jenhani. 2022. On the identification of accessibility bug reports in open source systems. In Proceedings of the 19th international web for all conference. 1–11.

Digital Library

[9]

Wajdi Aljedaani, Meiyappan Nagappan, Bram Adams, and Michael Godfrey. 2019. A comparison of bugs across the iOS and Android platforms of two open source cross platform browser apps. In 2019 IEEE/ACM 6th International Conference on Mobile Software Engineering and Systems (MOBILESoft). IEEE, 76–86.

[10]

Wajdi Aljedaani, Furqan Rustam, Stephanie Ludi, Ali Ouni, and Mohamed Wiem Mkaouer. 2021. Learning sentiment analysis for accessibility user reviews. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW). IEEE, 239–246.

[11]

Bader Alkhazi, Andrew DiStasi, Wajdi Aljedaani, Hussein Alrubaye, Xin Ye, and Mohamed Wiem Mkaouer. 2020. Learning to rank developers for bug report assignment. Applied Soft Computing 95 (2020), 106667.

[12]

Eman Abdullah AlOmar, Wajdi Aljedaani, Murtaza Tamjeed, Mohamed Wiem Mkaouer, and Yasmine N El-Glaly. 2021. Finding the needle in a haystack: On the automatic identification of accessibility user reviews. In Proceedings of the 2021 CHI conference on human factors in computing systems. 1–15.

Digital Library

[13]

Abdulaziz Alshayban, Iftekhar Ahmed, and Sam Malek. 2020. Accessibility issues in Android apps: state of affairs, sentiments, and ways forward. In 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE). IEEE, 1323–1334.

Digital Library

[14]

Sean Banerjee, Jordan Helmick, Zahid Syed, and Bojan Cukic. 2015. Eclipse vs. mozilla: A comparison of two large-scale open source problem report repositories. In 2015 IEEE 16th International Symposium on High Assurance Systems Engineering. IEEE, 263–270.

Digital Library

[15]

BBC. 2020. The BBC Standards and Guidelines for Mobile Accessibility. https://www.bbc.co.uk/guidelines/futuremedia/accessibility/mobile.

[16]

Mario Luca Bernardi, Gerardo Canfora, Giuseppe A Di Lucca, Massimiliano Di Penta, and Damiano Distante. 2012. Do developers introduce bugs when they do not communicate? the case of eclipse and mozilla. In 2012 16th European Conference on Software Maintenance and Reengineering. IEEE, 139–148.

Digital Library

[17]

Nicolas Bettenburg, Sascha Just, Adrian Schröter, Cathrin Weiß, Rahul Premraj, and Thomas Zimmermann. 2007. Quality of bug reports in eclipse. In Proceedings of the 2007 OOPSLA workshop on eclipse technology eXchange. 21–25.

Digital Library

[18]

Nicolas Bettenburg, Sascha Just, Adrian Schröter, Cathrin Weiss, Rahul Premraj, and Thomas Zimmermann. 2008. What makes a good bug report?. In Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering. 308–318.

Digital Library

[19]

Pamela Bhattacharya, Liudmila Ulanova, Iulian Neamtiu, and Sai Charan Koduru. 2013. An empirical analysis of bug reports and bug fixing in open source android apps. In 2013 17th European Conference on Software Maintenance and Reengineering. IEEE, 133–143.

Digital Library

[20]

Tingting Bi, Xin Xia, David Lo, and Aldeida Aleti. 2021. A First Look at Accessibility Issues in Popular GitHub Projects. In 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 390–401.

[21]

Tingting Bi, Xin Xia, David Lo, John Grundy, Thomas Zimmermann, and Denae Ford. 2022. Accessibility in Software Practice: A Practitioner’s Perspective. ACM Trans. Softw. Eng. Methodol. 31, 4, Article 66 (jul 2022), 26 pages. https://doi.org/10.1145/3503508

Digital Library

[22]

United States Access Board. 2017. Information and Communication Technology (ICT) Final Standards and Guidelines. https://www.access-board.gov/guidelines-and-standards/communications-and-it/ about-the-ict-refresh/final-rule.

[23]

Jieshan Chen, Chunyang Chen, Zhenchang Xing, Xiwei Xu, Liming Zhut, Guoqiang Li, and Jinshui Wang. 2020. Unblind your apps: Predicting natural-language labels for mobile GUI components by deep learning. In 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE). IEEE, 322–334.

Digital Library

[24]

Sen Chen, Chunyang Chen, Lingling Fan, Mingming Fan, Xian Zhan, and Yang Liu. 2022. Accessible or Not? An Empirical Investigation of Android App Accessibility. IEEE Transactions on Software Engineering 48, 10 (2022), 3954–3968. https://doi.org/10.1109/TSE.2021.3108162

Digital Library

[25]

Adelina Ciurumelea, Andreas Schaufelbühl, Sebastiano Panichella, and Harald C Gall. 2017. Analyzing reviews and code of mobile apps for better release planning. In 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 91–102.

[26]

Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement 20, 1 (1960), 37–46.

[27]

Henrique Neves da Silva, Andre Takeshi Endo, Marcelo Medeiros Eler, Silvia Regina Vergilio, and Vinicius HS Durelli. 2020. On the Relation between Code Elements and Accessibility Issues in Android Apps. In Proceedings of the 5th Brazilian Symposium on Systematic and Automated Software Testing. 40–49.

Digital Library

[28]

Android Developers. 2021. Accessibility Developer Checklist. https://stuff.mit.edu/afs/sipb/project/android/docs/guide/topics/ui/accessibility/checklist.html.

[29]

Android Developers. 2021. Build more accessible apps. https://developer.android.com/guide/topics/ui/accessibility.

[30]

Apple Developers. 2021. iOS Human Interface Guidelines for Accessibility. https://developer.apple.com/design/human-interface-guidelines/.

[31]

Google Developers. 2021. Accessibility Developer Checklist. https://developers.google.com/web/fundamentals/accessibility.

[32]

Paulo Sérgio Henrique Dos Santos, Alberto Dumont Alves Oliveira, Thais Bonjorni Nobre De Jesus, Wajdi Aljedaani, and Marcelo Medeiros Eler. 2023. Evolution may come with a price: analyzing user reviews to understand the impact of updates on mobile apps accessibility. In Proceedings of the XXII Brazilian Symposium on Human Factors in Computing Systems. 1–11.

Digital Library

[33]

Marcelo Medeiros Eler, Leandro Orlandin, and Alberto Dumont Alves Oliveira. 2019. Do Android app users care about accessibility? an analysis of user reviews on the Google play store. In Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems. 1–11.

Digital Library

[34]

Joseph L Fleiss, Bruce Levin, and Myunghee Cho Paik. 2013. Statistical methods for rates and proportions. john wiley & sons.

[35]

Cassio Andrade Furukawa, Michele dos Santos Soares, Maria Istela Cagnin, and Débora Maria Barroso Paiva. 2022. Support for Accessible Software Coding: Results of a Rapid Literature Review. CLEI electronic journal 25, 3 (2022), 1–1.

[36]

Syed Fatiul Huq, Abdulaziz Alshayban, Ziyao He, and Sam Malek. 2023. #A11yDev: Understanding Contemporary Software Accessibility Practices from Twitter Conversations. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (2023). https://api.semanticscholar.org/CorpusID:258217085

Digital Library

[37]

iOS Chrome Releases. 2021. Google Chrome - The Fast and Secure Web Browser.https://www.appannie.com/apps/ios/app/chrome-web-browser-by-google/details/.

[38]

Foutse Khomh, Tejinder Dhaliwal, Ying Zou, and Bram Adams. 2012. Do faster releases improve software quality? an empirical case study of mozilla firefox. In 2012 9th IEEE Working Conference on Mining Software Repositories (MSR). IEEE, 179–188.

[39]

Manoel Victor Rodrigues Leite, Lilian Passos Scatalon, André Pimenta Freire, and Marcelo Medeiros Eler. 2021. Accessibility in the mobile development industry in Brazil: Awareness, knowledge, adoption, motivations and barriers. Journal of Systems and Software 177 (2021), 110942. https://doi.org/10.1016/j.jss.2021.110942

[40]

Barbara Leporini, Maria Claudia Buzzi, and Marina Buzzi. 2012. Interacting with mobile devices via VoiceOver: usability and accessibility issues. In Proceedings of the 24th Australian Computer-Human Interaction Conference. 339–348.

Digital Library

[41]

Amiya Kumar Maji, Kangli Hao, Salmin Sultana, and Saurabh Bagchi. 2010. Characterizing failures in mobile oses: A case study with android and symbian. In 2010 IEEE 21st International Symposium on Software Reliability Engineering. IEEE, 249–258.

Digital Library

[42]

Lauren R Milne, Cynthia L Bennett, and Richard E Ladner. 2014. The accessibility of mobile health sensors for blind users. (2014).

[43]

Hoda Naguib, Nitesh Narayan, Bernd Brügge, and Dina Helal. 2013. Bug report assignee recommendation using activity profiles. In 2013 10th Working Conference on Mining Software Repositories (MSR). IEEE, 22–30.

[44]

Anh Tuan Nguyen, Tung Thanh Nguyen, Jafar Al-Kofahi, Hung Viet Nguyen, and Tien N Nguyen. 2011. A topic-based approach for narrowing the search space of buggy files from a bug report. In 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011). IEEE, 263–272.

Digital Library

[45]

Alberto Dumont Alves Oliveira, Paulo Sérgio Henrique Dos Santos, Wilson Estécio Marcílio Júnior, Wajdi M Aljedaani, Danilo Medeiros Eler, and Marcelo Medeiros Eler. 2023. Analyzing Accessibility Reviews Associated with Visual Disabilities or Eye Conditions. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 37, 14 pages. https://doi.org/10.1145/3544548.3581315

Digital Library

[46]

S. Panichella, A. Di Sorbo, E. Guzman, C. A. Visaggio, G. Canfora, and H. C. Gall. 2015. How can i improve my app? Classifying user reviews for software maintenance and evolution. In 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME). 281–290. https://doi.org/10.1109/ICSM.2015.7332474

Digital Library

[47]

Android Chrome Releases. 2021. Google Chrome: Fast & Secure.https://www.appannie.com/apps/google-play/app/20600000234348/details/.

[48]

Chrome Releases. 2021. Release updates from the Chrome team. https://chromereleases.googleblog.com/search/label/Chrome%20for%20Android.

[49]

Willem Rens. 2017. Browser forensics: adblocker extensions.

[50]

Anne Spencer Ross, Xiaoyi Zhang, James Fogarty, and Jacob O Wobbrock. 2017. Epidemiology as a framework for large-scale mobile application accessibility assessment. In Proceedings of the 19th international ACM SIGACCESS conference on computers and accessibility. 2–11.

Digital Library

[51]

Anne Spencer Ross, Xiaoyi Zhang, James Fogarty, and Jacob O Wobbrock. 2018. Examining image-based button labeling for accessibility in Android apps through large-scale analysis. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility. 119–130.

Digital Library

[52]

Nasir Safdari, Hussein Alrubaye, Wajdi Aljedaani, Bladimir Baez Baez, Andrew DiStasi, and Mohamed Wiem Mkaouer. 2019. Learning to rank faulty source files for dependent bug reports. In Big Data: Learning, Analytics, and Applications, Vol. 10989. International Society for Optics and Photonics, 109890B.

[53]

Leandro Coelho Serra, Lucas Pedroso Carvalho, Lucas Pereira Ferreira, Jorge Belimar Silva Vaz, and André Pimenta Freire. 2015. Accessibility evaluation of e-government mobile applications in Brazil. Procedia Computer Science 67 (2015), 348–357.

[54]

Shubham Shakya and Mayank Dave. 2022. Analysis, detection, and classification of android malware using system calls. arXiv preprint arXiv:2208.06130 (2022).

[55]

Christopher Vendome, Diana Solano, Santiago Linán, and Mario Linares-Vásquez. 2019. Can everyone use my app? an empirical study on accessibility in android apps. In 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 41–52.

[56]

World Wide Web Consortium (W3C). 2008. Web Content Accessibility Guidelines (WCAG) 2.0. https://www.w3.org/TR/WCAG20/.

[57]

World Wide Web Consortium (W3C). 2021. Web Content Accessibility Guidelines (WCAG) 2.2. https://www.w3.org/TR/WCAG22/.

[58]

Web Accessibility Initiative (WAI). 2015. Mobile Accessibility: HowWCAG 2.0 and Other W3C/WAI Guidelines Apply to Mobile. http://www.w3.org/TR/mobile-accessibility-mapping.

[59]

Bruce N Walker, Brianna J Tomlinson, and Jonathan H Schuett. 2017. Universal design of mobile apps: Making weather information accessible. In International Conference on Universal Access in Human-Computer Interaction. Springer, 113–122.

[60]

Jifeng Xuan, He Jiang, Zhilei Ren, and Weiqin Zou. 2012. Developer prioritization in bug repositories. In 2012 34th International Conference on Software Engineering (ICSE). IEEE, 25–35.

[61]

Shunguo Yan and PG Ramachandran. 2019. The current status of accessibility in mobile apps. ACM Transactions on Accessible Computing (TACCESS) 12, 1 (2019), 1–31.

Digital Library

[62]

Jie Zhang, Xiaoyin Wang, Dan Hao, Bing Xie, Lu Zhang, and Hong Mei. 2015. A survey on bug-report analysis. Science China Information Sciences 58, 2 (2015), 1–24.

[63]

Yuxin Zhang, Sen Chen, Lingling Fan, Chunyang Chen, and Xiaohong Li. 2023. Automated and Context-Aware Repair of Color-Related Accessibility Issues for Android Apps. arXiv preprint arXiv:2308.09029 (2023).

[64]

Yaxin Zhao, Lina Gong, Wenhua Yang, and Yu Zhou. 2023. How accessibility affect other quality attributes of software? A case study of GitHub. Science of Computer Programming (2023), 103027.

[65]

Bo Zhou, Iulian Neamtiu, and Rajiv Gupta. 2015. A cross-platform analysis of bugs and bug-fixing in open source projects: Desktop vs. android vs. ios. In Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering. 1–10.

Digital Library

Cited By

Aljedaani WEler MKeeland LJalali HAl-Raddah KMkaouer M(2024)Accessible Gaming Through Better Captions: A Study on Captions Preferences and Inclusivity of Deaf and Hard-of-Hearing PlayersProceedings of the 21st International Web for All Conference10.1145/3677846.3677858(75-86)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3677846.3677858
Alghamdi AAljedaani WEler MLudi S(2024)Accessibility Guidelines and Standards: Analyzing Stack Overflow PostsProceedings of the 21st International Web for All Conference10.1145/3677846.3677857(118-122)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3677846.3677857
Alghamdi AAljedaani WJalali HLudi SEler M(2024)Understanding developer challenges and trends in web accessibility: a stack overflow analysisUniversal Access in the Information Society10.1007/s10209-024-01174-3Online publication date: 19-Nov-2024
https://doi.org/10.1007/s10209-024-01174-3

Index Terms

Empirical Investigation of Accessibility Bug Reports in Mobile Platforms: A Chromium Case Study
1. General and reference
  1. Cross-computing tools and techniques
2. Social and professional topics

Index terms have been assigned to the content through auto-classification.

Recommendations

On the identification of accessibility bug reports in open source systems
W4A '22: Proceedings of the 19th International Web for All Conference

Today, mobile devices provide support to disabled people to make their life easier due to their high accessibility and capability, e.g., finding accessible locations, picture and voice-based communication, customized user interfaces and vocabulary ...
Effective Bug Triage Based on Historical Bug-Fix Information
ISSRE '14: Proceedings of the 2014 IEEE 25th International Symposium on Software Reliability Engineering

For complex and popular software, project teams could receive a large number of bug reports. It is often tedious and costly to manually assign these bug reports to developers who have the expertise to fix the bugs. Many bug triage techniques have been ...
LDA Categorization of Security Bug Reports in Chromium Projects
ESSE '20: Proceedings of the 2020 European Symposium on Software Engineering

Security bug reports (SBR) depict potential security vulnerabilities in software systems. Bug tracking systems (BTS) usually contain huge numbers of bug reports including security-related ones. Malicious attackers could exploit these SBRs. Henceforth, ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems

May 2024

18961 pages

ISBN:9798400703300

DOI:10.1145/3613904

Editors:
Florian Floyd Mueller
Monash University
,
Penny Kyburz
The Australian National University
,
Julie R. Williamson
University of Glasgow
,
Corina Sas
Lancaster University
,
Max L. Wilson
University of Nottingham
,
Phoebe Toups Dugas
Monash University/New Mexico State University
,
Irina Shklovski
University of Copenhagen

Copyright © 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 May 2024

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CHI '24

Sponsor:

CHI '24: CHI Conference on Human Factors in Computing Systems

May 11 - 16, 2024

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
463
Total Downloads

Downloads (Last 12 months)463
Downloads (Last 6 weeks)55

Reflects downloads up to 30 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Aljedaani WEler MKeeland LJalali HAl-Raddah KMkaouer M(2024)Accessible Gaming Through Better Captions: A Study on Captions Preferences and Inclusivity of Deaf and Hard-of-Hearing PlayersProceedings of the 21st International Web for All Conference10.1145/3677846.3677858(75-86)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3677846.3677858
Alghamdi AAljedaani WEler MLudi S(2024)Accessibility Guidelines and Standards: Analyzing Stack Overflow PostsProceedings of the 21st International Web for All Conference10.1145/3677846.3677857(118-122)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3677846.3677857
Alghamdi AAljedaani WJalali HLudi SEler M(2024)Understanding developer challenges and trends in web accessibility: a stack overflow analysisUniversal Access in the Information Society10.1007/s10209-024-01174-3Online publication date: 19-Nov-2024
https://doi.org/10.1007/s10209-024-01174-3

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Table of Contents