research-article

Towards real-time and large-scale web accessbility

Authors:

Luís P. Carvalho,

Tiago Guerreiro,

Kyle MontagueAuthors Info & Claims

ASSETS '23: Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility

Article No.: 35, Pages 1 - 9

https://doi.org/10.1145/3597638.3608403

Published: 22 October 2023 Publication History

Abstract

We rely on large-scale web accessibility evaluations to obtain snapshots of Internet Health and understand trends and behaviours impacting overall web accessibility. Such evaluations are financially and time exhaustive, making the possibility of more real-time measurements of Internet Health infeasible. In this paper, we investigate the impacts of optimising the page selection processes of large-scale web accessibility evaluations. We set out to conduct an automated accessibility evaluation of 1500 websites using the ‘Home+’ sampling method (for each website, we evaluated the home page and all pages linked belonging to the same domain) as our baseline; then compared the agreement rates of web accessibility evaluations on further sub-sampled datasets. Accessibility data was successfully captured on 987 websites. Our findings demonstrate that a strong accessibility evaluation agreement between the baseline and the sub-sample datasets could be reached with a sub-sample of just 20% of the pages, significantly reducing the effort and resources required to conduct large-scale web accessibility evaluations.

References

[1]

2021. Internet users, UK - Office for National Statistics - Date accessed: 04/05/2023. https://www.ons.gov.uk/businessindustryandtrade/itandinternetindustry/bulletins/internetusers/2020

[2]

2022. Report on the 2020/21 Monitoring Period - acessibilidade.gov.pt - Date accessed: 04/05/2023. https://www.acessibilidade.gov.pt/publicacao/monitoring1/

[3]

2022. WebAIM: The WebAIM Million - The 2023 report on the accessibility of the top 1,000,000 home pages - Date accessed: 09/01/2023. https://webaim.org/projects/million/

[4]

2023. Github Features · Security - Date accessed: 30/07/2023. https://github.com/features/security

[5]

2023. NVD - Home - Date accessed: 30/07/2023. https://nvd.nist.gov/

[6]

2023. Vulnerability Database - OSV - Date accessed: 30/07/2023. https://osv.dev/list

[7]

2023. Web Accessibility | Shaping Europe’s digital future - Date accessed: 04/05/2023. https://digital-strategy.ec.europa.eu/en/policies/web-accessibility

[8]

Patricia Acosta-Vargas, Sergio Luján-Mora, Tania Acosta, and Luis Salvador-Ullauri. 2018. Toward a combined method for evaluation of web accessibility. Advances in Intelligent Systems and Computing 721 (2018), 602–613. https://doi.org/10.1007/978-3-319-73450-7_57/COVER

[9]

Patricia Acosta-Vargas, Sergio Lujan-Mora, and Luis Salvador-Ullauri. 2016. Evaluation of the web accessibility of higher-education websites. 2016 15th International Conference on Information Technology Based Higher Education and Training, ITHET 2016 (11 2016). https://doi.org/10.1109/ITHET.2016.7760703

[10]

Mahmoud Alfadel, Diego Elias Costa, Emad Shihab, and Mouafak Mkhallalati. 2021. On the use of dependabot security pull requests. Proceedings - 2021 IEEE/ACM 18th International Conference on Mining Software Repositories, MSR 2021 (5 2021), 254–265. https://doi.org/10.1109/MSR52588.2021.00037

[11]

Humberto Lidio Antonelli, Sandra Souza Rodrigues, Willian Massami Watanabe, and Renata Pontin De Mattos Fortes. 2018. A survey on accessibility awareness of brazilian web developers. ACM International Conference Proceeding Series (6 2018), 71–79. https://doi.org/10.1145/3218585.3218598

Digital Library

[12]

Barbara Rita Barricelli, Pierlauro Sciarelli, Stefano Valtolina, and Alessandro Rizzi. 2017. Web accessibility legislation in Italy: a survey 10 years after the Stanca Act. Universal Access in the Information Society 2017 17:1 17, 1 (2 2017), 211–222. https://doi.org/10.1007/S10209-017-0526-Z

Digital Library

[13]

Ramiro Gonçalves, José Martins, Jorge Pereira, Manuel Au-Yong Oliveira, and João José P. Ferreira. 2012. Enterprise Web Accessibility Levels Amongst the Forbes 250: Where Art Thou O Virtuous Leader?Journal of Business Ethics 2012 113:2 113, 2 (4 2012), 363–375. https://doi.org/10.1007/S10551-012-1309-3

[14]

Stephanie Hackett and Bambang Parmanto. 2009. Homepage not enough when evaluating web site accessibility. Internet Research 19, 1 (2009), 78–87. https://doi.org/10.1108/10662240910927830

[15]

Alexander Hambley, Yeliz Yesilada, Markel Vigo, and Simon Harper. 2023. OPTIMAL-EM: Optimised Population Sourcing for Web Accessibility Evaluation. In 20th International Web for All Conference. ACM, New York, NY, USA, 171–172. https://doi.org/10.1145/3587281.3587962

Digital Library

[16]

Kelly A. Harper and Jamie DeWaters. 2008. A Quest for website accessibility in higher education institutions. The Internet and Higher Education 11, 3-4 (1 2008), 160–164. https://doi.org/10.1016/J.IHEDUC.2008.06.007

[17]

Soon G. Hong, Silvana Trimi, Dong W. Kim, and Joon H. Hyun. 2015. A Delphi Study of Factors Hindering Web Accessibility for Persons with Disabilities. https://doi.org/10.1080/08874417.2015.11645784 55, 4 (6 2015), 28–34. https://doi.org/10.1080/08874417.2015.11645784

[18]

Yavuz Inal, Frode Guribye, Dorina Rajanen, Mikko Rajanen, and Mattias Rost. 2020. Perspectives and Practices of Digital Accessibility: A Survey of User Experience Professionals in Nordic Countries. ACM International Conference Proceeding Series 20 (10 2020). https://doi.org/10.1145/3419249.3420119

Digital Library

[19]

Abid Ismail and K. S. Kuppusamy. 2018. Accessibility of Indian universities’ homepages: An exploratory study. Journal of King Saud University - Computer and Information Sciences 30, 2 (4 2018), 268–278. https://doi.org/10.1016/J.JKSUCI.2016.06.006

[20]

Shaun K. Kane, Jessie A. Shulman, Timothy J. Shockley, and Richard E. Ladner. 2007. A web accessibility report card for top international university web sites. ACM International Conference Proceeding Series 225 (2007), 148–156. https://doi.org/10.1145/1243441.1243472

Digital Library

[21]

Donald E. Knuth. 1969. Seminumerical algorithms. The Art of Computer Programming. Vol. 2. Addison-Wesley, Reading, MA. 139–140 pages.

[22]

K. S. Kuppusamy and V. Balaji. 2021. Evaluating web accessibility of educational institutions websites using a variable magnitude approach. Universal Access in the Information Society 22, 1 (3 2021), 241–250. https://doi.org/10.1007/S10209-021-00812-4/FIGURES/5

Digital Library

[23]

Serhat Kurt. 2011. The accessibility of university web sites: The case of Turkish universities. Universal Access in the Information Society 10, 1 (3 2011), 101–110. https://doi.org/10.1007/S10209-010-0190-Z/TABLES/6

Digital Library

[24]

Jonathan Lazar, Alfreda Dudley-Sponaugle, and Kisha Dawn Greenidge. 2004. Improving web accessibility: a study of webmaster perceptions. Computers in Human Behavior 20, 2 (3 2004), 269–288. https://doi.org/10.1016/J.CHB.2003.10.018

[25]

Humberto Lidio Antonelli, Sandra Souza Rodrigues, Willian Massami Watanabe, and Renata Pontin de Mattos Fortes. [n. d.]. A survey on accessibility awareness of Brazilian web developers ACM Reference format. Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion ([n. d.]). https://doi.org/10.1145/3218585

Digital Library

[26]

Rui Lopes, Daniel Gomes, and Luís Carriço. 2010. Web not for all: A large scale study of web accessibility. W4A 2010 - International Cross Disciplinary Conference on Web Accessibility Raleigh 2010 (2010). https://doi.org/10.1145/1805986.1806001

Digital Library

[27]

VellemanEric M., NahuisInge, and GeestThea. 2017. Factors explaining adoption and implementation processes for web accessibility standards within eGovernment systems and organizations. Universal Access in the Information Society 16, 1 (3 2017), 173–190. https://doi.org/10.1007/S10209-015-0449-5

Digital Library

[28]

Carlos Máñez-Carvajal, Jose Francisco Cervera-Mérida, and Rocío Fernández-Piqueras. 2019. Web accessibility evaluation of top-ranking university Web sites in Spain, Chile and Mexico. Universal Access in the Information Society 20, 1 (3 2019), 179–184. https://doi.org/10.1007/S10209-019-00702-W

Digital Library

[29]

J. Nielsen. 2000. Designing web usability. (2000). https://doi.org/10.3/JQUERY-UI.JS

[30]

Marian PADURE and Costin PRIBEANU. 2020. Comparing Six Free Accessibility Evaluation Tools. Informatica Economica 24, 1/2020 (3 2020), 15–25. https://doi.org/10.24818/ISSN14531305/24.1.2020.02

[31]

John T. Richards, Kyle Montague, and Vicki L. Hanson. 2012. Web accessibility as a side effect. ASSETS’12 - Proceedings of the 14th International ACM SIGACCESS Conference on Computers and Accessibility (2012), 79–86. https://doi.org/10.1145/2384916.2384931

Digital Library

[32]

Anne Spencer Ross, Xiaoyi Zhang, James Fogarty, and Jacob O. Wobbrock. 2020. An Epidemiology-inspired Large-scale Analysis of Android App Accessibility. ACM Transactions on Accessible Computing (TACCESS) 13, 1 (4 2020). https://doi.org/10.1145/3348797

Digital Library

[33]

Markel Vigo, Justin Brown, and Vivienne Conway. 2013. Benchmarking web accessibility evaluation tools: Measuring the harm of sole reliance on automated tests. W4A 2013 - International Cross-Disciplinary Conference on Web Accessibility (2013). https://doi.org/10.1145/2461121.2461124

Digital Library

[34]

Yeliz Yesilada, Giorgio Brajnik, Markel Vigo, and Simon Harper. 2013. Exploring perceptions of web accessibility: a survey approach. http://dx.doi.org/10.1080/0144929X.2013.848238 34, 2 (2 2013), 119–134. https://doi.org/10.1080/0144929X.2013.848238

Digital Library

Cited By

Iannuzzi NManca MPaternò FSantoro C(2025)Combined accessibility validation and monitoring of web sites and PDF documentsUniversal Access in the Information Society10.1007/s10209-025-01194-7Online publication date: 15-Feb-2025
https://doi.org/10.1007/s10209-025-01194-7
Regimbal JBlum JKuo CCooperstock J(2024)IMAGE: An Open-Source, Extensible Framework for Deploying Accessible Audio and Haptic Renderings of Web GraphicsACM Transactions on Accessible Computing10.1145/366522317:2(1-17)Online publication date: 29-Jul-2024
https://dl.acm.org/doi/10.1145/3665223

Index Terms

Towards real-time and large-scale web accessbility
1. Human-centered computing
  1. Accessibility
    1. Accessibility design and evaluation methods

Recommendations

Towards a unified definition of web accessibility
W4A '15: Proceedings of the 12th International Web for All Conference

To better understand what researchers and practitioners consider to be the key components of the definition of web accessibility and to propose a unified definition of web accessibility, we conducted an analysis of 50 definitions of web accessibility. ...
Large scale web accessibility evaluation: a European perspective
UAHCI'07: Proceedings of the 4th international conference on Universal access in human-computer interaction: applications and services

The 2002 eEurope Action Plan advocated adoption of the WAI Content Guidelines in public EU web sites. Many of the European countries have performed web accessibility evaluations. Most of the evaluations are based on the Web Content Accessibility ...
Guidelines for an accessible web automation interface
ASSETS '11: The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility

In recent years, the Web has become an ever more sophisticated and irreplaceable tool in our daily lives. While the visual Web has been advancing at a rapid pace, assistive technology has not been able to keep up, increasingly putting visually impaired ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ASSETS '23: Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility

October 2023

1163 pages

ISBN:9798400702204

DOI:10.1145/3597638

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGACCESS: ACM Special Interest Group on Accessible Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ASSETS '23

Sponsor:

SIGACCESS

ASSETS '23: The 25th International ACM SIGACCESS Conference on Computers and Accessibility

October 22 - 25, 2023

NY, New York, USA

Acceptance Rates

ASSETS '23 Paper Acceptance Rate 55 of 182 submissions, 30%;

Overall Acceptance Rate 436 of 1,556 submissions, 28%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
255
Total Downloads

Downloads (Last 12 months)135
Downloads (Last 6 weeks)35

Reflects downloads up to 09 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Iannuzzi NManca MPaternò FSantoro C(2025)Combined accessibility validation and monitoring of web sites and PDF documentsUniversal Access in the Information Society10.1007/s10209-025-01194-7Online publication date: 15-Feb-2025
https://doi.org/10.1007/s10209-025-01194-7
Regimbal JBlum JKuo CCooperstock J(2024)IMAGE: An Open-Source, Extensible Framework for Deploying Accessible Audio and Haptic Renderings of Web GraphicsACM Transactions on Accessible Computing10.1145/366522317:2(1-17)Online publication date: 29-Jul-2024
https://dl.acm.org/doi/10.1145/3665223

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten