research-article

Drop-Down Menu Widget Identification Using HTML Structure Changes Classification

Authors:

Humberto Lidio Antonelli,

Rodrigo Augusto Igawa,

Renata Pontin De Mattos Fortes,

Eduardo Henrique Rizo,

Willian Massami WatanabeAuthors Info & Claims

ACM Transactions on Accessible Computing (TACCESS), Volume 11, Issue 2

Article No.: 10, Pages 1 - 23

https://doi.org/10.1145/3178854

Published: 08 June 2018 Publication History

Abstract

Widgets have been deployed in rich internet applications for more than 10 years. However, many of the widgets currently available on the web do not implement current accessibility design solutions standardized in ARIA (Accessible Rich Internet Applications) specification, hence are not accessible to disabled users. This article sets out an approach for automatically identifying widgets on the basis of machine-learning algorithms and the classification of mutation records; it is an HTML5 technology that logs all changes that occur in the structure of a web application. Automatic widget identification is an essential component for the elaboration of automatic ARIA evaluation and adaptation strategies. Thus, the aim of this article is to take steps toward easing the software-engineering process of ARIA widgets. The proposed approach focuses on the identification of drop-down menu widgets. An experiment with real-world web applications was conducted and the results showed evidence that this approach is capable of identifying these widgets and can outperform previous state-of-the-art techniques based on an F-measure analysis conducted during the experiment.

References

[1]

Domenico Amalfitano, Anna Rita Fasolino, Armando Polcaro, and Porfirio Tramontana. 2014. The DynaRIA tool for the comprehension of Ajax web applications by dynamic analysis. Innov. Syst. Softw. Eng. 10, 1 (01 Mar 2014), 41--57.

Digital Library

[2]

Tim Berners-Lee. 1990. Information Management: A Proposal. W3C Archive. Retrieved May 15, 2018 from http://www.nic.funet.fi/index/FUNET/history/internet/w3c/proposal.html.

[3]

Leo Breiman. 2001. Random forests. Mach. Learn. 45, 1 (Oct. 2001), 5--32.

Digital Library

[4]

Andy Brown and Simon Harper. 2013. Dynamic injection of WAI-ARIA into web content. In Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility (W4A’13). ACM, New York, NY, Article 14, 4 pages.

Digital Library

[5]

Andy Brown, Caroline Jay, Alex Q. Chen, and Simon Harper. 2012. The uptake of web 2.0 technologies, and its impact on visually disabled users. Univers. Access Inform. Soc. 11, 2 (01 Jun 2012), 185--199.

Digital Library

[6]

Lucas Pedroso Carvalho, Lucas Pereira Ferreira, and André Pimenta Freire. 2016. Accessibility evaluation of rich internet applications interface components for mobile screen readers. In Proceedings of the 31st Annual ACM Symposium on Applied Computing (SAC’16). ACM, New York, NY, 181--186.

Digital Library

[7]

Sven Casteleyn, Irene Garrigós, and Jose-Norberto Mazón. 2014. Ten years of rich internet applications: A systematic mapping study, and beyond. ACM Trans. Web 8, 3, Article 18 (July 2014), 46 pages.

Digital Library

[8]

Alex Q. Chen, Simon Harper, Darren Lunn, and Andrew Brown. 2013. Widget identification: A high-level approach to accessibility. World Wide Web 16, 1 (2013), 73--89.

[9]

Iyad Abu Doush, Faisal Alkhateeb, Eslam Al Maghayreh, and Mohammed Azmi Al-Betar. 2013. The design of RIA accessibility evaluation tool. Adv. Eng. Softw. 57, 0 (2013), 1--7.

Digital Library

[10]

Carlos Duarte, Inês Matos, João Vicente, Ana Salvado, Carlos M. Duarte, and Luís Carriço. 2016. Development technologies impact in web accessibility. In Proceedings of the 13th Web for All Conference (W4A’16). ACM, New York, NY, Article 6, 4 pages.

Digital Library

[11]

Eyad Elyan and Mohamed Medhat Gaber. 2017. A genetic algorithm approach to optimising random forests applied to class engineered data. Inform. Sci. 384 (2017), 220--234.

[12]

Nádia Fernandes, Ana Sofia Batista, Daniel Costa, Carlos Duarte, and Luís Carriço. 2013. Three web accessibility evaluation perspectives for RIA. In Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility (W4A’13). ACM, New York, NY, Article 12, 9 pages.

Digital Library

[13]

Daniela Fogli, Loredana Parasiliti Provenza, and Cristian Bernareggi. 2014. A universal design resource for rich internet applications based on design patterns. Univers. Access Inform. Soc. 13, 2 (2014), 205--226.

Digital Library

[14]

Piero Fraternali, Sara Comai, Alessandro Bozzon, and Giovanni Toffetti Carughi. 2010. Engineering rich internet applications with a model-driven approach. ACM Trans. Web 4, 2 (2010), 1--47.

Digital Library

[15]

André Pimenta Freire, Cibele Maria Russo, and Renata Pontin de Mattos Fortes. 2008. The perception of accessibility in web development by academy, industry and government: A survey of the Brazilian scenario. New Rev. Hypermedia Multimedia 14, 2 (December 2008), 149--175.

Digital Library

[16]

Rafael José Geraldo and Renata Pontin Fortes. 2013. Dificuldades de usuários cegos na interação com a web: uma análise sobre as pesquisas. Rev. Sist. Comput. 3, 2 (Jul./Dez. 2013), 146--160.

[17]

Becky Gibson. 2007. Enabling an accessible web 2.0. In Proceedings of the 2007 International Cross-disciplinary Conference on Web Accessibility (W4A’07). ACM, New York, NY, 1--6.

Digital Library

[18]

Stéphanie Giraud, Teresa Colombi, Aurore Russo, and Pierre Thérouanne. 2011. Accessibility of rich internet applications for blind people: A study to identify the main problems and solutions. In Proceedings of the 9th ACM SIGCHI Italian Chapter International Conference on Computer-Human Interaction: Facing Complexity (CHItaly’11). ACM, New York, NY, 163--166.

Digital Library

[19]

Vicki L. Hanson and John T. Richards. 2013. Progress on website accessibility? ACM Trans. Web 7, 1, Article 2 (March 2013), 30 pages.

Digital Library

[20]

Naomi Harrington, Dean Pucsek, and Yvonne Coady. 2011. Approaching web accessibility through the browser: Automatically applying aria attributes. In Proceedings of the 1st ACM SIGPLAN International Workshop on Programming Language and Systems Technologies for Internet Clients (PLASTIC’11). ACM, New York, NY, 29--32.

Digital Library

[21]

Trevor Hastie, Robert Tibshirani, and Jerome Friedman. 2001. The Elements of Statistical Learning. Springer, New York, NY.

[22]

Rodrigo Augusto Igawa, Sylvio Barbon Jr, Kátia Cristina Silva Paulo, Guilherme Sakaji Kido, Rodrigo Capobianco Guido, Mario Lemes Proença Júnior, and Ivan Nunes da Silva. 2016. Account classification in online social networks with LBCA and wavelets. Inform. Sci. 332 (2016), 72--83.

Digital Library

[23]

Jonas Kluge, Frank Kargl, and Michael Weber. 2007. The effects of the AJAX technology on web application usability. In Proceedings of the 3rd International Conference on Web Information Systems and Technologies (WebIST’07). WebIST, Barcelona, Spain, 289--294. http://www.kargl.net/pubs/docs/2007-03-webist.pdf.

[24]

Marino Linaje, Adolfo Lozano-Tello, Miguel A. Perez-Toledano, Juan Carlos Preciado, Roberto Rodriguez-Echeverria, and Fernando Sanchez-Figueroa. 2011. Providing RIA user interfaces with accessibility properties. J. Symbol. Comput. 46, 2 (2011), 207--217. Automated Specification and Verification of Web Systems.

Digital Library

[25]

Marino Linaje, Juan C. Preciado, and Fernando Sanchez-Figueroa. 2009. Domain-specific model for designing rich internet application user interfaces. In Computer-Aided Design of User Interfaces VI. Springer, London, UK, 295--306.

[26]

Yang Liu, Jian-Wu Bi, and Zhi-Ping Fan. 2017. A method for multi-class sentiment classification based on an improved one-vs-one (OVO) strategy and the support vector machine (SVM) algorithm. Inform. Sci. 394--395 (2017), 38--52.

Digital Library

[27]

Valentyn Melnyk, Vikas Ashok, Valentyn Melnyk, Yury Puzis, Yevgen Borodin, Andrii Soviak, and I. V. Ramakrishnan. 2015. Look Ma, No ARIA: Generic accessible interfaces for web widgets. In Proceedings of the 12th Web for All Conference (W4A’15). ACM, New York, NY, Article 21, 4 pages.

Digital Library

[28]

Valentyn Melnyk, Vikas Ashok, Yury Puzis, Andrii Soviak, Yevgen Borodin, and I. V. Ramakrishnan. 2014. Widget Classification with Applications to Web Accessibility. Springer International Publishing, Cham, 341--358.

[29]

Ethan V. Munson and Maria da Graça Pimentel. 2008. Specialized documents. In Web Accessibility (Human-Computer Interaction Series), Vol. 4. Springer, London, UK, 274--285.

[30]

D. M. W. Powers. 2011. Evaluation: From precision, recall and f-measure to roc., informedness, markedness 8 correlation. J. Mach. Learn. Technol. 2, 1 (2011), 37--63.

[31]

Yury Puzis, Yevgen Borodin, Andrii Soviak, Valentyn Melnyk, and I. V. Ramakrishnan. 2015. Affordable web accessibility: A case for cheaper ARIA. In Proceedings of the 12th Web for All Conference (W4A’15). ACM, New York, Article 32, 4 pages.

Digital Library

[32]

J. R. Quinlan. 1986. Induction of decision trees. Mach. Learn. 1, 1 (March 1986), 81--106.

Digital Library

[33]

Lior Rokach and Oded Maimon. 2005. Decision Trees. Springer, Boston, MA, Chapter 9, 165--192.

[34]

S. Sandhya and K. A. S. Devi. 2011. Accessibility evaluation of websites using screen reader. In Proceedings of the 2011 7th International Conference on Next Generation Web Services Practices. IEEE, Salamanca, Spain, 338--341.

[35]

Takaaki Tateishi, Hisashi Miyashita, Tabuchi Naoshi, Shin Saito, and Kouichi Ono. 2007. DHTML accessibility checking based on static javascript analysis. In Universal Access in Human-Computer Interaction. Applications and Services, Constantine Stephanidis (Ed.). Lecture Notes in Computer Science, Vol. 4556. Springer, Berlin, 167--176.

Digital Library

[36]

Peter Thiessen and Stephen Hockema. 2010. WAI-ARIA live regions: eBuddy IM as a case example. In Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A’10). ACM, New York, NY, Article 33, 9 pages.

Digital Library

[37]

Carlos A. Velasco, Dimitar Denev, Dirk Stegemann, and Yehya Mohamad. 2008. A web compliance engineering framework to support the development of accessible rich internet applications. In W4A’08: Proceedings of the 2008 International Cross-disciplinary Conference on Web Accessibility (W4A’08). ACM, New York, NY, 45--49.

Digital Library

[38]

W3C. 2012. Packaged Web Apps (Widgets)—Packaging and XML Configuration (2nd ed.). W3C Recommendation. (November 2012). Retrieved May 15, 2018 from http://www.w3.org/TR/widgets.

[39]

W3C. 2014a. Accessible Rich Internet Applications (WAI-ARIA) 1.0. W3C Recommendation. (March 2014). Retrieved May 15, 2018 from http://www.w3.org/TR/wai-aria.

[40]

W3C. 2014b. HTML5—A vocabulary and associated APIs from HTML and XHTML. W3C Recommendation. (October 2014). Retrieved May 15, 2018 from http://www.w3.org/TR/html5.

[41]

W3C. 2016. WAI-ARIA Authoring Practices 1.1. W3C Working Draft. (March 2016). Retrieved May 15, 2018 from https://www.w3.org/TR/wai-aria-practices-1.1.

[42]

Willian Massami Watanabe, Ana Luiza Dias, and Renata Pontin De Mattos Fortes. 2015. Fona: Quantitative metric to measure focus navigation on rich internet applications. ACM Trans. Web 9, 4, Article 20 (Sept. 2015), 28 pages.

Digital Library

[43]

Willian Massami Watanabe, Renata P. Fortes, and Ana Luiza Dias. 2017. Acceptance tests for validating ARIA requirements in widgets. Univers. Access Inf. Soc. 16, 1 (March 2017), 3--27.

Digital Library

[44]

Willian Massami Watanabe and Renata Pontin de Mattos Fortes. 2016. Automatic identification of drop-down menu widgets using mutation observers and visibility changes. In Proceedings of the 31st Annual ACM Symposium on Applied Computing (SAC’16). ACM, New York, NY, 766--771.

Digital Library

[45]

Willian Massami Watanabe, Renata P. M. Fortes, and Ana Luiza Dias. 2012. Using acceptance tests to validate accessibility requirements in RIA. In Proceedings of the International Cross-Disciplinary Conference on Web Accessibility (W4A’12). ACM, New York, NY, Article 15, 10 pages.

Digital Library

[46]

Willian Massami Watanabe, Rafael José Geraldo, and Renata Pontin de Mattos Fortes. 2015. Keyboard navigation mechanisms in widgets: An investigation on ARIA’s implementations. J. Web Eng. 14, 1 8 2 (2015), 41--62.

Digital Library

[47]

Willian Massami Watanabe, Rafael José Geraldo, and Renata Pontin de Mattos Fortes. 2014. Keyboard navigation mechanisms in tab widgets: An investigation on ARIA’s conformance. In Proceedings of the 29th Annual ACM Symposium on Applied Computing (SAC’14). ACM, New York, NY, 721--726.

Digital Library

[48]

Shen Yin and Jiapeng Yin. 2016. Tuning kernel parameters for SVM based on expected square distance ratio. Inform. Sci. 370--371 (2016), 92--102.

Digital Library

Cited By

Silva JCardoso PDe Bettio RTavares DSilva CWatanabe WFreire A(2024)In-Page Navigation Aids for Screen-Reader Users with Automatic Topicalisation and LabellingACM Transactions on Accessible Computing10.1145/364922317:2(1-45)Online publication date: 22-Jul-2024
https://dl.acm.org/doi/10.1145/3649223
Ara JSik-Lanyi CKelemen A(2024)Accessibility engineering in web evaluation process: a systematic literature reviewUniversal Access in the Information Society10.1007/s10209-023-00967-223:2(653-686)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s10209-023-00967-2
Watanabe Wde Lemos GNascimento R(2024)Accessibility landmarks identification in web applications based on DOM elements classificationUniversal Access in the Information Society10.1007/s10209-022-00959-823:2(765-777)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s10209-022-00959-8
Show More Cited By

Index Terms

Drop-Down Menu Widget Identification Using HTML Structure Changes Classification
1. Human-centered computing
  1. Accessibility
  2. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Hypertext / hypermedia
      2. Web-based interaction
2. Information systems
  1. World Wide Web
    1. Web interfaces

Recommendations

Code Complexity Impact of Widgets Accessibility Implementation in JavaScript Open-Source Libraries
WebMedia '21: Proceedings of the Brazilian Symposium on Multimedia and the Web

Widgets are part of many web applications and implement distinct interaction mechanisms, in reusable user interface components. However, implementing accessible widgets is a challenge, associated to different factors highlighted in the state-of-the-art, ...
Automatic Identification of Widgets and their Subcomponents Based on a Classification Pipeline for DOM Mutation Records
W4A '19: Proceedings of the 16th International Web for All Conference

Since Web 2.0, web developers have increased the use of more sophisticated interaction mechanisms and visual effects, called widgets, to design the Rich Internet Applications' (RIA) user interface. However, many of the widgets currently available on ...
Keyboard navigation mechanisms in widgets: an investigation on aria's implementations

This study presents an investigation on how keyboard accessibility has been delivered in RIA - Rich Internet Applications. We conducted an evaluation on 32 websites which contained Tab Widgets and 74 websites which contained Menu Widgets, from the 150 ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Accessible Computing

ACM Transactions on Accessible Computing Volume 11, Issue 2

Paper from ASSETS 2016 and Regular Papers

June 2018

112 pages

ISSN:1936-7228

EISSN:1936-7236

DOI:10.1145/3229065

Editors:
Matt Huenerfauth
Rochester Institute of Technology, USA
,
Kathleen F. McCoy
University of Delaware, USA

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 June 2018

Accepted: 01 January 2018

Revised: 01 January 2018

Received: 01 April 2017

Published in TACCESS Volume 11, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

FAPESP, CNPq, and CAPES

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
351
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)4

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Silva JCardoso PDe Bettio RTavares DSilva CWatanabe WFreire A(2024)In-Page Navigation Aids for Screen-Reader Users with Automatic Topicalisation and LabellingACM Transactions on Accessible Computing10.1145/364922317:2(1-45)Online publication date: 22-Jul-2024
https://dl.acm.org/doi/10.1145/3649223
Ara JSik-Lanyi CKelemen A(2024)Accessibility engineering in web evaluation process: a systematic literature reviewUniversal Access in the Information Society10.1007/s10209-023-00967-223:2(653-686)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s10209-023-00967-2
Watanabe Wde Lemos GNascimento R(2024)Accessibility landmarks identification in web applications based on DOM elements classificationUniversal Access in the Information Society10.1007/s10209-022-00959-823:2(765-777)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s10209-022-00959-8
Ferner BGåsøy ANicolaysen MSandnes F(2023)A Comparison of Form Navigation with Tabbing and PointingUniversal Access in Human-Computer Interaction10.1007/978-3-031-35681-0_20(311-318)Online publication date: 9-Jul-2023
https://doi.org/10.1007/978-3-031-35681-0_20
Chiou PAlotaibi AHalfond WSpinellis DGousios GChechik MDi Penta M(2021)Detecting and localizing keyboard accessibility failures in web applicationsProceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3468264.3468581(855-867)Online publication date: 20-Aug-2021
https://dl.acm.org/doi/10.1145/3468264.3468581
Sensiate LLidio Antonelli HMassami Watanabe WPontin de Mattos Fortes R(2020)A Mechanism for Identifying Dynamic Components in Rich Internet ApplicationsProceedings of the 38th ACM International Conference on Design of Communication10.1145/3380851.3416785(1-8)Online publication date: 3-Oct-2020
https://dl.acm.org/doi/10.1145/3380851.3416785
Antonelli HSensiate LWatanabe Wde Mattos Fortes RStaggers JRichards DAmidon TPflugfelder E(2019)Challenges of automatically evaluating rich internet applications accessibilityProceedings of the 37th ACM International Conference on the Design of Communication10.1145/3328020.3353950(1-6)Online publication date: 4-Oct-2019
https://dl.acm.org/doi/10.1145/3328020.3353950
Rizo Ede Mattos Fortes RAntonelli HWatanabe W(2019)Automatic Identification of Widgets and their Subcomponents Based on a Classification Pipeline for DOM Mutation RecordsProceedings of the 16th International Web for All Conference10.1145/3315002.3317555(1-10)Online publication date: 13-May-2019
https://dl.acm.org/doi/10.1145/3315002.3317555
de Mattos Fortes RAntonelli HWatanabe W(2019)Dynamic Web ContentWeb Accessibility10.1007/978-1-4471-7440-0_21(373-395)Online publication date: 4-Jun-2019
https://doi.org/10.1007/978-1-4471-7440-0_21

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents