Keywords

1 Introduction

Access to digital resources is important for all, including people with disabilities. Digital library interfaces are useful mechanisms to enable people find and consume digital resources, which is important for their education or for general knowledge. One way to ensure their accessibility is to build universally designed solutions that will be usable by all. A typical challenge while building such interfaces is access to real participants for evaluation purposes. To overcome this limitation, researchers make use of heuristics in order to measure the accessibility and usability of an interface without participants. Specific heuristics have been developed for various domains, including for digital library interfaces [1, 2], which help researchers evaluate the level of usability and how easy and successfully users would find desired resources using the web interface. These heuristics, however, seem to focus on the average user, excluding users with disabilities.

Designing for people with disabilities requires a level of empathy and consideration for their condition. Instead of treating them as ‘test’ subjects, researchers and designers should develop relationship with participants in order to elicit and understand their needs. To this end, researchers have conceived the user-sensitive inclusive design concept, which suggests that designers should guide their design by developing empathy with the users [3]. Empathy is also to consider that users with disability have a difficulty to participate in studies, for which some studies suggest leveraging the Internet of Things to gather requirements from such people [4]. The formulation of heuristics would help to evaluate and plan digital library interfaces by reducing the need of “bothering” users to understand their requirements and get initial interface evaluation feedback.

Web accessibility heuristics have been developed and used to address web accessibility and diversity of users [5,6,7]. Though they are useful to evaluate the general accessibility of web solutions, they are not suitable for digital library interfaces. Library interfaces possess characteristics that are not encompassed by the general web accessibility heuristics. They could help to identify superficial elements of interface design, but would miss, for example, evaluating whether the interface has an accessible way to help people develop effective way of locating resources [8]. Digital libraries can be stand-alone or a federated collection of resources collected and organized in different silos, which increases the importance of designing accessible and effective interfaces.

In this paper, we address this gap by exploring characteristics of existing heuristics used for digital libraries and those for Web accessibility to devise new set of heuristics. The outcome will be a set of heuristics focused for the purposes of evaluating the accessibility of digital library interfaces. This will combine the best of both types of heuristics to build a comprehensive set of heuristics to achieve better results when evaluating the accessibility of digital library interfaces.

2 Related Work

Developing new type of heuristics is usually done using two methods: empirical-based and research-based. The empirical-based approach is conducted by developing heuristics based on actual data collected and analyzed, while the research-based approach is conducted by evaluating existing heuristics developed for other similar domains. Using the empirical-based approach, Nielsen has developed the first set of heuristics [9]. He started by categorizing hundreds of problems devised from many usability testing studies and grouping those into ten rules of heuristics. Many other heuristics are developed taking the Nielsen heuristics as a basis and modifying as needed for the new domain, such as heuristics developed to evaluate gaming of education [10].

Typically, after the new heuristics are developed, they are empirically tested and compared whether they perform better that the original heuristics from which they were derived [11]. Similar approach was adopted by Tsui et al. [12] to devise heuristics based on Nielsen for the evaluation of assistive robotics. They compared those and discovered that the new heuristics were three times more effective as they found 33 errors compared to 13 found using Nielsen’s heuristics.

Aitta et al. [1] developed new library heuristics, which addressed the library perspective in better details than Nielsen’s heuristics. Fifteen public library websites were evaluated using these heuristics. They were similar to those developed by Chisnell et al. [5], which focused on developing interfaces for the elderly users. Moreover, Chisnell et al. argued that most of the heuristics do not take into account people with different abilities. For example, Nielsen’s heuristics assume that all users have perfect physical and cognitive abilities and are able to conduct the tasks on a given interface. The reality is that people with diverse disabilities, including those with dyslexia and visual impairment, use the Web and library interfaces [13,14,15,16,17,18]. To ensure their usability and accessibility by such users, various adaptations are typically conducted, on a content and interface level [19, 20].

Considering this, Morrell et al. [21] developed heuristics addressing the use of interfaces by adults who also have certain level of disability. However, this was not suitable for Chisnell et al. [5], who were interested in evaluating the level of usability of websites used by the elderly, but not specifically developed for older adults. Hence, they developed a new set of heuristics to evaluate websites that the elderly use frequently. They evaluated fifty websites using these newly developed heuristics [5].

Driven by particular needs and using similar methods, several studies report developing specific heuristics. For instance, Drury [22] have derived heuristics based on theories and metaphors for the evaluation of collaborative system behavior interfaces, which were validated through an experiment. Bolchini et al. [23] have proposed an initial set of heuristics to evaluate the semiotics of web interfaces, particularly concentrating on information-rich websites. The authors proposed these semiotics heuristics to be used in combination with other existing evaluation methods. Travis and Tay [24] had concerns about screen size, which inspired them to develop heuristics to evaluate digital library interfaces used specifically on mobile devices.

Considering all these studies, our aim is to develop heuristics for evaluating the accessibility level of digital library interfaces. We adopt a mixed approach, (1) research-based approach by analyzing existing heuristics used to evaluate the accessibility of web interfaces and those used to evaluate the usability of digital interfaces, and (2) empirical-based approach by analyzing the data collected and reported in detail on a prior study [39].

3 Comparison of Existing Web Accessibility and Digital Library Heuristics

In order to develop heuristics for the purposes of evaluating the accessibility of digital library interfaces, we studied relevant existing heuristics used in two domains: Web accessibility and digital libraries.

3.1 Web Accessibility Heuristics

Considering that digital libraries are web-based interfaces, it is a viable approach to initially investigate the heuristics that are used to evaluate the accessibility of Web sites. Most studies that conducted accessibility inspections on Web sites used Web Content Accessibility Guidelines (WCAG 2.0) [25,26,27]. Many other studies, however, consider WCAG to be very long and with too many criteria to check while conducting an evaluation [28]. Granting that WCAG contains as much as 65 guidelines, according to Moreno et al. [29], those are still not fail-proof, because although pages might pass the WCAG test, they may remain to be inaccessible.

Taking these issues into consideration, a more concise accessibility heuristics have been developed in order to complement or replace WCAG guidelines. For instance, the IBM accessibility heuristics consists of twelve items, which are easy to keep in mind while evaluators inspect accessibility of web pages [30]. Moreover, as Mankoff et al. [31] recommended, website developers should be able to use brief accessibility evaluation techniques to get quick results and enable a development process that is more agile. We support such approach and in this paper, we rely mostly on existing concise heuristics.

Our state-of-the-art research on the existing Web accessibility heuristics revealed two sources, namely the IBM Web Accessibility Heuristics [6] and the guidelines developed by Zaphiris et al. [32], which are most relevant to our goal. Hence, these will be used as a basis for the generation of new heuristics, which could help to evaluate the accessibility of digital library interfaces.

3.2 Digital Library Heuristics

In order to develop the new heuristics, we also investigated existing studies that used heuristics to evaluate the usability, but not the accessibility, of digital library interfaces. Most of those studies used existing Nielsen heuristics [33,34,35,36]. There are some studies, however, that used modified version of Nielsen’s heuristics [1, 10, 11]. However, according to Chisnell et al. [5], most heuristics and guidelines are too broad or too general, which might require more expertise from the evaluators [1].

With this in mind, Aitta et al. [1] proposed a new heuristic item that actually takes into consideration the interface accessibility by stating: the interface should consider special groups, such as children, the elderly and people with disabilities. Though this indicates a new good trend, it represents a very general accessibility requirement and does not concretely help evaluators. The important note is that these proposed heuristics present a library viewpoint compared to Nielsen’s heuristics, which are also similar to heuristics developed by Chisnell et al. [5].

Other researchers, such as Laender et al. [37] have claimed that the heuristics should not only evaluate interface level accessibility, but also provide a way to evaluate accessibility of the resources found and listed by the interface. For example, the interface should clearly indicate when a resource found is behind a paywall, to avoid clicking the link if users choose not to complete the transaction.

The review of existing studies that we conducted in the field of digital libraries revealed two important sources: Aitta et al. [1] and Joo and Yeon Lee [38]. We will further investigate these in order to devise heuristics for the evaluation of digital libraries accessibility.

4 Proposing Novel Heuristics for Digital Library Interfaces

Our investigation into existing heuristics discovered four sources relevant to our goal, specifically:

  • IBM Web accessibility heuristics [6] with 12 items;

  • Guidelines when designing for the elderly by Zaphiris et al. [32] with 37 items;

  • Heuristics for library services by Aitta et al. [1] with 9 items; and

  • Heuristics for measuring the usability of academic libraries by Joo and Yeon Lee [38] with 14 items.

This total number of 72 items was independently rated by two domain experts in a scale from zero (not relevant) to three (very relevant). Only items that scored highest on average (with value three), were retained in the filtering process. This process generated an initial list of heuristics with 16 items. Because some items were repetitive or very similar, for example provide text alternative for all non-text content and images should have alt tags, the list remained with twelve items drawn from three sources as shown in the box labeled with ‘existing heuristics’ in Fig. 1.

Fig. 1.
figure 1

The process of devising digital library accessibility heuristics from two sources: existing heuristics and a prior study conducted on the accessibility of digital libraries.

Full size image

To these twelve heuristics, we added four additional items based on a prior study we conducted with low vision and users with dyslexia [39]. The qualitative study highlights accessibility issues discovered when users were searching for resources on OriaFootnote 1, which is a resource discovery tool used by Norwegian academic and research libraries. Ultimately, the list of heuristics proposed was of 16 items. Table 1 shows these heuristics along with a detailed description as well as the source from where they were generated.

Table 1. List of proposed heuristics to evaluate the accessibility of digital library interfaces.
Full size table

5 Conclusion and Future Works

Heuristics are useful for evaluating interfaces without participants. This requires that evaluators possess a great understanding about user needs, abilities and how users interact with the interfaces being evaluated. Considering this, heuristics should be detailed enough to help evaluators identify accessibility and usability issues with the interface. Hence, heuristics specific to various domains have been developed and proved more suitable and effective than generic heuristics.

Our extensive study of existing literature revealed no heuristics specifically tailored to evaluate the accessibility of digital library interfaces. This prompted us to propose an initial set of new heuristics derived from two main sources: existing heuristics and prior empirical research. Existing heuristics were derived from two domains: Web accessibility and digital library heuristics. Two domain experts independently examined elements of those heuristics and rated them for their relevance to evaluating digital library interfaces. Twelve heuristics were derived from these two domains. Additionally, four new heuristics were added based on an empirical observation we conducted on another study.

The sixteen new heuristics we devised in this study specifically aim to help evaluators uncover accessibility issues with digital library interfaces. It remains in future studies to evaluate these heuristics when testing library interfaces. Moreover, comparing these newly proposed heuristics with existing heuristics is a part of future work we intend to conduct.