Open AccessReview

A Systematic Literature Review of the Design Approach and Usability Evaluation of the Pain Management Mobile Applications

Umm e Mariya Shah

and

Thiam Kian Chiew

Faculty of Computer Science and Information Technology, University of Malaya, Lembah Pantai 50603, Kuala Lumpur, Malaysia

Author to whom correspondence should be addressed.

Symmetry 2019, 11(3), 400; https://doi.org/10.3390/sym11030400

Submission received: 20 December 2018 / Revised: 9 January 2019 / Accepted: 18 January 2019 / Published: 19 March 2019

(This article belongs to the Special Issue Information Technology and Its Applications 2021)

Download

Browse Figures

Figure 1
Study selection process: Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) flow diagram. From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097. doi:10.1371/journal.pmed1000097. For more information, visit <a href="http://www.prisma-statement.org" target="_blank">www.prisma-statement.org</a>. "> Figure 2
Year-wise trend of mobile device usage in pain management applications. "> Figure 3
Year-wise trend of information and communication technology (ICT) domains in pain management applications. "> Figure 4
Pain data input methods. "> Figure 5
Frequency of input for pain data. "> Figure 6
Pain management mobile applications aspects of design concerns. "> Figure 7
Pain management mobile applications aspects of design concerns (cont.). "> Figure 8
Usability evaluation methods. "> Figure 9
Usability measures in the reviewed literature. ">

Versions Notes

Abstract

The increasing popularity of mHealth is a promising opportunity for pain self-management. Mobile apps can be easily developed, but understanding the design and usability will result in apps that can retain more users. This research aims at identifying, analyzing, and synthesizing the current state-of-the-art of: (a) the design approach and (b) usability assessment of pain management mobile applications. A systematic literature review was conducted on 27 studies retrieved from Medline, PubMed, EMBASE, Web of Science, and Scopus. The review revealed that most of the apps were for chronic pain. No app was specifically for men or for the elderly. None of the studies involved expert-based system inspection methods. Only one study used two different approaches of automated and empirical evaluation. We mapped the identified usability issues to ISO 9241-11 and ISO/IEC 25010, and aggregated the recommendations for improvement. Moreover, we also identified certain issues that are solely concerned with the patient’s behavior. We organized the issues into taxonomies of design considerations for building usable pain self-management mobile applications. As pain is prevalent among the elderly, pain management will be much needed while moving toward an aging society. However, we found that the involvement of the elderly in the development of pain management mobile apps is very minimal, which may affect the utility and usability of the apps.

Keywords:

healthcare; pain management; usability; mobile applications; systematic literature review

1. Introduction

Pain is a primary cause to seek physician care [1]. It is the fifth vital sign of basic body functions to be assessed during a routine health examination [2]. Prolonged pain could turn into a chronic condition, which is among the top six major diseases involving high societal cost [3]. It is also ranked fourth among the top 11 global causes of disability and suffering [1]. Thus, an optimal solution is required to ease the suffering and prevent declining mobility, which also inhibits the routine activities. To deal with the situation, pain management provides ways to assess and manage pain effectively and improve the quality of life [4]. Mobile technology plays an effective role in pain management by providing mobile applications as a non-pharmacological approach to cope with chronic pain [5,6,7,8]. The role of pain management mobile applications can also contribute to information sharing with patients by educating them for self-management and improving their quality of life [9,10]. However, more studies are required to measure the impact [8]. In this way, better health service and communication could be established between the healthcare professionals and patients [11,12].

Hundreds of pain management mobile applications are available on the major mobile application distribution platforms. Several studies have been conducted on the commercially available pain management mobile applications, focusing on the content, functionality, and quality of the applications [6,7,13,14,15,16,17]. However, usability, which greatly affects the success or failure of any mobile applications, is not thoroughly studied for pain management mobile applications. Only one study focused on the usability of the commercially available mobile applications other than the quality of content and functionality [15]. It was recommended to develop a participatory design and integrate users’ needs to produce usable applications. A high level of usability involves low development and maintenance costs, and results in a high retention of users [18]. Nevertheless, mobile devices and user-centric mobile applications have made the usability evaluation more critical [18]. Thus, understanding the design and usability requirements of mobile applications is an essential task for every mobile application development project.

Therefore, the aim of this research is to fill the gap in the existing literature. This study will systematically review the pain management mobile applications that are documented in the scientific literature to identify, analyze, and synthesize the current state-of-the-art of the design approaches and the usability features and assessment approaches of pain management mobile applications targeted at the evaluation studies. To fulfill the research aim, this review sets out to answer some research questions, as mentioned in the research method section. The design taxonomy, usability issues, and recommendations/solutions will be identified to help pain management mobile application developers and researchers easily elicit usability requirements when developing the applications. The identified aspects will be mapped to the usability characteristic in an ISO/IEC 25010 Quality Model to aid in the systematic usability evaluation of pain management mobile applications.

This study is part of our research project on pain management mobile applications. To the best of our knowledge, no systematic literature review exists in this context at the time of this report.

2. Research Method

We followed the approach proposed by Kitchenham et al. [19] for conducting this systematic literature review (SLR). This approach is adapted from the guidelines followed by medical researchers to adhere to SLR in software engineering research [20]. This review is also guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement [21] to evaluate the quality of the SLR. The subsequent sections will show various steps of the research methodology, i.e. planning the review, conducting the review, and visualizing the review results, followed by discussions. The Supplementary Materials shown the S1 PRISMA Checklist.

2.1. Research Questions

There are four main research questions for this review. Two of the questions are further divided into sub-questions, as follows:

RQ1:

What are the general characteristics of the pain management mobile apps with respect to pain type, targeted population, and outcome measures?

RQ2:

What are the technologies adopted by the pain management mobile applications with respect to mobile devices, application platform, and techniques that are involved in the proposed solutions?

RQ3:

What are the adopted approaches for data entry of pain management mobile applications?

RQ3.1:: What are the types of pain data assessment that are adopted?
RQ3.2:: What are the pain data input modes that are involved?
RQ3.3:: What are the pain data input methods that are used?
RQ3.4:: What is the optimal pain data input frequency?

RQ4:

What are the approaches to evaluate the usability of pain management mobile applications?

RQ4.1:: What methods are used to evaluate the usability?
RQ4.2:: Which usability features are more targeted in the studies?

2.2. Search Strategy

An electronic search space was defined. The electronic databases were explored by the specific keywords to identify the potential studies for further analysis. A snowball sampling technique was also adopted to add more relevant studies via references. Inclusion and exclusion criteria were defined to select the suitable literature. Multiple screenings were performed to remove the redundancies and irrelevancies by mutual consensus of the researchers. The selected articles were further assessed quality-wise to avoid any bias.

2.3. Information Sources

Five popular scientific databases were identified to conduct the electronic search and retrieve the potential studies for this review. The databases include Medline, PubMed, EMBASE, Web of Science, and Scopus. Some additional records were also identified via Google Scholar.

2.4. Search Terms

As the scope of the topic is broad with no specific taxonomy, a range of relevant key terms was defined to provide maximum coverage. Boolean combination (AND/OR) of the keywords was used to minimize the retrieval of irrelevant literature. The keywords include “pain management and assessment”, “information and communication technology”, “interactive health communication applications”, “mobile technology”, “mobile devices”, “mobile applications”, “smartphones”, “hand held devices”, “personal digital assistant”, “electronic diaries”, “ehealth”, “mhealth”, “usability evaluation”, and “usability framework”. The defined search strings were used across all of the databases.

2.5. Inclusion and Exclusion Criteria

Studies included in the review comprised the following characteristics:

The study must be a peer-reviewed publication.
The study must be published within the timeframe of 2005 to 2017.
The study must be in English.
The study subject must be humans with no age limitation.
The study must include details of the design and development of patient-oriented pain management mobile applications or systems.
The study must involve a touchscreen portable handheld device such as a smartphone, Personal Digital Assistants (PDA), or tablet, etc.
The study must involve manual entry or the partial automatic extraction of data by some sensor devices.
The study must provide information on the usability evaluation method that was involved in that research.
The study must provide results regarding user feedback on the application, and highlight the usability issues.

Studies were excluded based on the following concerns:

If it does not fulfill the inclusion criteria.
If the study is not focused on the design, development, and evaluation of the pain management mobile applications.
If it is a review, commentary, or editorial paper.
If it involves an ordinary cell phone for reminder services or for voice recognition only.
If it involves a physician-oriented app for decision support or assessment.

2.6. Study Selection Procedure

Multiple screening was done with consensus, in relevance to the defined set of rules to enhance the quality of the reported SLR and reduce the biasness in the selected studies. At first, title-based screening was done to extract the relevant studies from the searched results, and duplicate results were removed. Abstract-based screening was performed to select the articles for full-text review. Studies were then examined based on the inclusion and exclusion criteria. It resulted in 27 potential studies for this review. The visual presentation of the study selection process is shown in Figure 1.

2.7. Quality Assessment

A quality assessment measure was adopted to scrutinize the worth of the selected research studies for the current review. An investigation was carried out independently by two researchers to reduce biases. Differences were discussed to agree upon a mutual consensus. Quality and bias assessment help provide additional reliance on the review findings. Each study was graded on an ordinal response scale: 0 (No), 0.5 (Partially), and 1 (Yes). Quality assessment criteria and the grades acquired by the studies are shown in Table 1:

2.8. Data Extraction

The selected studies have gone through a full-text review. Necessary information was extracted and recorded in a predefined form to get a general perception of the studies. The attributes in this context are the title of the publication, authors, year, problem, study focus, contribution, limitation, and future work. More data was extracted to justify the research questions. The attributes for this purpose consisted of: application or system name, information and communication technology (ICT) domain involved, device type, operating system, pain type, age group, gender, technology literacy level, outcome measures, involvement of stakeholders, data input techniques, approaches for pain data assessment, visualization techniques, frequency of data input, methods to evaluate usability, and relevant results. The recorded data were further sorted by the year of publication to observe any chronological trend.

3. Results and Discussion

A total of 27 papers was reviewed, and the analysis indicates that mostly, there were one or two publications each year. In 2014 and 2015, the maximum numbers of publications were observed i.e. four and, five respectively. Whereas, no research work was found for the year 2008. Two publications in 2017 were found when this research was conducted in July 2017.

3.1. Overview

RQ1:: What are the general characteristics of the pain management mobile apps with respect to pain type, targeted population, and outcome measures?

Pain Type

The term “type” has been used for different categorizations of pain, e.g. chronic or acute, general or disease related. For clarity of discussion, we first divide pain types into two categories, i.e. acute and chronic pain. It was found that 25 publications presented apps for pain management that were relevant to chronic pain, and two research papers proposed solutions for acute pain management [22,23]. Under the chronic pain condition, three apps were for general chronic pain type [16,24,25], and 22 apps were for disease/regional-specific pain type. Among these 22 apps, nine focused on cancer pain [26,27,28,29,30,31,32,33,34], four were for pain in sickle cell disease [35,36,37,38], two were for back pain [39,40], two were for headache/migraine [41,42] and there was one app each for musculoskeletal [43], fibromyalgia [44], juvenile idiopathic arthritis [45], pelvic pain in dysmenorrhea [46], and neck pain [40]. This is shown in Table 2.

Most of the evaluated apps were for chronic pain, with cancer pain as the prominent pain type (under disease-specific type). Next to it were sickle cell pain and general chronic pain. Only two apps were developed for the postoperative acute pain. This difference indicates that chronic pain is more relevant than acute pain in patient-oriented pain management mobile apps.

Targeted Population

Selected papers were analyzed from two perspectives of a population: gender and age. There were four women-specific studies (15% of the identified studies) [41,43,44,46]. Two articles (7% of the studies) [25,40] didn’t specify gender, and the remaining 21 publications (78% of the studies) presented apps for both genders.

As per age perspective, four publications (15% of the studies) [16,25,33,43] did not specify the age group of the targeted population, and nine publications (33% of the studies) [28,29,30,32,34,36,38,42,45] targeted adolescents and young adults of a mean age of 15. The remaining 14 publications (52% of the studies) gave pain management solutions for adults, middle-aged, and aged people. Nevertheless, we can see the explicit involvement of the elderly in only three publications (11% of studies) [22,26,27].

The majority of the apps were irrespective of the gender. There is no specific app for men. Although pain is most prevalent among the elderly (by age groups), there is insufficient involvement of the elder population in the design and development process of pain management mobile apps. Adopting a user-centered development approach and involving the elderly in the usability evaluation studies will be useful to increase motivation and acceptance among elderly for pain management mobile apps. Understanding the requirements, uncertainties, and difficulties encountered by the elderly, as well as addressing the technological, psychological, sociological, intellectual, and physical aspects of a user interface specific to this age group are the challenges to overcome in this context [47].

Nine publications (33% of the studies) [16,22,23,28,30,37,40,41,45] mentioned that the potential users had an average to high technology literacy level for mobile devices, while three publications (11% of studies) [27,31,44] showed that the proposed app was designed for patients with low or no technology literacy. One app was designed for patients that can have any technology literacy level [46]. The remaining 14 publications (52% of the studies) didn’t mention the expected level of the technology literacy of the prospective users. To ensure that the low technologically literate group may also benefit, there is a need to resolve the usability and accessibility issues faced by this group.

Outcome Measures

Nine or 33% of the publications [22,23,24,25,27,35,37,39,48] were only based on record keeping for the pain summaries (intensity, duration, and location), while 18 or 65% of the solutions also gathered data for the associated symptoms of pain such as fatigue, nausea, vomiting, oral problems, breathlessness or sleep quality, etc.

RQ2:: What are the technologies adopted by the pain management mobile applications with respect to the mobile devices, application platform, and techniques that are involved in the proposed solutions?

The result of the year-wise technological trend with respect to the device type is shown in Figure 2. Whereas, Figure 3 represents the year-wise trend of the involvement of information and communication technology (ICT) domains in the pain management applications. From 2005 to 2017, there was a technology shift in the use of mobile devices for pain management solutions. Previously, solutions used handheld devices, mainly Personal Digital Assistants (PDAs). Since 2009, the smartphone has become a prominent mobile device for pain self-management interventions. Smartphones provide features of both personal computing and mobile communication. This also represents an increased trend of new platform adoption i.e., from the Palm operating system to Android, iOS, or the Windows operating systems. Mobile applications on smartphones are flexible enough to be operated on any advanced wireless mobile devices, with better control of the enhancement or modification as technology advances.

Figure 3 shows that Internet technology and mobile computing are the two prominent domains that are involved in pain management solutions. Some other domains also contributed to this field such as web technology, virtual reality, artificial intelligence, and computer graphics. The domains together have made the solutions attractive with increased acceptability by the patients. It is also worthy to incorporate important features such as context awareness and easy access. Data visualization, i.e. two-dimensional (2D) and three-dimensional (3D) modeling, is another important area that is appealing to end-user perception [49].

RQ3:

What are the adopted approaches for data entry of pain management mobile applications?

RQ3.1:: What are the types of pain data assessment adopted?

Pain management is dependent on pain assessment based on the analysis of self-reported data. The analysis results in the generation of a patient report that helps the healthcare provider to provide feedback to the patients and make the necessary judgment to proceed in the treatment regime. Our analysis found that there are two types of assessment: 1) Static or Dynamic, which is based on the number of questions; and 2) Standard or Customized, which is based on the nature of questions and response set. All except three studies involved static assessment.

In [24], item response theory (IRT) and computerized adaptive testing (CAT) were used to propose a dynamic assessment method for recording pain and secondary symptoms. When the set precision level was achieved, the dynamic assessment was completed. In [45], there was a built-in response loop to skip irrelevant questions as the patient proceeded with the answers. In [22], the healthcare provider set the patient treatment plan in the system, and then operated accordingly. Although dynamic assessment lessens the respondents’ burden, no significant difference was observed between the two according to a survey [24]. Almost all of the apps have adopted the standard approach of an assessment questionnaire and answers for the patients. Only three apps provided a customized set of questions and responses for the patients. [16] introduced the concept of preliminary discussion between the therapist and patient, and the resulting information then populates the chronic pain intervention with the individual’s personalized goals, activities, and parameters for physical activity.

RQ3.2:: What are the pain data input modes involved?

There are three modes of pain self-reporting: active, passive, and hybrid [50]. In active reporting, a patient has to operate the application to record pain and the associated symptoms. In the passive mode, there is either an embedded feature of the mobile device or something that is wearable to automatically gather and record data without the active involvement of patients. In contrast, the hybrid mode is a blend of both the active and passive functionalities [50]. Almost all of the apps had adopted the active approach of data input from the users. None of the apps used passive data input. Only one study proposed the hybrid mode of data input, i.e. through the inbuilt accelerometer and Global Positioning System (GPS) technology to automatically gather the required information [16].

Proper self-reporting is the basis for the effective analysis of recorded data, results, and feedback. Hence, simple and precise reporting methods must be adopted to make data input easier for the patients and enhance usability [51]. Ease in app usage also has a direct impact on self-efficacy [52]. Nevertheless, with passive or hybrid input techniques, privacy is an additional concern apart from convenience and effectiveness.

Incorporating a suitable reporting method in relation to the targeted audience is another area of consideration. For example, the elderly population has several cognitive, physiological, and physical limitations. Thus, their preference for data input methods could be different. It is reported that the elderly prefer wearable devices over mobile applications, [53,54] i.e., the active mode of data input in a simplified way [54]. Nevertheless, the wearable device could only be helpful to input pain intensity, whereas there are some other attributes of pain data to be input as well. SOCRATES (Site, Onset, Character, Radiation, Associated Symptoms, Timings, Exacerbating and relieving factors, and Severity) is a mnemonics that is used for clinical assessment of pain [15]. A mobile application should allow the input of these pain data easily apart from pain intensity so that the mobile application is more useful in the clinical context.

RQ3.3:: What are the pain data input methods used?

Figure 4 displays pain data input methods of the assessment questionnaire, which include the visual rating scale, multiple choice question, body outline drawing, options list, and free text questions.

The result shows that a visual rating scale is the prominent data input method. Next to it are multiple choice questions and a body outline diagram/pain drawing. Using pain drawing is a simple method to record pain location and pain type, and has been successfully practiced for more than five decades [39]. This technique was adopted not only for adolescents, but also across all age spectrums. As the body outline diagram can only gather pain type and location, it requires other techniques to support complete pain assessment. Three apps [29,30,34] utilized all of the pain data input methods stated above to effectively record pain data and related symptoms.

Although no prominent chronological trend was observed regarding the use of the input methods, it is notable that poor input methods will cause low usability, and may result in user frustration. For instance, members of the aging population, which is the major age group that suffers from pain, are considered late adopters of the mobile technology [55], and slower in the first-time use compared to young adults [56]. A loss of muscle mass, dexterity, or visual impairments and cognitive decline may affect the learning ability and task performance of the elderly [47]. Thus, the input method of the pain management application must be adapted to this limitation. Nevertheless, no usability study in terms of the pain data input method in relation to age group has been conducted.

RQ3.4:: What is the optimal pain data input frequency?

As shown in Figure 5, out of 27 studies, six of them (22%) didn’t mention the frequency of input for pain data. Among the other 21 studies: four solutions (15%) involved only one time of self-reporting per day [31,40,42,57], nine (33%) were designed for self-reporting twice in a day [28,29,32,33,34,35,36,38,58], four (15%) [27,43,44,45] provided an intake of pain data three times by the patients per day, and four (15%) mentioned the record keeping of pain symptoms in very short intervals, i.e. once every two to three hours [22,25,39,41].

There is no clear guideline for the frequency of input that could be retrieved from the literature. Some researchers are of the view to adopt the scenario of pain self-assessment once a day to avoid overburdening the patients. Some are of the view to capture pain data more frequently to support the concept of ecological momentary assessment than relying on the recall method. The minimal and optimal data input frequencies, as per the targeted population and type of pain they are suffering from, remain open for investigation.

3.2. Finding: A Taxonomy of Design Concerns

Based on all of the above findings, a taxonomy of the design concerns of pain management mobile applications is derived as shown in Figure 6 and Figure 7. This taxonomy will be useful for the researchers and application developers in developing pain management mobile applications. By referring to this taxonomy, they can quickly identify the system requirements for the targeted mobile application, in relation to the aspects of pain type, population, outcome measures, technology, and data entry. Different design options are presented for each aspect. For example, the researchers and developers can correlate the data input method with respect to pain type and the targeted population to provide enhanced usability. Outcome measures and the suitable technology of the pain management mobile application can also be chosen based upon the patients’ and clinicians’ preferences. The taxonomy will also aid the application developers with keep the highlighted design perspectives in consideration while developing or modifying an application for pain management.

3.3. Usability of Pain Management Mobile Applications

RQ4:

What are the approaches to evaluate the usability of pain management mobile applications?

RQ4.1:: What methods are used to evaluate usability?

All of the studies adopted empirical methods to evaluate usability, which include a questionnaire, interview, thinking aloud, observational notes, and email. A questionnaire is the mostly used method, which was adopted by 16 studies [22,23,24,26,28,29,31,32,33,35,39,41,42,43,44,48]. Similarly, an interview is also the mostly used method, and it was involved in 16 studies [23,26,27,28,29,30,31,33,34,38,41,42,43,45,57,58], followed by thinking aloud (four studies) [28,34,45,57], observational notes (four studies) [23,25,34,38], focus group discussion (one study) [33], and inquiry by email (one study) [58]. It was also found that only one study involved an automated usability evaluation via a software activity log [26].

Less than 40% (11) of the studies followed a single usability evaluation technique: seven studies used only a questionnaire, three studies involved an interview, and one study involved reviewing the videotaped session of usability testing and taking notes of the user behavior and dialogue. Nine (33%) of the studies adopted two techniques for usability evaluation. In this regard, the studies involved an interview in combination with a questionnaire [29,31,41,42,43], thinking aloud [45,57], observational notes [38], or email [58]. Only five studies followed multiple usability evaluation methods, i.e. a combination of more than two usability evaluation methods. In this regard, a questionnaire, an interview, and thinking aloud methods were involved in two studies [28,34]; one study adopted a questionnaire, an interview and a focus group discussion [33]; one study involved a questionnaire, an interview, and observational notes [23]; while [26] investigated usability by combining empirical and automated methods, i.e. a questionnaire, an interview, and a software activity log. These results are shown in Figure 8.

According to [59], the taxonomy of usability evaluation methods includes automatic, empirical, formal, and informal methods. Empirical evaluation is the most commonly used method. It can be further classified as a user study method (involving users) and a system inspection method (involving experts). The results of this systematic review show that all of the studies had adopted empirical usability evaluation methods, but they practiced user study methods only. None of the studies involved expert-based system inspection methods. Although many studies involved a combination of two or more empirical usability evaluation methods, only one study investigated usability with the methods from two different categories, i.e. automatic (using system log) and empirical evaluation. Moreover, the accuracy of the evaluation instruments that were used in these studies is not clear. One study [48] used a questionnaire that is the adapted version of the System Usability Scale (SUS). In [45], an iterative rapid design development approach was adopted that depends on the theory of the hermeneutical circle testing of the interface by emphasizing usability. Adopting more than one usability evaluation approach by using validated instruments can help elicit more usability issues. However, minimizing redundancy while adopting more than one usability evaluation approach is essential for maintaining cost effectiveness.

The evaluation was mostly done by real-time users where almost all of the studies involved patients for the usability evaluation. Only six studies [26,30,31,33,38,39] involved both patients and healthcare providers. The results also revealed that some of the studies followed the iterative development approach in which the pain management mobile application was tested by the users and modified accordingly, in each iteration. Among these, four studies [38,39,41,45] involved two iterations, whereas three iterations [29,34,42] and four iterations [36] were carried out in three studies and one study, respectively. For evaluation, the potential consumers use the mobile applications for a specified period, ranging over a few minutes, days, weeks, and several months. The minimum time duration was 15 minutes [48], and the maximum evaluation time was four months [24].

In designing any successful product or software application, the mere inclusion of various features is not very significant. Instead, user satisfaction and ease of use are more important to make software usage successful. A user-centered design approach helps elicit the requirements effectively. An evaluation of features by the end users and real-time testing via a working prototype are also necessary before the final release of a software application [46]. When end users test software in the real environment, it helps improve the design and increase the product’s usability [60]. In this case, the iterative design approach can help elicit more usability issues. The user-centered design also varies. It may involve some stakeholders in some design phases, and some can be involved throughout the design process. An effective pain management system also involves healthcare providers and clinical expertise in the designing and testing process. However, it has been observed from the selected literature that only a few applications involved clinical expertise.

RQ4.2:: Which usability features are more targeted in the studies?

According to ISO 9241-11 [61], usability is defined as the extent to which a user of any product can use that product with effectiveness, efficiency, and satisfaction to achieve a specified goal, in a particular context of use. The ISO/IEC 9126-1 [62] quality model incorporates understandability, learnability, operability, and attractiveness as quality-in-use to measure usability. This standard has been replaced by ISO/IEC 25010 [63], which renames some usability sub-characteristics in the ISO/IEC 9126-1 standard and introduces some new usability sub-characteristics. The usability characteristic is divided into sub-characteristics of appropriateness, recognizability, learnability, operability, user error protection, user interface aesthetics, and accessibility.

We categorized the terminologies that are used in the studies to measure usability, with respect to ISO 9241-11 and ISO/IEC 25010, as shown in Table 3.

The most targeted usability features in the studies are operability and satisfaction, followed by appropriateness recognizability/understandability and effectiveness. Seventeen studies have measured the operability of the pain management mobile applications, 16 studies measured satisfaction, 15 studies measured appropriateness recognizability, and 11 studies measured the effectiveness of the applications. User interface aesthetics/Attractiveness was measured by nine studies. Efficiency and Learnability were measured by five studies each. The least measured usability features were user error protection and Accessibility, which were measured by four studies and two studies, respectively.

Accessibility is the least studied usability sub-characteristic [64], which measures the use of the application by people with limited abilities [65]. Nevertheless, some accessibility problems may also affect non-disabled users [64]. Although usability evaluation doesn’t mean targeting all of the issues faced by every user [65], with technologies becoming potential and necessary solutions for the elderly’s healthcare needs in an aging society, it is necessary to make the solutions accessible to the elderly, who suffer from pain the most [66]. In addition, learning new skills is another challenge for older adults due to cognitive constraints. Providing pain self-management solutions that are easy to learn will eliminate the cognitive load and increase the acceptance of the technology among the older users. Error prevention is also a vital component. An approach to notifying the users, for example before making any deletion, gives encouragement to verify their task, thus leading to a low error rate and improved usability [67]. Ironically, this much-needed usability sub-characteristic by the elderly is also least studied (apart from accessibility).

Figure 9 shows the occurrences of usability measures in the reviewed literature.

3.4. Finding: Usability Issues and Solutions

This review has identified 27 unique usability issues. We also mapped the issues with the solutions or recommendations, as reported in the literature. Five issues do not have any specific solution or recommendation. Table 4 represents the synthesis of these findings. These findings will be helpful for the mobile application designers and developers to deal with during the development stage to produce more usable products. The solutions and recommendations will also be useful in the formulation or reforming of the checklist in usability evaluation studies.

3.5. Discussion: A Usability Matrix of Usability Issues

We designed a usability matrix comprising the ISO 9241-11 and ISO/IEC 25010 standards to categorize the usability issues, as depicted in Table 5.

In ISO 9241-11, effectiveness is the accuracy to achieve a goal and the completion rate of a certain task. Similarly, in ISO/IEC 25010, appropriateness recognizability represents the suitability of a product according to the needs of the user that ultimately helps to achieve a specific goal. Also, ISO/IEC 25010 defines accessibility as addressing the limitations of the users that have the widest range of characteristics, so that the goals could be achieved in the specified context of use. Thus, the usability issues that fall under the appropriateness recognizability and accessibility of ISO/IEC 25010 were mapped with the effectiveness of ISO 9241-11.

Efficiency signifies the effort exerted by the user to perform a task, according to ISO 9241-11. On the other hand, ISO/IEC 25010 defines operability as the means of ease to operate and control a product or system. Therefore, the usability issues that are identified as operability issues are mapped with efficiency. In addition, learnability symbolizes the ability to learn the system’s usage with minimal effort in ISO/IEC 25010. Therefore, the learnability issues are mapped against efficiency, too.

Satisfaction is meant to be the comfort and acceptability of use as per the standards of ISO 9241-11. Similarly, in ISO/IEC 25010, appropriateness recognizability represents the suitability of a product according to the needs of the user that eventually leads to the acceptability of the product or system. User error protection is defined as the system’s ability to prevent a user from making errors, and user interface aesthetics is defined as a smooth and satisfying interaction of the user with the system in ISO/IEC 25010. These definitions relate to the ease and acceptability of use. Thus, the usability issues that belong to appropriateness recognizability, user error protection, and user interface aesthetics are mapped with satisfaction. It has to be noted that appropriateness recognizability is mapped to both effectiveness and satisfaction in this matrix.

One-to-one mapping criteria have been adopted while mapping the usability issues w.r.t. ISO 9241-11 and ISO/IEC 25010. Moreover, usability issues are mapped on the basis of the context as specified in the studies.

Other than the usability issues of device and application listed in Table 5, there are other issues that are concerned with patient behavior, such as:

Over-report or under-report of skill practices via self-reporting [35].
Missing diary entries: being away from home and forgetting to take the diary, being too busy, forgetting to complete the diary, major illness or hospitalization, and the device being inaccessible due to loss or theft [35,57,58].
Frustration in carrying the smartphone in patients with low technology literacy [44].

In [43], one-to-one communication between the patient and their healthcare provider was suggested before starting a mobile-based pain self-management regime. It will help to establish trustworthiness. As pain self-management is a two-way phenomenon, guidance or pain management advice by the healthcare provider is essential for the patient to continue and adjust the regime according to the results and feedback [22,31]. In [24], it is stated that patients can be disheartened by the feedback. In this case, motivation is essential to make patients persist in the self-management program.

We believe that usability measures must be taken in order to address the user behavior issues and thus retain or enhance the users’ motivation to continuously use the applications for achieving optimal outcomes of pain management. Accommodating individual user’s behavior can be realized through static or dynamic personalization. Static personalization can be done through one-off user profiling and then customizing the application based on the user profile. Dynamic personalization can be done through a user activity log that provides feedback to the personalization mechanism, which in turn adapts the application based on the feedback. Alternatively, dynamic personalization can also be done through predictive analytics, which predict the usage behavior (e.g. using machine learning) and proactively adapt the application for the user. In addition, adopting voice input and other natural language processing techniques will increase the usability of the applications too, especially to address the visual and haptic constraints of elderly users.

4. Conclusions

Providing a self-management solution to the patients suffering from pain is vital in order to enhance their quality of life. This review will be helpful for pain management researchers as well as application developers, as it provides insight into the design and usability assessment approaches of pain management mobile applications. This review enables them to understand design considerations and resolve usability issues, specifically for elderly patients with low literacy or less technological experience, so that they can also be benefited by this technology. The elderly are a growing population, and have several requirements in order to sustain their living such as health monitoring (e.g. pain management), information seeking, social interaction, etc. They can also benefit from mobile technology to fulfill their needs. However, as old people are facing some challenges regarding cognition complexity, motivational issues, physical impairment, and perception barriers, it is difficult for them to accept new technology with little effort. Keeping the levels of complexity, uncertainty, and difficulty low for them is vital in order to ensure their acceptance of the new technology, and thus provides an alternative but essential solution to enhance their quality of life.

On the other hand, having a cost-effective hybrid mode of data input method helps prevent overburdening the users and motivates the usage of the application. Moreover, a clearly defined optimal data input frequency for the targeted groups as per the type of pain from which they are suffering ensures effective pain management while relieving the users from overly or insufficient data input. Simple, precise yet appropriate reporting is a key to the perceived usefulness of the application by the users that will result in a high retention rate.

There are a few limitations to this study. One possible limitation is that the scope was limited to five databases. Thus, other relevant literature has not yet been identified. A snowball sampling can be adopted to include more literature. In addition, the study focused on patient-oriented pain management applications only. Other applications that support physicians in their clinical decisions can be reviewed in the future to obtain more insights about pain management, as it is a two-way procedure between the patients and healthcare providers.

Supplementary Materials

The following are available online at https://www.mdpi.com/2073-8994/11/3/400/s1, S1: PRISMA Checklist (PDF). S2: Table S1. Summary of Study Characteristics–Pain Management Mobile Applications aspects of design concerns; Table S2. Summary of Study Characteristics–Pain Management Mobile Applications aspects of usability and some additional data.

Author Contributions

U.e.M.S.: She conducted a search of the scientific databases, screened, selected, evaluated and summarized the findings of this review. She also contributed to writing the manuscript. T.K.C.: He helped to identify the problem area and define the overall research methodology. He critically reviewed, revised and approved the evaluation criteria, the findings, and the final manuscript. This research was conducted under his guidance.

Acknowledgments

This research was supported by the University of Malaya, Malaysia under Grand Challenge—HTM (Wellness) Grant GC003B—14HTM (ICT for Healthcare Connectedness).

Conflicts of Interest

The authors declare that they have no conflict of interest.

References

Arneric, S.P.; Laird, J.M.; Chappell, A.S.; Kennedy, J.D. Tailoring chronic pain treatments for the elderly: Are we prepared for the challenge? Drug Discov. Today 2014, 19, 8–17. [Google Scholar] [CrossRef] [PubMed]
Morone, N.E.; Weiner, D.K. Pain as the fifth vital sign: Exposing the vital need for pain education. Clin. Ther. 2013, 35, 1728–1732. [Google Scholar] [CrossRef]
Gereau, R.W.; Sluka, K.A.; Maixner, W.; Savage, S.R.; Price, T.J.; Murinson, B.B.; Sullivan, M.D.; Fillingim, R.B. A pain research agenda for the 21st century. J. Pain 2014, 15, 1203–1214. [Google Scholar] [CrossRef] [PubMed]
Hardy, P.A. Chronic Pain Management: The Essentials; Cambridge University Press: Cambridge, UK, 1997. [Google Scholar]
Martin, M.Y.; Pisu, M.; Kvale, E.A.; Johns, S.A. Developing effective cancer pain education programs. Curr. Pain Headache Rep. 2012, 16, 332–342. [Google Scholar] [CrossRef] [PubMed]
Lalloo, C.; Jibb, L.A.; Rivera, J.; Agarwal, A.; Stinson, J.N. “There’s a pain app for that”: Review of patient-targeted smartphone applications for pain management. Clin. J. Pain 2015, 31, 557–563. [Google Scholar] [CrossRef] [PubMed]
Alexander, J.C.; Joshi, G.P. Smartphone applications for chronic pain management: A critical appraisal. J. Pain Res. 2016, 9, 731–734. [Google Scholar] [CrossRef] [PubMed]
Sundararaman, L.V.; Edwards, R.R.; Ross, E.L.; Jamison, R.N. Integration of mobile health technology in the treatment of chronic pain: A critical review. Reg. Anesth. Pain Med. 2017, 42, 488–498. [Google Scholar] [CrossRef]
Marceau, L.D.; Link, C.; Jamison, R.N.; Carolan, S. Electronic diaries as a tool to improve pain management: Is there any evidence? Pain Med. 2007, 8 (Suppl. 3), S101–S109. [Google Scholar] [CrossRef]
Chuna, Y.J.; Patterson, P.E. A suggestion for future research on interface design of an internet-based telemedicine system for the elderly. Work 2012, 41 (Suppl. 1), 353–356. [Google Scholar]
Consulting, V.W. Mhealth for Development: The Opportunity of Mobile Technology for Healthcare in the Developing World; UN Foundation-Vodafone Foundation Partnership: Washington, DC, USA, 2009. [Google Scholar]
Norris, A.C.; Stockdale, R.S.; Sharma, S. A strategic approach to m-health. Health Inf. J. 2009, 15, 244–253. [Google Scholar] [CrossRef]
De la Vega, R.; Miro, J. Mhealth: A strategic field without a solid scientific soul. A systematic review of pain-related apps. PLoS ONE 2014, 9, e101312. [Google Scholar] [CrossRef]
Portelli, P.; Eldred, C. A quality review of smartphone applications for the management of pain. Br. J. Pain 2016, 10, 135–140. [Google Scholar] [CrossRef] [Green Version]
Reynoldson, C.; Stones, C.; Allsop, M.; Gardner, P.; Bennett, M.I.; Closs, S.J.; Jones, R.; Knapp, P. Assessing the quality and usability of smartphone apps for pain self-management. Pain Med. 2014, 15, 898–909. [Google Scholar] [CrossRef]
Rosser, B.A.; McCullagh, P.; Davies, R.; Mountain, G.A.; McCracken, L.; Eccleston, C. Technology-mediated therapy for chronic pain management: The challenges of adapting behavior change interventions for delivery with pervasive communication technology. Telemed. J. e-Health Off. J. Am. Telemed. Assoc. 2011, 17, 211–216. [Google Scholar] [CrossRef]
Wallace, L.S.; Dhingra, L.K. A systematic review of smartphone applications for chronic pain available for download in the united states. J. Opioid Manag. 2014, 10, 63–68. [Google Scholar] [CrossRef]
Nayebi, F.; Desharnais, J.-M.; Abran, A. The State of the Art of Mobile Application Usability Evaluation; CCECE: Montreal, QC, Canada, 2012; pp. 1–4. [Google Scholar]
Kitchenham, B. Procedures for Performing Systematic Reviews; Keele University: Keele, UK, 2004; Volume 33, pp. 1–26. [Google Scholar]
Mariano, D.C.; Leite, C.; Santos, L.H.; Rocha, R.E.; de Melo-Minardi, R.C. A guide to performing systematic literature reviews in bioinformatics. arXiv, 2017; arXiv:1707.05813. [Google Scholar]
Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G. Preferred reporting items for systematic reviews and meta-analyses: The prisma statement. Ann. Intern. Med. 2009, 151, 264–269. [Google Scholar] [CrossRef]
Pombo, N.; Araujo, P.; Viana, J.; da Costa, M.D. Evaluation of a ubiquitous and interoperable computerised system for remote monitoring of ambulatory post-operative pain: A randomised controlled trial. Technol. Health Care Off. J. Eur. Soc. Eng. Med. 2014, 22, 63–75. [Google Scholar]
Ingadottir, B.; Blondal, K.; Thue, D.; Zoega, S.; Thylen, I.; Jaarsma, T. Development, usability, and efficacy of a serious game to help patients learn about pain management after surgery: An evaluation study. JMIR Serious Games 2017, 5, e10. [Google Scholar] [CrossRef]
Anatchkova, M.D.; Saris-Baglama, R.N.; Kosinski, M.; Bjorner, J.B. Development and preliminary testing of a computerized adaptive assessment of chronic pain. J. Pain Off. J. Am. Pain Soc. 2009, 10, 932–943. [Google Scholar] [CrossRef]
Luckmann, R.; Vidal, A. Design of a handheld electronic pain, treatment and activity diary. J. Biomed. Inf. 2010, 43, S32–S36. [Google Scholar] [CrossRef] [Green Version]
Kearney, N.; Kidd, L.; Miller, M.; Sage, M.; Khorrami, J.; McGee, M.; Cassidy, J.; Niven, K.; Gray, P. Utilising handheld computers to monitor and support patients receiving chemotherapy: Results of a uk-based feasibility study. Supportive Care Cancer Off. J. Multinatl. Assoc. Supportive Care Cancer 2006, 14, 742–752. [Google Scholar] [CrossRef]
Hachizuka, M.; Yoshiuchi, K.; Yamamoto, Y.; Iwase, S.; Nakagawa, K.; Kawagoe, K.; Akabayashi, A. Development of a personal digital assistant (pda) system to collect symptom information from home hospice patients. J. Palliat. Med. 2010, 13, 647–651. [Google Scholar] [CrossRef]
Baggott, C.; Gibson, F.; Coll, B.; Kletter, R.; Zeltzer, P.; Miaskowski, C. Initial evaluation of an electronic symptom diary for adolescents with cancer. JMIR Res. Protoc. 2012, 1, e23. [Google Scholar] [CrossRef]
Stinson, J.N.; Jibb, L.A.; Nguyen, C.; Nathan, P.C.; Maloney, A.M.; Dupuis, L.L.; Gerstle, J.T.; Alman, B.; Hopyan, S.; Strahlendorf, C.; et al. Development and testing of a multidimensional iphone pain assessment application for adolescents with cancer. J. Med. Internet Res. 2013, 15, e51. [Google Scholar] [CrossRef]
Jibb, L.A.; Stevens, B.J.; Nathan, P.C.; Seto, E.; Cafazzo, J.A.; Stinson, J.N. A smartphone-based pain management app for adolescents with cancer: Establishing system requirements and a pain care algorithm based on literature review, interviews, and consensus. JMIR Res. Protoc. 2014, 3, e15. [Google Scholar] [CrossRef]
Maguire, R.; Ream, E.; Richardson, A.; Connaghan, J.; Johnston, B.; Kotronoulas, G.; Pedersen, V.; McPhelim, J.; Pattison, N.; Smith, A.; et al. Development of a novel remote patient monitoring system: The advanced symptom management system for radiotherapy to improve the symptom experience of patients with lung cancer receiving radiotherapy. Cancer Nurs. 2015, 38, E37–E47. [Google Scholar] [CrossRef]
Fortier, M.A.; Chung, W.W.; Martinez, A.; Gago-Masague, S.; Sender, L. Pain buddy: A novel use of m-health in the management of children’s cancer pain. Comput. Biol. Med. 2016, 76, 202–214. [Google Scholar] [CrossRef] [Green Version]
Hochstenbach, L.M.; Zwakhalen, S.M.; Courtens, A.M.; van Kleef, M.; de Witte, L.P. Feasibility of a mobile and web-based intervention to support self-management in outpatients with cancer pain. Eur. J. Oncol. Nurs. 2016, 23, 97–105. [Google Scholar] [CrossRef] [Green Version]
Jibb, L.A.; Cafazzo, J.A.; Nathan, P.C.; Seto, E.; Stevens, B.J.; Nguyen, C.; Stinson, J.N. Development of a mhealth real-time pain self-management app for adolescents with cancer: An iterative usability testing study. J. Pediatr. Oncol. Nurs. 2017, 34, 283–294. [Google Scholar] [CrossRef]
McClellan, C.B.; Schatz, J.C.; Puffer, E.; Sanchez, C.E.; Stancil, M.T.; Roberts, C.W. Use of handheld wireless technology for a home-based sickle cell pain management protocol. J. Pediatr. Psychol. 2009, 34, 564–573. [Google Scholar] [CrossRef]
Jacob, E.; Stinson, J.; Duran, J.; Gupta, A.; Gerla, M.; Ann Lewis, M.; Zeltzer, L. Usability testing of a smartphone for accessing a web-based e-diary for self-monitoring of pain and symptoms in sickle cell disease. J. Pediatr. Hematol. Oncol. 2012, 34, 326–335. [Google Scholar] [CrossRef] [PubMed]
Demiris, G. The diffusion of virtual communities in health care: Concepts and challenges. Patient Educ. Couns. 2006, 62, 178–188. [Google Scholar] [CrossRef] [PubMed]
Bakshi, N.; Stinson, J.N.; Ross, D.; Lukombo, I.; Mittal, N.; Joshi, S.V.; Belfer, I.; Krishnamurti, L. Development, content validity, and user review of a web-based multidimensional pain diary for adolescent and young adults with sickle cell disease. Clin. J. Pain 2015, 31, 580–590. [Google Scholar] [CrossRef] [PubMed]
Serif, T.; Ghinea, G. Recording of time-varying back-pain data: A wireless solution. IEEE Trans. Inf. Technol. Biomed. 2005, 9, 447–458. [Google Scholar] [CrossRef]
Blodt, S.; Pach, D.; Roll, S.; Witt, C.M. Effectiveness of app-based relaxation for patients with chronic low back pain (relaxback) and chronic neck pain (relaxneck): Study protocol for two randomized pragmatic trials. Trials 2014, 15, 490. [Google Scholar] [CrossRef] [PubMed]
Sorbi, M.J.; Mak, S.B.; Houtveen, J.H.; Kleiboer, A.M.; van Doornen, L.J. Mobile web-based monitoring and coaching: Feasibility in chronic migraine. J. Med. Internet Res. 2007, 9, e38. [Google Scholar] [CrossRef] [PubMed]
Huguet, A.; McGrath, P.J.; Wheaton, M.; Mackinnon, S.P.; Rozario, S.; Tougas, M.E.; Stinson, J.N.; MacLean, C. Testing the feasibility and psychometric properties of a mobile diary (mywhi) in adolescents and young adults with headaches. JMIR mHealth uHealth 2015, 3, e39. [Google Scholar] [CrossRef] [PubMed]
Nes, A.A.; Eide, H.; Kristjansdottir, O.B.; van Dulmen, S. Web-based, self-management enhancing interventions with e-diaries and personalized feedback for persons with chronic illness: A tale of three studies. Patient Educ. Couns. 2013, 93, 451–458. [Google Scholar] [CrossRef] [Green Version]
Garcia-Palacios, A.; Herrero, R.; Belmonte, M.A.; Castilla, D.; Guixeres, J.; Molinari, G.; Baños, R.M.; Botella, C. Ecological momentary assessment for chronic pain in fibromyalgia using a smartphone: A randomized crossover study. Eur. J. Pain 2014, 18, 862–872. [Google Scholar] [CrossRef]
Stinson, J.N.; Petroz, G.C.; Tait, G.; Feldman, B.M.; Streiner, D.; McGrath, P.J.; Stevens, B.J. E-ouch: Usability testing of an electronic chronic pain diary for adolescents with arthritis. Clin. J. Pain 2006, 22, 295–305. [Google Scholar] [CrossRef]
Kangas, E.; Kinnunen, T. Applying user-centered design to mobile application development. Commun. ACM 2005, 48, 55–59. [Google Scholar] [CrossRef]
Holzinger, A.; Searle, G.; Nischelwitzer, A. On some Aspects of improving mobile applications for the elderly. In Proceedings of the International Conference on Universal Access in Human-Computer Interaction, Beijing, China, 22–27 July 2007; Springer: Berlin, Germany; pp. 923–932. [Google Scholar]
Spyridonis, F.; Gronli, T.-M.; Hansen, J.; Ghinea, G. Evaluating the usability of a virtual reality-based android application in managing the pain experience of wheelchair users. In Proceedings of the 2012 Annual International Conference of the Engineering in Medicine and Biology Society (EMBC), San Diego, CA, USA, 28 August–1 September 2012; pp. 2460–2463. [Google Scholar]
Liu, C.; Zhu, Q.; Holroyd, K.A.; Seng, E.K. Status and trends of mobile-health applications for ios devices: A developer’s perspective. J. Syst. Softw. 2011, 84, 2022–2033. [Google Scholar] [CrossRef]
Richardson, J.E.; Reid, M.C. The promises and pitfalls of leveraging mobile health technology for pain care. Pain Med. 2013, 14, 1621–1626. [Google Scholar] [CrossRef]
Gasser, R.; Brodbeck, D.; Degen, M.; Luthiger, J.; Wyss, R.; Reichlin, S. Persuasiveness of a mobile lifestyle coaching application using social facilitation. In Persuasive Technology; Springer: Berlin, Germany, 2006; pp. 27–38. [Google Scholar]
Cho, J.; Park, D.; Lee, H.E. Cognitive factors of using health apps: Systematic analysis of relationships among health consciousness, health information orientation, ehealth literacy, and health app use efficacy. J. Med. Internet Res. 2014, 16, e125. [Google Scholar] [CrossRef]
Rodríguez, I.; Cajamarca, G.; Herskovic, V.; Fuentes, C.; Campos, M. Helping elderly users report pain levels: A study of user experience with mobile and wearable interfaces. Mob. Inf. Syst. 2017, 2017, 9302328. [Google Scholar] [CrossRef]
Parker, S.J.; Jessel, S.; Richardson, J.E.; Reid, M.C. Older adults are mobile too! Identifying the barriers and facilitators to older adults’ use of mhealth for pain management. BMC Geriatr. 2013, 13, 43. [Google Scholar] [CrossRef]
Smith, A.L.; Chaparro, B.S. Smartphone text input method performance, usability, and preference with younger and older adults. Hum. Factors 2015, 57, 1015–1028. [Google Scholar] [CrossRef]
MacKenzie, I.S.; Tanaka-Ishii, K. Text Entry Systems: Mobility, Accessibility, Universality; Elsevier: Amsterdam, The Netherlands, 2010. [Google Scholar]
Nguyen, A.M.; Humphrey, L.; Kitchen, H.; Rehman, T.; Norquist, J.M. A qualitative study to develop a patient-reported outcome for dysmenorrhea. Qual. Life Res. Int. J. Qual. Life Asp. Treat. Care Rehabil. 2015, 24, 181–191. [Google Scholar] [CrossRef]
Jonassaint, C.R.; Shah, N.; Jonassaint, J.; De Castro, L. Usability and feasibility of an mhealth intervention for monitoring and managing pain symptoms in sickle cell disease: The sickle cell disease mobile application to record symptoms via technology (smart). Hemoglobin 2015, 39, 162–168. [Google Scholar] [CrossRef]
Nielsen, J. Usability inspection methods. In Proceedings of the Conference Companion on Human Factors in Computing Systems, Boston, MA, USA, 24–28 April 1994; ACM: New York, NY, USA, 1995; pp. 377–378. [Google Scholar]
Abras, C.; Maloney-Krichmar, D.; Preece, J. User-centered design. In Encyclopedia of Human-Computer Interaction; Bainbridge, W., Ed.; Sage Publications: Thousand Oaks, CA, USA, 2004; Volume 37, pp. 445–456. [Google Scholar]
ISO 9241-11: Ergonomic Requirements for Office Work with Visual Display Terminals (vdts); The International Organization for Standardization: Geneva, Switzerland, 1998; Volume 45.
International Organization for Standardization; International Electrotechnical Commission. Software Engineering—Product Quality: Quality Model; ISO/IEC: Geneva, Switzerland, 2001; Volume 1. [Google Scholar]
Organización Internacional de Normalización. ISO-IEC 25010: 2011 Systems and Software Engineering-Systems and Software Quality Requirements and Evaluation (Square)-System and Software Quality Models; ISO: Geneva, Switzerland, 2011. [Google Scholar]
Petrie, H.; Kheir, O. The relationship between accessibility and usability of websites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, San Jose, CA, USA, 28 April–3 May 2007; ACM: New York, NY, USA, 2007; pp. 397–406. [Google Scholar]
Billi, M.; Burzagli, L.; Catarci, T.; Santucci, G.; Bertini, E.; Gabbanini, F.; Palchetti, E. A unified methodology for the evaluation of accessibility and usability of mobile applications. Univers. Access Inf. Soc. 2010, 9, 337–356. [Google Scholar] [CrossRef]
Holzinger, A.; Searle, G.; Wernbacher, M. The effect of previous exposure to technology on acceptance and its importance in usability and accessibility engineering. Univers. Access Inf. Soc. 2011, 10, 245–260. [Google Scholar] [CrossRef]
Norman, D. The Design of Everyday Things: Revised and Expanded Edition; Constellation: New York, NY, USA, 2013. [Google Scholar]

Figure 1. Study selection process: Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) flow diagram. From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097. doi:10.1371/journal.pmed1000097. For more information, visit www.prisma-statement.org.

Figure 2. Year-wise trend of mobile device usage in pain management applications.

Figure 3. Year-wise trend of information and communication technology (ICT) domains in pain management applications.

Figure 4. Pain data input methods.

Figure 5. Frequency of input for pain data.

Figure 6. Pain management mobile applications aspects of design concerns.

Figure 7. Pain management mobile applications aspects of design concerns (cont.).

Figure 8. Usability evaluation methods.

Figure 9. Usability measures in the reviewed literature.

Table 1. Quality assessment criteria and grades.

No	Quality Assessment Criteria	Ordinal Response Scale	Grades Acquired by the Studies
QA1	Relevancy of research design within the context of the study.	0 (No)/0.5 (Partially)/1 (Yes)	26 studies, 98%
QA2	Clearly defined aims and objectives of the study.	0 (No)/0.5 (Partially)/1 (Yes)	26 studies, 98%
QA3	Clearly stated findings and limitations of the study.	0 (No)/0.5 (Partially)/1 (Yes)	25 studies, 94%
QA4	Valuable contribution of the study, based on the findings.	<20% (No)/20–80% (Partially)/>80% (Yes)	23 studies, 85%

Table 2. Pain management apps for specific pain types.

Pain Types	Acute Pain	Chronic Pain
General pain	-	3
Cancer pain	-	9
Sickle cell pain	-	4
Back pain	-	2
Musculoskeletal pain	-	1
Headache or migraine	-	2
Fibromyalgia pain	-	1
Juvenile idiopathic arthritis	-	1
Dysmenorrhea pelvic pain	-	1
Neck pain	-	1
Postoperative pain	2	-

Table 3. Mapping of terminologies used in the studies with the usability characteristics of ISO 9241-11 and ISO/IEC 25010.

Characteristics	Definitions	Terminologies Used in the Studies for Usability Evaluations
Effectiveness	“Accuracy and completeness with which users achieve specified goals” [61]	error prevention, usefulness, self-care behavior and symptom management, monitoring and managing symptoms, support of behavioral training key targets
Efficiency	“Resources expended in relation to the accuracy and completeness with which users achieve goals” [61]	completion time
Satisfaction	“Freedom from discomfort and positive attitudes toward the use of the product” [61]	usefulness, willingness to complete the survey again, self-efficacy, likability, acceptability, feedback
Appropriateness recognizability/ Understandability	“Degree to which users can recognize whether a product or system is appropriate for their needs” [63]	clarity of content, ease to understand, helpfulness of the tool in understanding the impact of pain, simplicity of the content, clarification of the wording of questions, readability, language, understanding, paraphrasing, comprehension (meaning and understanding of the question), completeness, comprehensiveness of the queries and response set, knowledge
User interface aesthetics/Attractiveness	“Degree to which a user interface enables a pleasing and satisfying interaction for the user” [63]	font size, color scheme, visual appearance and layout, inconsistency, aesthetics
Operability	“Degree to which a product or system is easy to operate, control, and appropriate to use” [63]	Navigation, ease of data input, ease of use, User-friendliness, the difficulty of completing the assessment, the responsiveness of the screens to touch, navigation, functionalities (submit, clear, cursor movement), user interaction, navigation of the site, issues with the interface, complexity, knowledge, communication and support, feedback
Learnability	“Degree to which a product or system enables the user to learn how to use it with effectiveness, efficiency in emergency situations” [63]	memory retrieval (ability to accurately recall the answer), complexity, workload demand
User error protection	“Degree to which a product or system protects users against making errors” [63]	system protection against making errors, error prevention
Accessibility	“Degree to which a product or system can be used by people with the widest range of characteristics and capabilities to achieve a specified goal in a specified context of use” [63]	the color scheme for the color blind, accessibility

Table 4. Usability issues and their solutions or recommendations.

	Usability Issues	Solutions/Recommendations
1	The difficulty with the interpretation of the report or confusing reports [24,42].	Additional reports were added along with the functionality of viewing web-based report and export diary data. It was also recommended to provide comprehensive reports to be viewed within the mobile application rather than be externally viewed [42].
2	Confusion in understanding the terminology or wording [30,34,39,42,45,57].	The wordings of questions were simplified and in the user manual; definitions of certain words/terminologies were also provided [42,45]. It was also suggested that the wording of the system’s questions and management instructions should be appropriate for the targeted population [30,34,57].
3	Buttons not working properly [34].	The “clickable” surface area corresponding to the app buttons and cursors was increased [34].
4	Items with overlapping concepts [34,57].	Keywords were made bold and underlined so that adolescents could easily distinguish the meaning of pain assessment questions [34]. Diary items were refined based on the feedback to remove the overlapping concepts [57].
5	Slow responsiveness of the application [34,42].	In order to expedite the process of data input and increase the efficiency of the application, some measures were taken, such as an addition of auto text or default values, alphabetically sorting the response set, and the provision of the most frequently entered options at the top of list. Moreover, data will be stored locally to overcome the issue and synchronized with the server simultaneously [42].
6	Patients were forced to complete the entire sequence of questions, irrespective of whether they were not having any pain, as patients didn’t move the slider to the “No Pain” anchor [45].	The addition of help features at various stages of data input to prevent users from errors e.g. reminding patients to move the slider to the end of the VAS if they are experiencing no pain [45].
7	A few participants missed inputting the medications used, as they did not know the generic names [45].	Changes were made to the diary by putting the brands as well as the generic names of medications [45].
8	Difficult to locate and identify the triggers due to the complex hierarchal presentation and too many options to choose from [42].	Relocation of the feature to make it accessible. The question was set to appear on the top of the screen while scrolling down to keep the question in mind during searching. Most of the repeated triggers were listed down in the list of the most frequently used triggers [42].
9	Problem with retrospective entries e.g., past midnight entry without starting a new day or day-long constant pain entry [42].	Removed the restriction on creating a diary entry if one already existed [42].
10	Difficulty in diary entry due to a few inapplicable symptoms for constant pain [42].	Option to skip or remove the unusable items was provided [42].
11	Difficulties in setting the start and end time of pain as wake or sleep time [42].	Default items were set for daily items [42].
12	Difficulty in learning to use the application [39,42].	Provision of instruction slides on the first launch of the application to explain its working and the addition of help buttons in various modules [42]. In [44], the feature of audio-recorded instructions was also incorporated to increase the accessibility for a wider number of people, e.g., elderly people or people with any visual impairment.
13	Failed to review all of the response options due to unseen or a not prominent enough prompt to scroll down for more responses [25].	All of the response options for a query were presented on one screen to avoid the need for scrolling [25].
14	Application very cumbersome to use [34,38,48]	Navigation was streamlined to minimize the steps to move from one module to the other [34], or the redistribution of items was done to shorten the length of the pain diary [38]. A second response loop was also designed, i.e,. if the answer of a question is marked “yes”, then the diary will proceed to ask the next relevant question for elaboration; otherwise, those nested questions will be skipped [38].
15	Difficulty in handling visual analog scale, the slider was too sensitive and tricky to use [45]	The slider was transformed to be thicker on the VAS so that it would move more easily. In some inputs, the slider was replaced by the radio buttons to make the input easier [45].
16	Problem in using stylus-based input, i.e., inputting number with stylus or selection from word descriptor list [24,25,39,45].	Some space was added for entering new pain word descriptors. Also, word descriptors were arranged in alphabetical order, and the scroll bar was moved away from the word descriptors for clarity [45].
17	The difficulty with number selection boxes where the tapping of the arrow is required for selection [25]	Replaced the number selection box with a pop-up number pad [25].
18	Missing ‘‘back” button to return to the earlier screens to revise answers [25]	Back button was added to review the answers before submission [25].
19	Frequent crashing/Software malfunction [23,34,58]	The application was reprogrammed and tested internally to resolve the malfunctioning [34].
20	Difficulty in selecting an area on the body image [38] and not enough spots to highlight the problem area on the body diagram [45].	Zoom function in the body diagram was provided for visual clarity [38]. In [39], it was also suggested to provide 3D visualization. In [45], the presentation location of some of the joints was improved. Also, labeled images of the body diagram were added in the instruction manual. Moreover, navigation assistance indicators were provided to move the body diagram left or right.
21	Less effectiveness and safety of a standalone system without active input of the healthcare professional [30].	It was suggested in [39] that the applications should be connected with the central healthcare systems. The active input of a clinical expert could improve the effectiveness and safety of the system. In that case, a threshold level should be incorporated into the system to trigger an email alert to a registered nurse. The nurse will then contact the adolescent to assist in clinical decision making. Moreover, the threshold level could be established for the system to receive additional advice from the healthcare professional, if no improvement in pain condition is observed [30]. There must be a time-out feature in the system, in case of seeking advice from the healthcare professionals, to communicate with their care providers or other medical aid [30,39].
22	Lack of acknowledgment for the patient [30,39].	An alert system is also necessary to acknowledge the patient upon successful transmission of pain data to the central system [30,39].
23	Over alerting or repeated notification [41,58].	No specific solution recommended
24	Confusing pop-up screen messages [24].	No specific solution recommended
25	Difficult to control the slider while scrolling down the screen [24].	No specific solution recommended
26	Too small font size [39]	No specific solution recommended
27	Color scheme not suitable for color blinds [39].	No specific solution recommended

Table 5. Mapping of Usability Issues w.r.t. ISO 9241-11 and ISO/IEC 25010.

ISO 9241-11 IS0/IEC 25010	Effectiveness	Efficiency	Satisfaction
Appropriateness recognizability	A few participants missed inputting the medications used, as they did not know the generic names [45]. Less effectiveness and safety of a standalone system without the active input of the healthcare professional. [30] Lack of acknowledgment to the patient [30,39].	The difficulty with the interpretation of the report [24].	Confusing pop-up screen messages [24]. Confusion in understanding the terminology or wording [30,34,39,45,57]. Items with overlapping concepts [34,57]. Over alerting or repeated notification [41,58].
Learnability	-	Difficulty in learning to use the application [39].
Operability	-	Difficult to control the slider while scrolling down the screen [24,25]. Application very cumbersome to use [31,48]. Difficulty in handling visual analog scale, the slider was too sensitive, and tricky to use [38,45]. Problem in using stylus-based input i.e., inputting number with stylus or selection from word descriptor list [24,25,39,45]. The difficulty with number selection boxes where tapping of the arrow is required for selection [25]. Missing ‘‘back” button to return to the earlier screens to revise answers [25]. Buttons not working properly [34]. Slow responsiveness of the application [34,42]. Difficult to locate and identify the triggers due to the complex hierarchal presentation and too many options from which to choose [42]. Problem with retrospective entries e.g., past midnight entry without starting a new day or day-long constant pain entry [42]. Difficulty in diary entry due to a few inapplicable symptoms for a constant pain [42]. Difficulties in setting start and end time of pain as wake or sleep time [42]. Failed to review all of the response options due to an unseen or not prominent prompt to scroll down for more responses [25]. Difficulty in selecting an area on the body image [38] and not enough spots to highlight the problem area on the body diagram [45].	-
User Error Protection	-	-	Frequent crashing/software malfunction [23,34,58]. Patients were forced to complete the entire sequence of questions, irrespective of the fact they were not having any pain, as patients didn’t move the slider to the “No Pain” anchor [45].
User Interface Aesthetics	-	-	Too small font size [39].
Accessibility	1. Color scheme not suitable for color blinds [39]	-	-

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Share and Cite

MDPI and ACS Style

Shah, U.e.M.; Chiew, T.K. A Systematic Literature Review of the Design Approach and Usability Evaluation of the Pain Management Mobile Applications. Symmetry 2019, 11, 400. https://doi.org/10.3390/sym11030400

AMA Style

Shah UeM, Chiew TK. A Systematic Literature Review of the Design Approach and Usability Evaluation of the Pain Management Mobile Applications. Symmetry. 2019; 11(3):400. https://doi.org/10.3390/sym11030400

Chicago/Turabian Style

Shah, Umm e Mariya, and Thiam Kian Chiew. 2019. "A Systematic Literature Review of the Design Approach and Usability Evaluation of the Pain Management Mobile Applications" Symmetry 11, no. 3: 400. https://doi.org/10.3390/sym11030400

APA Style

Shah, U. e. M., & Chiew, T. K. (2019). A Systematic Literature Review of the Design Approach and Usability Evaluation of the Pain Management Mobile Applications. Symmetry, 11(3), 400. https://doi.org/10.3390/sym11030400

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu