Measurement of Adherence to mHealth Physical Activity Interventions and Exploration of the Factors That Affect the Adherence: Scoping Review and Proposed Framework

Introduction

Background

There is strong evidence that physical activity (PA) is associated with improvements in physical health, mental health, and well-being [1]. However, many people are at risk of inactivity and there is poor uptake of, and adherence to, PA interventions [2]. Promoting PA and maintaining people’s adherence are crucial public health issues.

Technological innovation (eg, mobile health [mHealth]) is developing rapidly and is being widely applied in the health care field [3]. mHealth mainly focuses on the delivery and monitoring of health care services [4] and could also be an alternative approach to delivering PA interventions, overcoming the limitations of traditional PA approaches (eg, classes or workshops) [5]. Compared with traditional approaches, mHealth can use vivid video and pictures and may be more attractive and acceptable [6]. The use of mHealth can help deliver exercise programs to a wide audience at a low cost. In addition, mHealth technologies can provide timely feedback, reminders and support, continuous monitoring, and outcomes assessment [7,8].

There is a common issue with innovative health technologies, including mHealth, which is users’ adherence to mHealth programs. For example, people who download an exercise app do not always use, or continue to use, the app. Research suggests that suboptimal exposure to the PA program lessens the effects of these interventions [9]. Measuring users’ adherence to mHealth and exploring the factors that could influence users’ adherence and the association between adherence and intervention outcomes are important to understand how PA and other outcomes can be improved.

Although there are systematic reviews summarizing evidence on the adherence to technology-based interventions, previous reviews are rather generic, do not differentiate mHealth from other technology-based interventions, do not focus on PA, and do not address the issue that adherence to a PA intervention may differ from adherence to other interventions such as those for medications or therapy. For example, Donkin et al [10] summarized measurement methods for adherence to any e-therapies and evaluated the association of adherence with intervention outcomes. Perski et al [11], focusing on digital behavior change interventions, developed a conceptual framework to explain the impacts of potential factors on people’s engagement with digital behavior change interventions. By contrast, other reviews may focus on PA but restrict themselves to specific digital technologies. For example, Attig et al [12], focusing on wearable trackers for PA, summarized reported reasons for abandoning their use, such as usability issues and privacy concerns.

mHealth is a commonly used innovative solution to deliver health interventions. The characteristics of instant access, portability, and direct feedback make mHealth different from other technologies such as desktop computers [4]. We believe that these characteristics could affect the adherence of users of mHealth-based interventions in a way that is different from adherence to other technologies. Therefore, it would be better to consider mHealth-based interventions more specifically rather than grouping them together with other, generic technologies. Given this, we consider that the measurement of users’ adherence to mHealth-based PA should not only reflect generic technologies’ features, where applicable, but also incorporate mHealth-specific factors. For example, the automatically recorded number of days when mHealth devices are worn and the automatically recorded daily amount of PA, such as step counts, are commonly used indicators in measuring users’ adherence to mHealth-based PA interventions [8]. However, such indicators have not been well considered in existing reviews that cover information technologies in general. Overall, there is a lack of evidence on determinants of the adherence to mHealth-based PA and the influence of adherence on health outcomes; in addition, there seems to be no agreed measurement method of adherence to mHealth devices that aim to improve PA engagement.

A scoping review of the literature was therefore carried out to explore how adherence to mHealth aiming at improving PA engagement is measured, to investigate which factors affect users’ adherence, and the association between adherence and intervention outcomes. A framework is needed to identify the association among factors, adherence measurement, and health outcomes. The framework can be used in future to guide the measurement of adherence to mHealth-based PA programs and facilitate further research on the effectiveness of mHealth-based programs.

Objectives

The aims of this study were to synthesize evidence about (1) how adherence to mHealth PA interventions has been measured in the literature, (2) the factors that influence the adherence, and (3) the association between adherence to mHealth PA interventions and health outcomes. An additional aim was to propose an operational concept of adherence to mHealth-based PA programs and a framework for identifying the links among the determinants of adherence, adherence measurements, and intervention outcomes.

Methods

The PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines were followed for reporting this review [13]. Refer to Multimedia Appendix 1 for the PRISMA-ScR checklist. The review protocol was not registered on the web.

Literature Search and Search Strategy

To identify relevant studies, the initial electronic searches were run in September 2020 in 4 databases: MEDLINE, PsycINFO, EMBASE, and CINAHL [14]. Update searches were performed in February 2022. Refer to Multimedia Appendix 2 for the search strategies. There were no restrictions on publication year, but searches were limited to English language publications.

Eligibility Criteria

We considered a study eligible if it met the criteria presented in Table 1. Following the World Health Organization’s definitions of mHealth [15], we defined mHealth-based PA programs as interventions that use mobile devices to deliver PA. The devices could be smartphones, smartwatches, PDAs such as wristbands, and other wireless technologies. In addition, there were no restrictions on how adherence to mHealth interventions was defined and assessed in this review as long as the authors described the measurement or definition of adherence. Furthermore, this review used the PA definition provided by the World Health Organization [16]: any bodily movement produced by skeletal muscles that requires energy expenditure.

Study Selection

We used EndNote (Clarivate Analytics) to manage records identified through the electronic searches. After removing duplicate records, we screened titles and abstracts at first to identify potentially eligible studies and then screened their full texts to include eligible studies. Given that this is a scoping review, only 1 reviewer (YY) was involved in this process. However, any problem was resolved by consulting another researcher (CT or EB).

Data Extraction

We used predefined data extraction forms and extracted the following details: (1) characteristics of the included studies: study design, study population, sample size, the description of the intervention, mHealth used, the goal of the intervention, control program, follow-up duration, and outcome measurement; and (2) factors that could influence adherence and the relationship between adherence and outcomes.

Data Synthesis

The critical interpretive synthesis approach was used to synthesize both qualitative and quantitative data [17]. Concepts identified in the full texts of included studies were labeled. The research questions were used as a top-down coding frame. We coded text fragments that were explicitly or implicitly related to any of the following three topics: (1) adherence measurement, (2) predictors of adherence, and (3) association between adherence and intervention outcomes. Synthetic constructs (ie, concepts that explain similar themes) were developed from the codes, and relationships between the synthetic constructs were specified.

Framework Development Methods

On the basis of the aforementioned synthetic constructs, we developed an integrative adherence framework by following the theoretical causal pathway to map the synthetic constructs of the scoping review. This framework can show the links among the determinants affecting adherence, multidimensional adherence measurements, and the association between adherence and health outcomes.

Results

Search Results

The electronic database searches retrieved a total of 10,902 records. Title and abstract screening of these 10,902 records resulted in 150 (1.38%) requiring full-text inspection. Of these 150 papers, 54 (36%) were included in this review (Figure 1).

‎View this figure

Characteristics of the Included Studies

The characteristics of the included studies are summarized and reported in Textbox 1 (refer to Multimedia Appendix 3 [18-71] for full details). Most of the interventions were delivered through smartphone apps. The sample sizes of the included studies ranged from 10 to 16,948 (median 86). The average age of participants in the included studies ranged from 23.6 (SD 4.6) years to 73.2 (SD 7.3) years (median 51.7, SD 11.2).

Summary of Adherence Measurement Methods

We identified 31 specific adherence measurement methods used (Table 2). The top 3 most frequently used methods were manually entering self-monitored health behavior data into the device (17 studies), recording PA data (eg, step count) automatically recorded on the devices (10 studies), and recording the frequency of daily access to the app (8 studies).

These 31 measurement methods were related to 8 measurement indicators that generally reflect 4 measurement dimensions: length, breadth, depth, and interaction (Table 2). Among the 4 dimensions, the breadth dimension was the most frequently measured (35 studies). Of the 54 included studies, 31 (57%) measured adherence in only 1 dimension, 15 (28%) measured adherence in 2 dimensions, and 6 (11%) measured adherence in 3 dimensions, whereas 2 (4%) included all 4 dimensions.

Factors That Affect Adherence to mHealth PA Programs

In the included studies, there are 3 factors affecting the adherence reported: user characteristics, technology-related factors, and contextual factors (Table 3). For the 5 specific user characteristics, the included studies showed inconsistent evidence on the influence of age, education status, and weight on adherence, whereas the studies by Ryan et al [59] and Guertler et al [64] consistently suggested that men had higher adherence. The study by Edney et al [39] reported that being overweight reduced adherence. The included studies consistently showed that almost all 12 technology-related factors (9 mHealth functions and 3 specific factors related to the experience of using mHealth devices) increased users’ adherence. Only 1 contextual factor was identified in this review: weekdays have higher adherence than weekends [59].

Association Between Adherence and Intervention Outcomes

Associations between adherence and intervention outcomes were reported in 13 studies. Higher adherence was reported as having associations with higher PA in 5 studies [36,39,61,65,70] or with physical function in 1 study [27], with greater weight loss in 6 studies [26,38,39,46,59,71], and with greater reductions in glycated hemoglobin levels in 2 studies [38,51]. Höchsmann et al [43] reported that participants’ adherence was positively associated with intrinsic PA motivation inventory change scores.

Proposed Framework of the Adherence to mHealth PA Interventions

The framework proposed by Perski et al [11] explicitly links potential influences on engagement and relationships between engagement and intervention effectiveness. We drew on the same structure for our adherence framework. We mapped our aforementioned review findings into a similar framework to illustrate interactions among the factors affecting adherence to mHealth PA interventions, the 4 dimensions of the adherence measurement, and intervention effectiveness (Figure 2).

Generally, most specific factors were related to aspects of user characteristics, the technology itself, and contexts that could increase users’ adherence. The adherence to mHealth could be reflected by indicators associated with 4 dimensions (length, breadth, depth, and interaction) and could be measured largely by automatically recorded data and self-reported methods. Higher adherence could improve intervention effectiveness on health behaviors as well as psychological and clinical outcomes.

‎View this figure

Discussion

Principal Findings

In this scoping review, we synthesized evidence from 54 studies. The findings are as follows:

1. Users’ personal characteristics (eg, sex and education status), mHealth devices’ features, and contextual factors were reported to increase users’ adherence to mHealth PA interventions.
2. Adherence was reportedly reflected or measured in 4 dimensions and their corresponding indicators: breadth (device functions used and completion of modules), depth (meeting tasks or challenges and behavior change), length (device use time and length of use), and interaction (active or passive interaction with the program elements).
3. Higher adherence was associated with better outcomes in terms of health behaviors (eg, increasing PA), PA motivation, and clinical outcomes (eg, decrease in glycated hemoglobin levels, blood pressure, and weight).

These findings were summarized to propose an initial comprehensive adherence framework based on a theoretical causal pathway with three parts: (1) the factors that can affect users’ adherence to mHealth PA interventions, (2) multidimensional adherence measurements, and (3) the association between adherence and intervention outcomes.

Adherence Measurement Issues

This scoping review identified 3 issues related to the measurement of adherence to mHealth PA programs in the literature. First, there is little consensus on how adherence should be defined and measured. In total, 31 specific adherence measurement methods were identified in the literature. The heterogeneity of the methods of assessing adherence makes comparison across studies difficult. Second, adherence measurement should be evidence based or theory informed; however, only the study by Adu et al [32] referred to a framework to measure adherence. Third, adherence to PA interventions is a complex definition rather than a single-dimension method [72], but many of the studies (31/54, 57%) included in this review measured adherence using only 1 dimension.

This review attempts to address the aforementioned issues through developing a new adherence conceptual framework based on the review findings. This framework, with 4 dimensions, is expected to inform the comprehensive measurement of users’ adherence to mHealth PA interventions. The four dimensions are as follows:

1. The length dimension can reflect whether the users still use the mHealth devices and how much time they spend on the devices. This dimension should be considered essential in measuring adherence and can be reflected by the time to attrition, the frequency of access to the devices, and the duration of use.
2. The breadth dimension can reflect how many device functions are involved in PA and how many modules are completed by participants. This dimension can help understand the usability of each function of an mHealth device and the engagement and involvement of users. It was the most frequent consideration in measuring adherence in the included studies. This dimension is especially worth considering when the mHealth program has multiple functions or a number of intervention modules.
3. The depth dimension can assess how well users adhere to the task of an mHealth program. It can be measured by whether participants complete the program tasks or meet the PA target. For mHealth programs included in this review, tasks were usually related to behavior changes that could be considered to reflect the depth dimension. For example, the device recorded or self-reported PA time.
4. The interaction dimension reflects how users interact with mHealth programs. In the included studies, the most frequently used methods to assess interaction included recording and counting the number of posts written or users’ responses and assessing the users’ access to educational materials.

Evidence on the topic of adherence or the related concept of engagement was included in 3 previous research studies [11,12,73]; however, all 3 differ from this review in terms of the focus of the interventions. Perski et al [11] developed a conceptual framework to highlight potential influences on engagement with a digital behavior change intervention and relationships between engagement and target behaviors. The authors’ framework specified potential direct and indirect influences on engagement but did not aim to show how the engagement could be measured. The framework we have developed in this review focuses on the topic of adherence measurement and also explores the factors that influence users’ adherence to mHealth. In addition, Perski et al [11] considered the engagement of users with a broad range of digital behavior change interventions rather than focusing on mHealth-based PA interventions. Attig et al [12] emphasized the exploration of factors that are related to the abandonment of an activity tracker rather than conceptualizing a framework for abandonment. Attig et al [12] identified less-intensive device use, less device interaction, and amount of PA achieved as important factors that affect abandonment of activity tracker use. We agree with the importance of these factors and have included them in our proposed adherence framework. Couper et al [73] analyzed data from a randomized controlled trial that aimed to evaluate the effectiveness of a web-based intervention in promoting dietary changes. They produced a 2D adherence measurement approach that consisted of breadth and depth. Breadth was defined as how widely users could access all available functions on the website, and depth was defined as how deeply users were engaged in the web-based material. Couper et al [73] considered that the breadth-depth engagement led to intervention outcomes: users’ retention and behavior changes. However, we consider retention and behavior change as specific aspects of length and depth, respectively. This is because both retention and behavior change are only intermediate, moderating effects in terms of mHealth use patterns, rather than determining the influence of users’ adherence on clinical outcomes resulting from PA, such as specific balance and strength outcomes. Given this, we consider our evidence-based adherence framework to be specifically relevant to mHealth-delivered PA interventions.

When choosing methods to measure adherence, researchers need to consider the following issues:

1. The characteristics of the mHealth program (eg, whether the mHealth program has multiple functions). The length and breadth dimensions are generally applicable to most mHealth programs, whereas the interaction and depth dimensions are usually applicable only to mHealth programs with relevant functions.
2. The purpose of the study. For example, if the study’s purpose is to assess the effectiveness of the mHealth intervention, users’ deep engagement with the mHealth intervention is needed, meaning that the interaction and depth dimensions should be considered.

The final issue regarding adherence measurement is how adherence data can be collected. Automatic recording and self-reporting adherence are 2 common approaches, but they both have strengths and limitations. Automatic recording can be objective but may not detect some specific types of activity (eg, stationary movement or upper or lower body movement) [74]. Self-reporting is easy to use and applicable to all types of PA, but the validity of this approach can be affected by social desirability bias [75]. The ideal way to measure adherence to mHealth PA interventions may be a combination of the objective measurement using mHealth and self-reporting.

Factors That Influence Users’ Adherence

Some findings of this review are inconsistent with previous work: this review suggests that male sex, older age, and secondary education could be predictors of higher adherence. However, in a systematic review investigating the predictors of adherence to web-based psychological interventions, Beatty and Binnion [76] suggest that female sex and older age predict higher adherence. Another review finds that younger age is associated with higher adherence to internet interventions [77]. These disagreements may be related to the differences in the types of interventions and target populations considered.

This review found 12 technology-related factors that could affect users’ adherence, including 9 related to mHealth functions and 3 related to users’ experience. This is consistent with another systematic review in terms of technology-related factors (eg, PA tracking, PA goal setting, and customization of exercise) [78]. In the study by Attig et al [12], the reasons for abandonment included data inaccuracy, privacy concerns, discomfort, loss of motivation, and loss of tracking feasibility. This finding suggested that the mHealth devices themselves are the key to users’ adherence, and in designing mHealth, the target users could be involved to improve device functions and thus adherence.

Adherence and Intervention Outcomes

As this review and the adherence framework demonstrates, adherence could affect intervention outcomes, including health behavior outcomes such as PA level, psychological indicators such as PA motivation, and clinical indicators such as weight loss and blood pressure. The findings suggest that further studies should explore the association between adherence and health outcomes from the aforementioned 3 perspectives.

Hawley-Hague et al [72] and Donkin et al [10] suggest that different types of clinical outcomes could be predicted using different types of adherence indicators (eg, log-ins were related to outcomes in physical health interventions, whereas module completion was most related to outcomes in psychological health interventions). Understanding how adherence influences the effectiveness of interventions could be crucial to understanding how adherence should be defined. However, this review identified only a limited number of studies that evaluate the association between adherence to mHealth and health outcomes. Further study is needed to explore in depth the association between each adherence dimension and each health outcome dimension.

Limitations of This Review

This review includes limitations. First, given that this is a scoping review, the searches for this review were limited to publications in English; hence, evidence published in other languages could have been missed. Second, the proposed adherence framework was based on the findings of the scoping review alone; therefore, it may not include all possible factors because of the limited evidence in the literature. Further research could improve this framework by considering evidence from other sources such as expert group meetings. Third, this framework explores the adherence predictors and the association between adherence and health outcomes. However, we did not further explore how these predictors affect each adherence dimension and the associations between each adherence dimension and intervention outcomes. This is because the included studies have limited evidence regarding these issues. Further studies are needed to explore the adherence to mHealth programs by each adherence dimension.

Conclusions

This review suggests that adherence can be measured using the dimensions of length, breadth, depth, and interaction; that users’ characteristics, technology-related factors, and contextual factors can affect adherence; and that adherence is significantly associated with outcomes in terms of health behaviors, psychology, and clinical measures. These findings inform the development of a framework, linking together the adherence predictors, comprehensive adherence assessment, and clinical effectiveness. The framework could facilitate a comprehensive measurement of adherence as well as guide mHealth device development and further studies on adherence to mHealth PA interventions. Further research is needed to validate this framework; for example, by considering evidence from other sources such as expert group meetings or using Delphi approaches.