Behavior Change Techniques and Content Analyses

There is no consensus as to the best method for analyzing content of new media. A common method has been to use health behavior theory to create codes and/or percentage scores of the number of theoretical constructs represented. This process has been used with active video games [15], exercise apps [16], and weight loss apps [17].

A more involved method uses recently developed systematic taxonomies of BCTs to code for content that matches components of traditional behavioral interventions. Behavior change techniques are “observable, replicable, and irreducible component[s] of an intervention designed to alter or redirect causal processes that regulate behavior” [18]. A general hierarchical taxonomy of 93 BCTs has recently been published [18], and a similar taxonomy specific to physical activity and dietary interventions is also available [19]. These validated taxonomies likely provide a more informational and rigorous coding tool than previously used theory-based instruments.

In addition to describing the content of biomedical media, there is also a need to determine the extent to which the behavior change strategies are evidence-based. Several recent analyses have reviewed mobile apps using different sources for their evidence base. Sources have included the Expert Committee for Pediatric Obesity Prevention [20], the Health Education Curriculum Analysis Tool [21], best practices such as those used by the Diabetes Prevention Program [22,23], and clinical recommendations from the American Association of Diabetes Educators [24] and US Public Health Service [25,26].

Although no compendium of evidence-based best practices exists for exercise and weight loss behavioral interventions, several meta-analyses and meta-regressions have provided a general idea of the BCTs typically associated with successful change [27-29]. Investigating the prevalence of these techniques in particular may provide insight into future directions for research and development.

Further, determining how evidence-based BCTs are implemented may also improve standard coding methods. Preliminary evidence suggests that for several of the most common and effective techniques, fidelity to theory-based recommendations in their implementation enhances their effectiveness [30-32]. Thus, more in-depth analysis of implementation would provide valuable additional information, particularly for practitioners and those developing theory-based interventions.

Behavior Change Techniques and eHealth/mHealth

Several recently published articles have provided an overview of the current state of mobile apps for physical activity and weight loss. The results suggest that most apps do not include many BCTs that are thought to be essential to behavioral intervention. For example, one study found that 28% of pediatric obesity prevention apps included goal-setting [20], and a broader study of physical activity apps found that 44% included monitoring of some kind [21]. A study that compared weight loss apps to components of the widely used and validated Diabetes Prevention Program reported better results, with 90% or more including weight loss goals and dietary goals. However, only 20% included a physical activity goal, and fewer than 5% included problem-solving or habit formation [22]. A sample of diabetes apps were found to include a median of only two out of seven self-management behaviors recommended by the American Association of Diabetes Educators [24]. An analysis of fitness video games found greater percentages that included these techniques, such as feedback (17/18), rewards of some kind (16/18), and practice (15/18) [15]. Two content analyses of physical activity apps that used taxonomies of 23 and 26 BCTs, respectively, found that the apps included an average of 5/23 and 8/26 [42,43].

Of the activity monitor apps analyzed here, all 13 included monitoring and goal-setting. Although weight loss was not the primary purpose of most of these systems, 62% (8 /13) included weight loss goals. The prevalence of problem solving (1/13) and habit formation (1/13) was similar to that found in the other apps. As might be expected by the nature of fitness video games versus activity monitor apps, practice was much more prevalent in the video games than in the monitor systems. Rewards were fairly common in the monitor systems (6/13), but not as common as in true video games. These rewards were typical of “gamified” reward systems, including badges and achievements.

The three systems with the most techniques coded (Jawbone, Fitbit, and Nike) included 27, 20, and 19 techniques, respectively, out of the 93 possible. The absolute number of techniques found in a monitoring system may not be informative; in fact, a system with fewer but more effective techniques may ultimately produce a greater impact than a system with more numerous but less effective ones. Further, there exist several iterations of the behavior change taxonomy, ranging from 26 techniques to 93. The total number of possible techniques to be coded makes comparison across studies difficult. A recent meta-analysis of walking and cycling interventions reported that the interventions studied included a mean of approximately six BCTs, ranging from 0-12 out of a possible 26 [36]. A similar study of physical activity interventions among overweight/obese adults found 3-12 techniques out of a possible 40 [38]. The number of techniques found in the monitoring systems studied here ranged from 9-27 out of 93, with most including 12 or more. Upon recalculation using only those techniques that also exist in the 40-item taxonomy [19], we found that the activity monitor systems included 6-12 out of 40 techniques, with an average of 9 techniques per system. Recalculation using the original 26-item taxonomy found an average of 8 techniques out of 26 (range 6-12). Thus, although exact comparison is impossible due to differences in taxonomies over time, it appears that the monitors include a similar number of techniques as can be found in behavioral interventions and a potentially greater number of techniques than found in standard physical activity mobile apps.

Across multiple meta-analyses of physical activity interventions, several techniques were reported to occur in more than half of studied interventions: self-monitoring of behavior, goal-setting, providing instruction, problem solving, and prompting practice [27,28,36,44]. Self-monitoring and goal-setting were also common among the monitoring systems, but instruction, problem solving, and practice were very uncommon. Though the basic content of physical activity interventions and activity monitor systems (self-regulatory techniques such as self-monitoring, goal-setting, and feedback) are the same, monitor systems differ greatly from traditional interventions in the other implemented techniques.

Adherence to Theoretical and Empirical Best Practices

Of the 14 BCTs identified as potentially effective based on their success in previous interventions, five were widely represented across the devices: goal-setting (behavior), review of behavioral goals, feedback of behavior, self-monitoring of behavior, and rewards. Problem solving, action planning, commitment, instruction on how to perform the behavior, and behavioral practice were rare. It may be that these less common techniques are not prioritized by developers or consumers, or perhaps they are more difficult to implement. Problem solving was found in one app, but it provided only generalized tips to overcome problems (some specific to detected behavior and some discussed as common problems). Individualized problem solving would likely be a complex undertaking that would require self-report of barriers and a system for providing automated counseling. Such a system might increase app size unacceptably or be difficult to program. Action planning and commitment occurred in the context of specific challenges that users pledged or committed to undertaking. Gamification such as this appears to be a promising avenue for implementing less common BCTs.

A recent study used classification and regression trees to statistically investigate the effectiveness of combinations of BCTs [45]. The investigators found that a combination of techniques that are now called goal-setting and providing information about consequences was most successful (the analysis used a previous iteration of this taxonomy with slightly different names). They also found that interventions using feedback provision in the absence of review of behavioral goals or information about consequences were the least effective of those studied. Self-monitoring and feedback provision are the backbone of monitor systems, but many systems do not include any kind of information provision regarding specific consequences of behavior. It may be that these bare-bones apps that focus on function do not provide sufficient motivation to encourage consistent activity over time.

Several previous studies have scored apps based on their adherence to theoretical constructs. For a general study of several types of health apps, the mean score found was approximately 8/100, with the highest-scored app receiving 14/100 [17]. For a similar study specific to physical activity apps, the mean score was approximately 10/100, with the highest-scored app rated 28 [16]. It would appear that the activity monitor apps included in this analysis follow theory (here, specifically Social Cognitive Theory) more closely than apps that are not associated with activity monitors. It may be that activity monitors by definition provide behavioral tools that are suggested by Social Cognitive Theory, such as regular, instant, and precise feedback.

The above information leads to the question of what the ideal monitor and monitoring system might include. It is not surprising that two of the most well-known and popular monitors, from Fitbit and Jawbone, were highly adherent to theoretical principles (Fitbit) and evidence-based principles (Jawbone). The Jawbone Up24 was particularly impressive for including all but one of the best practice techniques investigated. However, despite their utility in previous clinical and community interventions, we do not yet know whether these techniques work well in concert and in the context of a wearable device and monitoring app.

Regardless of the number or effectiveness of BCTs included, success for an individual is likely highly influenced by individual preferences and practical issues. For example, the Misfit Shine is the only waterproof monitor of those tested and thus would likely be the most effective for someone who prefers to swim. The BodyMedia, Fitbit, Fitbug, and Jawbone systems provided energy balance information including food logs, which may make them more suitable for weight loss attempts than systems that monitored only activity and weight (although several other monitors can link to other apps that provide this service). Little is known about the reliability and validity of these devices, which could also influence user preferences. Because of the complicated series of variables that potentially influence effectiveness, a decision aid similar to those used in patient-centered outcomes research would be a logical next step for helping potential users choose a monitor in light of their preferences for techniques, game and social functions, appearance, and usability.

Clinical Applications

There exists a large and growing amount of literature demonstrating the utility of Internet- and technology-enhanced (generally called eHealth) energy balance interventions. Although several reviews have found that computer-mediated or telephone-mediated weight loss interventions were less powerful than traditional face-to-face interventions [46,47], a recent meta-regression did not find a significant effect of in-person contact on weight loss at 12 months [40]. Thus, it is currently unclear whether technology-mediated interventions can consistently replicate the effectiveness of standard clinical interventions. Activity monitor apps by definition include self-monitoring and individualized feedback, which are associated with greater effectiveness in technology-based trials [46]. These monitors may be a medium by which more effective tools can be integrated into self-directed, distance interventions.

Although little is known about the efficacy of electronic activity monitors, several clinical trials have provided preliminary data. To our knowledge, three trials have tested BodyMedia’s SenseWear armband, a clinical/research grade armband that is very similar to the commercially available BodyMedia Fit armband. An early study found that adding continuous use of the armband to a 12-week standard behavioral weight loss program produced additional weight loss of approximately 2 kilograms [10]. This finding was not statistically significant in this small sample, but it may be clinically significant if distributed over a large population with respect to disease prevention and health cost reduction. A later study compared a 6-month standard behavioral weight loss (SBWL) program, SBWL plus the armband system, and the armband system alone and found a 5-kilogram difference between SBWL plus armband and SBWL [12]. A 9-month study found a 3-kilogram difference between a SBWL and SBWL plus armband group; however, this difference was not statistically significant [11].

Beyond these more typical implementations for clinical weight loss interventions, electronic activity monitors may also be a useful measure of patient-reported outcomes. Some researchers have begun using patterns of patient ambulation during and after hospitalization as a proxy measure for health, as these patterns can predict readmission [48] and other health outcomes such as quality of life and functional status [49]. Consistent, objective measures provided by these monitors could allow clinicians to identify at-risk individuals for secondary prevention and rehabilitation interventions. The CYCORE (Cyberinfrastructure for Comparative Effectiveness Research) project has demonstrated initial feasibility and acceptability of a system of home-based sensors, including activity monitors, that transmit information to oncologists for early detection of dehydration among head and neck cancer patients [50]. They could also be used to help determine appropriate lengths for hospital stays and to monitor functional independence post-release [51].

Health care professionals’ preferences likely will play a role in how successful a given system is for users who are prescribed the device. Physicians, interventionists, and counselors may find that some of the companion apps are easier to integrate into their personal approach to patient care than others. Ease of surveillance may also play a role in provider choice of monitors. Some monitors more easily lend themselves to various types of surveillance, either by allowing “friends” to view user data, by partnering with other apps that allow for practitioner or friend surveillance, or by allowing users to export their data to third parties. Official methods of transmitting data to practitioners securely do not appear to currently exist in these apps. However, upcoming health information aggregator apps like those made by Apple and Google may provide a method for automatically updating physicians in the future.

Public Health and Community Applications

From a public health perspective, electronic activity monitors hold promise for large-scale, cost-effective activity and energy balance interventions. Much like previous studies of Internet-based behavioral weight loss interventions, monitor-based interventions may be less powerful than standard face-to-face programs [10,12,52]. However, they may also have a greater public health impact due to greater reach, adoption, implementation, and/or maintenance [53]. Initial investigations of the BodyMedia armband have found that it provides a more cost-effective weight loss intervention than standard behavioral weight loss interventions or combinations of the two [52].

Some of the monitors demonstrated a greater emphasis on energy balance, providing tools for monitoring intake, comparing intake to expenditure, and monitoring weight loss (eg, BodyMedia, Fitbit, Fitbug). The apps for BodyMedia, Fitbit, Fitbug, and Withings communicated with smart scales, which automatically uploaded weight measurements to the apps. These monitors and their apps provide interactive tools that mimic a large proportion of the techniques of behavioral weight loss interventions that require skilled interventionist time. These tools could reduce the time needed by interventionists for counseling by creating automated feedback.

Several of the monitoring systems included measurement and cues related to sedentary behavior. Preliminary studies that provided feedback based on baseline analyses of sedentary behavior (using research-grade monitors) produced promising results [54,55]. The existence of commercial monitors that can provide continuous real-time feedback related to sedentary behavior as well as physical activity increases the options available to interventionists. Lumo (sit time, stand ups), Polar (resting, sitting, and low intensities), and Jawbone (longest idle period) monitors measured sedentary behavior and provided mobile phone reminder alerts when sedentary periods extended past a pre-set threshold. These monitors and others that adopt this functionality could be used to implement larger-scale and lower-cost sedentary behavior interventions than those in the past [53].

Rehabilitation Applications

Electronic activity monitors have the potential to significantly improve objective measurement of physical activity for people with chronic diseases and disabilities who receive physical therapy, occupational therapy, and other types of rehabilitation services. While much has been written about the use of pedometers [56,57], accelerometers [56,58,59], and self-report questionnaires [59,60] to measure physical activity for rehabilitation patients, very little has been published expanding to other types of physical activity measurement for these populations. One exception is the development of wearable sensors, such as those described by Bonato et al [61]; however, widespread adoption and testing of wearable technology devices is not evident in the peer-reviewed literature. However, there is agreement among researchers that an effective means of quantifying physical activity is needed. Electronic activity monitors have the potential to offer a solution for gaps in current monitoring systems. For example, these monitors offer researchers and consumers the opportunity to gather physical activity data in real-world conditions such as home and community settings. They also have the capacity to provide real-world behavioral motivation using prompts and intensity measures that are variable or absent in current monitoring methods. Talkowski et al have pointed out the need for accurate physical activity intensity measures that are not currently being accurately evaluated [60]. These authors note that the number of hours of therapy is often a proxy for estimating the intensity of a rehabilitation program, whereas the length of time in therapy may not offer a uniform intensity across patients and over time. An electronic physical activity monitor would provide an objective measure of treatment intensity.

Potential for Unintended Consequences

Though numerous positive applications of these electronic activity monitors exist, there is always the possibility for unintended adverse consequences or ethical dilemmas. The potential for sharing of global positioning system (GPS) location data and personal health information produces clear privacy concerns. Surveillance of the collected data by health care providers may also lead to situations where intervention is deemed ethically necessary. Clear protocols will be necessary to guide provider behavior in such cases and to reduce risks associated with potential privacy breaches.

Because these monitors are commercially available, they can be used by individuals without consultation with medical or public health professionals. Although this widespread availability has benefits for accessibility, it might also increase the risk of negative outcomes if potentially dangerous activity programs are begun without professional oversight. The default activity goals may be inappropriate for older adults, individuals with disabilities or chronic conditions, or children [62]. Though some apps allow users to change their goals, or set goals for them based on a baseline measurement period, others provide pre-set goals that cannot be adjusted. These goals may provoke inappropriately intense activity that could lead to injury.

The validity and reliability of these monitors’ step estimates is as yet unclear. Substantial literature surrounding the validity of the research grade BodyMedia armband exists (eg, [63,64]), but it is not clear whether differences between the research and commercial versions may affect energy expenditure estimates. There is preliminary evidence that one of the Fitbit monitors (worn on the waist) may produce valid estimates of steps, but distance output is inaccurate [65]. Another study found that older Fitbit monitors underestimated energy expenditure [66]. Little is known about newer, wrist-worn monitors or how monitors may differ (both from other commercial monitors and compared to gold standard measures).

Limitations

As a content analysis, this project was by definition preliminary and exploratory. Thus, our conclusions are tentative and require further study. In particular, our coding related to theory-based recommendations and our designation of specific techniques as potentially more effective than others are intended to be first steps towards formal tests as to the true impact of various recommendations or techniques. Only research with human subjects—from small qualitative investigations to large-scale randomized trials—can investigate hypotheses related to feasibility, acceptability, and effectiveness.

The systems tested here were those that measure continuous lifestyle activity that were available for purchase in late 2013. Monitors that had been discontinued (eg, Motorola MOTOACTV and Larklife) could not be tested, nor could a large number of monitors expected to be released in 2014. Follow-up tests should be conducted to include these newer monitors and compare them to earlier models. Also, we did not include monitors that focused specifically on bouts of physical activity (eg, heart rate monitors by Garmin, Polar, Mio) or mobile phone apps that measured activity using GPS or accelerometry within the phone (eg, RunKeeper, phone-based pedometer apps). Our focus on products compatible with Apple iOS, which occurred for practical reasons, may have also led to missing some monitors only available for Android devices if they were not also listed on Amazon or CNET lists. To represent the full range of available options and for use in possible future decision aids, further testing of all these monitoring systems will be necessary.

We chose to use the latest and broadest taxonomy available, which likely contributed to the greater number of techniques found in these systems. Many of the techniques in the larger taxonomy are not used in physical activity intervention (eg, many of the associations techniques are more appropriate for addiction-related interventions) and likely should not be included in activity monitor apps. It is also possible that some techniques are counterproductive or only productive in conjunction with specific other techniques. Even otherwise appropriate techniques, such as behavioral practice, may be unnecessary when the activity being promoted is an activity of daily living like walking. Which techniques are most efficacious is, of course, an empirical question not yet answered.

Finally, coding of ever-changing apps is quite difficult. Some interrater disagreement occurred because only one of the 3 testers engaged in a behavior that triggered use of a specific technique. As all monitors were tested using personal computers and iOS mobile devices, the experiences of Android users may differ from our experiences. Regular app updates also led to differential coding. Although we updated our results prior to publication, it is likely that more techniques will be included across the 13 systems and new systems will be available in the near future.

Dr Lyons is supported by a research career development award (K12HD052023: Building Interdisciplinary Research Careers in Women’s Health Program) from the Office of Research on Women’s Health, the Office of the Director, the National Institute of Allergy and Infectious Diseases, and the Eunice Kennedy Shriver National Institute of Child Health and Human Development at the National Institutes of Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Dr Lyons is also supported by a Beginning Grant-in-Aid from the American Heart Association (13BGIA17110021). Dr Rowland is supported by a field-initiated grant (H133G080120) from the National Institute on Disability and Rehabilitation Research.

This study was conducted with the support of the Institute for Translational Sciences, supported in part by a Clinical and Translational Science Award (UL1TR000071) from the National Center for Advancing Translational Sciences, National Institutes of Health, and from the Sealy Center on Aging and Claude D Pepper Older Americans Independence Center (P30AG024832) at the University of Texas Medical Branch. This publication was also funded in part by the US Agency for Healthcare Research and Quality through Grant R24HS22134.

The authors would like to acknowledge the assistance of Eloisa Martinez, who served as third coder and provided additional research assistance with this project. Sedonia Holmes and Kelley Prevou also provided administrative and organizational assistance.