Achieving Digital Wellbeing Through Digital Self-control Tools: A Systematic Review and Meta-analysis

A common goal that is shared across a large proportion of the articles in our General Corpus (\(N = 29\), [11, 64, 80, 82, 88, 102, 103, 105, 106, 113, 114, 115, 127, 134, 139, 160, 161, 166, 167, 171, 172, 173, 177, 186, 198, 202, 205, 214, 224]) is the need of designing and evaluating novel interventions that improve people’s digital wellbeing in different contexts. Since digital wellbeing is a relatively recent topic, researchers are continuously investigating new strategies to assist users in self-regulating the usage of their devices and services, with the aim of improving user’s self-discipline and self-improvement [202]. Thirteen articles that propose novel interventions [11, 82, 88, 105, 106, 134, 139, 171, 172, 173, 177, 198, 202], in particular, motivate their DSCTs as a means to investigate new intervention strategies. Some of these articles investigate similar approaches. The “interaction restraint” proposed by Park et al. [172] and Kim et al. [106], for example, is an intervention mechanism that aims at degrading the interactivity of a device or of a specific service through an unnecessary task to be performed at the beginning of each usage session. Other strategies are instead solutions that have been originally applied for other problems, and that have been transposed to the digital wellbeing context. This is the case of the research of Pinder et al. [177], which seeks to understand whether approach biases for smartphone-addicted users can be reduced through cognitive bias modification, a nonconscious behavior change technology that has been already applied to other domains like healthy eating [176]. Similarly, Park et al. [173] motivate their work by questioning the usefulness of a virtual reality therapy in the field of digital wellness. Interestingly, Lyngs et al. [139] take a step back from what they call “external interventions,” e.g., timers and lockout mechanisms, to “shift the focus to how the internal mechanisms of an app can support user agency” (p. 1): redesigning mechanisms like autoplay features and infinite scrolling on mobile applications, indeed, might be more effective than any externally-imposed interventions, according to the authors.

The need of developing novel interventions is also typically associated with a specific user’s context and/or activity (\(N = 6\), [102, 103, 113, 114, 161, 214]). Through the Let’s FOCUS tool [103], for example, Kim et al. explore “how software-based interventions can be designed and deployed in colleges” (p. 63:2). Lock n’ Lol [113] and NUGU [114], instead, are specifically designed to help a group of students studying together.

Six other articles, instead, focus on developing novel intervention solutions that are generalizable [80, 127, 160, 166, 167, 186]. The work of Hill et al. [80], for example, analyzes contemporary mobile operating systems to uncover software features that researchers might require to design interventions for moderate Internet use on smartphones of different vendors. Other similar works aim at providing holistic approaches for coping with distractions [186] and overcoming limitations of existing digital wellbeing applications, e.g., excessive “pick and mix” of interventions [166], poor standardization [167], and single-device conceptualizations [160].

Rather than designing novel interventions, a common goal for studying and proposing DSCTs is to understand how interventions work (\(N = 14\), [33, 43, 47, 62, 81, 112, 117, 118, 144, 145, 149, 150, 159, 229]), e.g., to assess their effectiveness on changing users’ behaviors. Not surprisingly, such a goal derives from articles included in the Implemented Tools corpus, only. Seven articles [47, 144, 145, 149, 150, 159, 229], in particular, analyze the performances of commercially available DSCTs. Mark et al. [149, 150] use Freedom [63], a commercial DSCT with which users can set up blocks on their browser, to understand whether blocking distractions is an effective method to improve users’ digital wellbeing in real-world settings. Collins et al. [47], instead, investigate whether RecueTime [185] has an impact on the users’ awareness about their social network usage. Interestingly, the articles of Monge Roffarello and De Russis [159] and Lyngs et al. [144] review a large set of commercially available DSCTs for smartphones and PCs, by searching the Google and Apple app stores, as well as browser extension web stores. Through their first version of the Socialize tool, in particular, Monge Roffarello and De Russis assess “in-the-wild” the most popular existing intervention strategies implemented by mobile DSCTs, e.g., timers and lock-out mechanisms, with the aim of providing “an overall perspective of contemporary mobile apps for digital wellbeing, and identifying possible issues and opportunities to improve such solutions” (p. 1). Similarly, another work of Lyngs et al. [145] investigates the effectiveness of two popular interventions adopted for reducing Facebook usage on the PC, i.e., removing the newsfeed and goal reminders.

Other articles (\(N = 8\), [33, 43, 62, 81, 112, 116, 117, 118]) of the Implemented Tools corpus present novel DSCTs, but the underlying research goal is the need of understanding how the implemented strategies influence users’ behavior. HabitLab by Kovacks et al. [116, 117, 118], for instance, has been described and evaluated in different studies, e.g., to understand whether rotating browser interventions like timers and persuasive messages increases effectiveness with respect to using static interventions [117] or to understand whether the time saved on some unwanted apps is (or not) redirected to other unproductive activities [116]. FamilyLink, instead, has been proposed by Ko et al. [112] to investigate “how participatory parental mediation of smartphone usage by adolescents can overcome restrictive and unilateral mediation approaches” (p. 867). In a similar context, Hiniker et al. [81] developed their Coco’s Videos platform, thanks to which the same kids can plan their media consumption, to “investigate one possible alternative to today’s parental controls and move away from the traditionally authoritarian designs” (p. 10).

Another common research goal that characterizes the articles included in our General Corpus is to understand how people use their devices and services, including the same DSCTs (\(N = 19\), [16, 19, 65, 83, 101, 104, 108, 116, 132, 133, 140, 162, 165, 188, 189, 195, 204, 213, 223]). On the one hand, understanding how users benefit from using DSCTs and in what ways is fundamental to inspire future research in this field and develop better solutions for digital wellbeing, e.g., to enhance people’s productivity by means of effective feedback [108]. On the other hand, delving into how users use technology is a necessary step to lay the foundation for effective DSCTs.

To investigate how users use DSCTs (\(N=8\), [16, 104, 108, 116, 133, 188, 189, 223], Implemented Tools corpus), researchers adopt novel and/or existing implementations of tools for digital self-control. Kim et al. [104], for example, evaluate their PomodoroLock tool to explore how office workers would use the provided coercive strategies to cope with distractions. Similarly, Liu et al. [133] present an operating system widget to test “whether providing visual feedback regarding the duration of task suspension can improve primary task resumption” (p. 767). As reported by Rooksby et al. [189], investigating how users interact with DSCTs is also useful to understand which usage data might be of personal interest and value to the user. Providing users with better statistics, in particular, would allow them to gain real-time awareness about how they spend their time online, thus allowing an efficient logging and quantification of the time that is spent using devices like smartphones [16]. Stemming from such an assumption, the aim of the study of Whittaker et al. [223] is to analyze whether real-time awareness provided by the meTime tool improves user focus. In addition, some of the articles investigating DSCTs use to study the influence of these tools on usage patterns. The work of Kovacs et al. [116], for instance, extends the HabitLab tool through a mobile application, and it explores whether the time saved thanks to the DSCT is actually saved or if it is just redirected to other unproductive activities, e.g., usage sessions with mobile apps or websites that are not subjected to any intervention. Monge Roffarello and De Russis [188], instead, propose a DSCT targeting smartphone habits, i.e., recurrent usage patterns that are associated with stable contextual cues. The tool is able to detect such behaviors, and it can assist users to change the habits that are perceived as meaningless.

Research works that analyze technology use to extract and propose digital self-control interventions (\(N=11\), [19, 65, 83, 101, 132, 140, 162, 165, 195, 204, 213], Qualitative Discussions corpus), instead, are based on quantitative and qualitative studies, from surveys, e.g., [204], to mixed-method studies composed of semi-structured interviews and log analysis, e.g., [140]. These kind of analysis can help researchers and practitioners to turn their focus from “an overly-determined and prescribed view of technology engagement” to digital self-control tools that promote a more “conscious, mindful, intrinsically-motivated and perhaps even more ethical form of interaction” (Genova et al. [65], p. 1). With this underlying goal, Tran et al. [213] run their study to examine how smartphone users make sense of their habitual phone use, with the aim of finding effective ways to mitigate compulsive smartphone sessions. Similarly, Lukoff et al. [140] conducted their study to understand the smartphone usage patterns that leave people frustrated, while Aranda and Baig [19] explore the dynamics of excessive smartphone use. Besides smartphones, another set of studies explore behaviors change goals (and possible digital interventions) for social media usage, e.g., by exploring how and why people take breaks from social media [195]. Sleeper et al. [204], for example, use behavior-change goals to “explore how people view SNSs as impacting their lives, and to describe the range of goals participants have” (p. 1,058). According to the authors, such an analysis is fundamental to exploring DSCTs for social networks like Facebook, Instagram, and Twitter. Finally, some articles investigate how people use technology in specific contextual situations, e.g., in the presence of others at home [165] or in caregiving contexts [83], to propose solutions for digital self-control tailored for these specific use cases.

A challenge that characterizes the development of DSCTs regards the adaptability of the implemented interventions (\(N = 7\), [83, 105, 112, 132, 189, 214, 224]). As reported in the following, such an adaptability involves different aspects. Researchers [132, 189] highlight that interventions should adapt to the target device or service. This is particularly important since each device has its own type of usage. Differently from personal devices like smartphones, for instance, tablets can be shared with other members of the family [189]. Even usage sessions are intrinsically different between mobile devices, with smartphones that are used more often than tablets but with shorter usage sessions [84]. Furthermore, while portable devices are typically used for a variety of tasks, from chatting to watching films, other devices have a more specific type of usage, e.g., desktop PCs for working and smartwatches for tracking sport activities [160]. Besides adapting to different devices, other 4 articles [83, 105, 214, 224] underline that interventions should adapt to the current users’ needs and usage patterns. Indeed, since different users have different smartphone usage patterns, “it still remains a challenge how the goals [i.e., the interventions] needs to be appropriated to each user” (P03 [105], p. 16:23). Without taking into account the preferences of the user and their current needs, in particular, finding a tradeoff between helping the user and avoiding unnecessary overloads becomes a challenge [214, 224]: as reported by Hiniker et al. [83] in their study on smartphone usage while caring for children, device-resistance is not universal, and some people may not have any desire or intention to limit their time with mobile devices. Finally, another kind of adaptability regards the user itself [112]. Besides using devices in different ways, indeed, users themselves have different levels of physical capabilities and technology competences. Ko et al. [112], in particular, highlight this challenge in the evaluation of FamilyLink, a parental-control DSCT that involves both parents and children.

A challenge that is strictly related to adaptability is the need of implementing accurate interventions (\(N = 20\), [11, 16, 33, 47, 80, 82, 101, 104, 106, 108, 133, 144, 159, 160, 162, 165, 189, 195, 223, 224]). According to the analyzed articles, researchers should improve the capabilities of DSCTs to detect the contextual situations in which the user is using their devices, as well as the current usage intentions of the user [33, 80, 82, 101, 104, 108, 133, 144, 160, 162, 165, 195, 223, 224]. People, indeed, often work at irregular hours [223] by using the same device both for work and leisure [108]. As highlighted by Oduor et al. [165], in particular, “a key design challenge is that “work time” and “home time” are not necessarily distinct” (p. 1324). At the same time, the same applications and websites with which users interact daily have often a “neutral” nature that is difficult to classify [33, 104, 165]. YouTube, for instance, can be used either for entertainment or for educational purposes. As a result, exactly defining what the user is currently doing with their devices [160] and what are their current feelings [195] are challenges that need to be solved in order to deliver the right intervention at the right time, e.g., to avoid “novelty effects” [224]. This is also reflected on the need of avoiding bugs [47, 106, 159, 189]. For example, an intervention delivered at an inappropriate time, e.g., a lockout task for a “good” or “meaningful” use [106], incurs considerable inconvenience to users. The same happens for bugs related to erroneous data tracking: wrong statistics affect the accuracy of the DSCT, which in turns affects usability [159]. While detecting such kinds of bugs may be difficult, it is also an urgent challenge to increase the user’s trust in the tool [47], thus reducing the risk of tool abandonment [189]. Finally, implementing an accurate DSCT also means allowing users to understand the visualized statistics and the delivered interventions [11, 16, 47, 108, 189]. Such a challenge mainly emerges from the complexity of the current visualizations adopted by DSCTs, which may result difficult to understand by novice users who are not used to interpreting charts [16]. Moreover, device usage statistics are often used in a retrospective manner, with users that may experience difficulties in remembering the exact purpose of their application usage [108]. Consequently, without accompanying statistics with sufficient details, e.g., contextual information, users cannot gain “a true understanding of their work patterns from the data [47]” (p. 376), and they might not understand why an intervention has been delivered. To make judgments about productivity and overuse, users, therefore, need “the details of what they were doing with their devices [189]” (p. 292).

Researchers agree that a challenge to design and implement adaptable and accurate DSCTs is to overcome technical constraints (\(N = 18\), [62, 80, 88, 101, 103, 104, 108, 115, 145, 149, 159, 160, 167, 189, 198, 214, 223, 224]). By investigating how to reduce the environmental impact of the Internet through moderate use, for instance, Hill et al. [80] highlight that manipulating Internet traffic is not straightforward due to the current restrictions imposed by contemporary operating systems. More specifically, mobile operating systems have several restraints to preserve energy [224]. There are also differences between the different mobile operating systems available on today’s market. The majority of DSCTs targeting mobile devices, indeed, are developed for Android-based smartphones [144], with different articles in our Implemented Tools corpus [62, 103, 149, 159] that highlight difficulties in developing the iOS counterpart. The Apple operating system, indeed, is typically more restrictive than Android, since it does not allow developers to implement fundamental operations that characterize the development of DSCTs, e.g., accessing sensitive information such as phone usage statistics [159], blocking the usage of other apps [103], muting notifications [103], launching background operations [103], and tracking Internet usage [80]. Moreover, even the operating system of Google is characterized by different versions that are continuously evolving,⁹ with each version that introduces new features and/or restrictions. DSCTs are particularly vulnerable to regular updates [115] and, at the same time, to software aging [198], i.e., the natural degradation of software capabilities. As these tools typically run on the same platforms that they aim at limiting [198], indeed, changes in the hosting platforms may impact their functionality (see also the “hostile designs,” through which the systems under modification become intentionally inhospitable [2, 59]). Without the possibility of developing stable and cross-platform solutions, these problems influence the deployment and the evaluation of DSCTs [62, 189]. Furthermore, technical constraints also hinder the possibility of developing multi-device DSCTs [88, 145, 160, 223]. As already reported in Section 3.1, targeting single devices is not sufficient to capture all the nuances of people’s digital wellbeing [125], since many people work on their PC but spend large parts of their day online using phones and tablets [223].

Minimizing attrition is another fundamental challenge that is mentioned in the analyzed articles (\(N = 6\), [47, 82, 117, 118, 171, 198]). Attrition can be defined as the tendency of users to abandon a given technology after having used it for some time. DSCTs are particularly affected by this problem, since they “may over-restrict users in attempts to compensate for a lack of human accountability” (Schwartz et al. [198], p. 1), and, consequently, they may be abandoned by users because their settings do not match users’ expectations or are too aggressive [118, 198]. Therefore, motivating the user to keep using a DSCT over a long period of time is fundamental: according to numerous behavioral theories [178], indeed, the key aspect to change a behavior and forming a new habit is repetition, especially at the beginning of the process when the new behavior is starting to emerge. In the context of DSCTs, motivation is also important since using devices like smartphones is a socially situated practice, and it is therefore hard for an individual “to change a behavior when surrounded by social cues that run counter to his or her goals [82]” (p. 4,755). Furthermore, as described by Collins et al. [47], motivating users is necessary to avoid an “acceptance epiphany,” i.e., when users distractedly use the statistics displayed by the DSCT to accept their current behavior rather than investing effort to change it. Some articles on our Implemented Tools corpus explicitly try to address the attrition challenge. Schwartz et al. [198] outline “a new direction of research that is specifically focused on reducing risk of user abandonment of a given digital self-control tool” (p. 1). Their article provides, in particular, a theoretical contribution of two principles that relate risks and effectiveness in DSCTs. Similarly, Kovacs et al. investigate the influence of periodically changing the intervention [117] and/or its difficulty [118] on users’ attrition rate, while the Aiki tool of Inie and Lungu [88] has been designed to avoid adverse effects of blocking potentially necessary breaks.

Another challenge that is mentioned in the analyzed articles is related to the evaluation of DSCTs (\(N = 12\), [33, 81, 116, 117, 118, 140, 145, 166, 167, 171, 198, 224]). Researchers, in particular, are aware that the effectiveness of DSCTs should be measured through long-term evaluations [81, 116, 117, 171, 198], preferably with follow-up periods [33, 171], since behavior change is a long and complex process: new habits, for instance, may take time to become established [178], and experiments [121] demonstrate that a new behavior needs from a few weeks to almost a year of repetition to become automatic, with substantial variation at the individual level. While researchers are starting to recognize the need of evaluations that investigate the effectiveness of DSCTs in the long term, Section 6.1 shows that such a challenge is still underexplored. This may cast doubts on the promising results that typically accompany the studies included in our corpus (see Section 6.2). According to Kovacs et al. [116], in particular, while behavior change systems are typically effective during experiments, more effort should be put into evaluating “whether behavior change systems remain effective outside studies and bring longitudinal behavioral change” (p. 3). Besides long-term evaluations, researchers also point to the need of adopting better measures [33, 118, 140, 145, 167]. According to Lyngs et al. [145], for instance, most studies about Facebook overuse rely on self-reported measures, but prior work [57, 169] already demonstrated that self-report often does not correspond with the actual use of digital devices. More explicitly, Okeke et al. [167] state that “self-reports capture what people say they do but not what they actually do, which is a major challenge in behavior change” (p. 1). Furthermore, the log of simple data like time spent or visits may not be sufficient to measure the effectiveness of DSCTs [224], but, at the same time, extracting more complex measures, e.g., users’s sense of meaningfulness, is undoubtedly harder [140]. Finally, 2 articles in our corpus [145, 166] highlight that a factor that influence the evaluation of DSCTs is the fact that these tools typically combine a variety of different interventions at the same time, making it difficult to interpret causality [145] and understand the specific effects of a particular behavioral technique.

The last challenge that emerges from our analysis concerns the business model of contemporary tech companies (\(N=8\), [19, 139, 140, 144, 160, 204, 224]). In the “attention economy [54]”, indeed, tech companies compete to grab users’ attention and maximize advertisement revenue, by designing technologies that lead users into continuous and frequent usage. While Section 4.2 shows that researchers already investigated possible solutions to mitigate this one-sided view, in which technology is primarily designed to satisfy companies’ revenue, dealing with (and possibly change) this business model is undoubtedly one of the most critical challenge that characterizes the field of interventions for digital self-control. DSCTs, indeed, may hinder business profitability. Widdicks and Pargman [224], in particular, pose the following open question: “how do companies create Internet services that encourage users to “do business” with them, yet promote moderate Internet use and ensure users’ do not become ’hooked’ on their service?” (p. 6). There is therefore the need of exploring what could potentially motivate key stakeholders to support DSCTs [139]. While designing for meaningfulness may result in a lower user engagement in the short term, for example, it could also increase user loyalty in the long term [140]. All in all, designing technology that is also able to disincentive itself is, at the same time, “an open challenge and an ethical responsibility that HCI researcher should explore to counter problems like technology unwanted (over)use” (Monge Roffarello and De Russis [160], (p. 12).

By analyzing all the research works in our General Corpus, we found that only 12 articles out of the 51 articles describing studies conducted with human subjects (\(23.52\%\), [64, 83, 139, 140, 161, 162, 165, 171, 173, 189, 198, 204]) mention a report of ethical approval for the conducted studies, e.g., through a revision of an Institutional Review Board (IRB). It is worth noting that most universities require IRB protocols as a prerequisite to human subjects works and that researchers may use an IRB without explicitly mentioning it in their publications. Given the sensitive nature of topics like digital wellbeing and DSCTs, however, we believe that this kind of information should be always included.

Autonomy in Studies on Digital Self-control. According to Beauchamp and Childress [27], the principle of autonomy refers to the respect for people’s decision-making ability. A large number of articles in our corpus (\(N = 41\), [33, 47, 62, 65, 80, 81, 88, 103, 104, 105, 106, 108, 114, 116, 117, 127, 133, 139, 140, 144, 145, 150, 159, 160, 161, 162, 165, 166, 167, 167, 171, 172, 173, 186, 195, 198, 205, 213, 214, 223, 229]) address autonomy in describing the conducted studies from different perspectives. Overall, 15 articles [33, 116, 117, 127, 133, 149, 150, 161, 162, 171, 173, 186, 198, 213, 214] mention the usage of an informed consent to be accepted by users before starting the collection of their usage data. Such informed consents are also used to describe the study protocol [116], e.g., which data are collected, and to assure users that “their data would be kept anonymous” (P13 [149], p. 4). Other 15 articles take into account users’ autonomy through the application or the discussion of privacy protecting measures [33, 80, 112, 114, 117, 145, 159, 165, 166, 167, 171, 186, 189, 198, 223], e.g., data anonymization [33, 145, 149, 159, 186, 214, 223], with the aim of “respecting and ensuring confidential treatment of peoples’ personal information” (Thieme et al. [210], p.34:26). In the Evaluated Tools corpus, for instance, we found that Riedl et al. [186] obfuscate usage data that are related to websites that have not been explicitly added on lists of blocked sites by the user. Similarly, the browser extension developed by Lyngs et al. [145] collects data by using anonymous identifiers, and it does not store any “identifying information about the actual content engaged with” (p. 4). Rather than anonymizing data, Whittaker et al. [223] allow participants to review their collected logfiles before submitting them, with the possibility of removing entries that they do not wish to share. Similarly, Borghouts et al. [33] allow participants to delete or adapt “any sensitive or confidential information in their data, such as application and website names” (p. 32:17). NUGU by Ko et al. [114], instead, shares data by aggregating usage statistics among groups of people. Finally, 19 articles [47, 62, 81, 88, 104, 105, 106, 108, 114, 145, 160, 165, 171, 172, 173, 189, 195, 205, 229] describe the adoption of some inclusion criteria to recruit participants in their studies. Kim et al. [105], for example, evaluated their GoalKeeper DSCT by recruiting users who “considered themselves as excessive smartphone users and who were willing to reduce their use” (p. 16:10). Similar criteria are described in several other articles included in the Evaluated Tools corpus [47, 62, 81, 104, 106, 108, 114, 171, 172, 173, 205, 229].

Only 6 articles out of the 62 included in the General Corpus (\(0.08\%\)) explicitly discuss or propose ethical guidelines related to the field of interventions for digital self-control., e.g., to inform the design of the implemented DSCTs and avoid problems like deception and lack of transparency [198]. The majority of these articles (N = 4, [64, 80, 165, 224]), in particular, are included in the Qualitative Discussions corpus. In proposing hypothetical interventions to improve smartphone usage in social contexts, Genç and Coskun [64] acknowledge that the envisioned solutions may not be welcomed by users, as they may create unintended outcomes or violating users’ privacy. Consequently, the authors highlight the need of “learning more about users’ reactions towards the solutions generated to identify which of these is the most suitable ones for a specific context and a specific user group” (p. 10). As users perceive privacy as an important aspect to be considered when using DSCTs [112, 117, 159, 189], Okeke et al. [166] consider privacy while developing the proposed interventions. Their Good Vibrations tool, in particular, uses low-priority notifications that are not visible when the phone screen is locked, thus protecting users’ privacy from prying eyes. Also, Oduor et al. [165] elaborate on possible privacy issues that could derive from their interventions for limiting smartphone use at home, while Hill et al. [80] and Widdicks and Pargman [224] state that tracking and manipulating Internet access, e.g., to limit its usage, requires new policies and regulation, e.g., through additions to the European General Data Protection Regulation (GDPR). Given the importance of reducing Internet usage for environmental reasons, in particular, Widdicks and Pargman [224] envision an idealized world in which service providers “design applications and services that help users limit their Internet use [...], and create responsible, ethical and sustainable designs—utilizing these aspects as a deliberative selling point to keep customers and profit” (p. 4). The same authors highlight the role of new policies as a means to convince businessess that may be reluctant to introduce moderate Internet use within the design of their technologies. As some businesses may be reluctant to introduce moderate Internet use within the design of their technologies, researchers could engage with policy.

Justice in Digital Self-control Tools. The ethical principle of justice focuses on the fair distribution of benefits, risks, and costs [191]. In the context of interventions for digital self-control, the articles that include such a principle (\(N = 20\), [19, 81, 82, 83, 103, 108, 112, 113, 133, 139, 140, 160, 165, 166, 171, 189, 205, 213, 224, 229]) discuss the need of developing/evaluating DSCTs for/with wide and diverse populations. The majority of these articles [81, 83, 103, 108, 112, 113, 133, 139, 140, 160, 165, 171, 205, 213, 224, 229] cite such a need as a limitation or a future work to be explored, since the conducted studies often involve participants from the same geographical area [160, 213], e.g., Korea [113] or the Seattle metropolitan area [83], and with similar characteristics, e.g., students [112, 171] and workers [108]. This limits the generalizability of the retrieved findings [103, 108, 113, 140], which require additional research to be consolidated. Patterns of phone use, indeed, are likely to differ between different communities of users [83]. For this reason, in discussing the evaluation of their FamilyLink tool, Ko et al. [112] state that “additional research in different schools and cultural environments comprising students and parents from various socio-economic backgrounds is required” (p. 876). Liu et al. [133], instead, argue that designers should consider gender as a factor in persuasive design, since it “is important to match individual differences to maximize the motivation effect” (p. 775). Only 3 articles [19, 82, 166] describe an effort to include different populations in their studies. Hiniker et al. [82] state that their MyTime tool was evaluated with participants representing 20 different states and 4 races and ethnicities. Okeke et al. [166] describe an experiment that was conducted “with a diverse pool of participants, which is an advantage when compared to traditional non-online behavior change experiments that have been overwhelmingly conducted with participants from Western, Educated, Industrialized, Rich, and Democratic (WEIRD) societies” (p. 4:9). Finally, the study of Aranda and Baig [19] includes participants coming from different countries, from Swiss to Argentina.

Beneficence and Non-maleficence in Digital Self-control Tools. The underlying motivation of all the research works included in the articles under analysis is, obviously, to positively contribute to users’ digital wellbeing. The principle of beneficence, however, is not only related to “doing good,” but also to “doing it well” [23], i.e., to balance benefits against risks and costs with the aim of preventing harm. In this respect, only 9 articles in our corpus [64, 65, 105, 159, 160, 165, 198, 204, 224] explicitly declare to pursue a beneficence principle. For what concerns the implemented DSCTs (Implemented Tools corpus), for example, the Time Sidekick prototype [198] has been designed to meet principles of voluntariness, privacy, and non-deception, while GoalKeeper [105] focuses on the formation of “positive-long term goals related to various application domains such as entertainment, enlightenment, and sociality” (p. 16:5). Similarly, Monge Roffarello and De Russis [159] argue that DSCTs should promote the formation of new “healthy” habits, e.g., by increasing the usage of positive reinforcement techniques. Articles in the Qualitative Discussions corpus, instead, suggest researchers to adopt beneficence principles before designing interventions, e.g., to positively affect users [224] and to“design from a positive and hopeful perspective on engagement” (Genova et al. [65], p. 6).

Dually, the principle of non-maleficence can be seen as the explicit intention of not causing harm through the adopted strategies and in the conducted evaluations. Only a small subset of articles (\(N = 11\), [64, 80, 88, 103, 108, 112, 113, 133, 165, 173, 189]) explicitly include such a principle. As acknowledged by some researchers in our Implemented Tools corpus [103, 133, 173], interventions adopted by DSCTs may have a negative impact on users’ wellbeing, thus motivating the need to carefully considering a non-maleficence principle in the design process. This means supporting more abstract representations that mitigate the privacy problem [112], reducing the frequency of feedback exposure [108], and avoiding too invasive and distractive reminders [113]. According to the researchers, in particular, using “punishment” methods has a double-bladed effect, since “it has higher effectiveness but also creates higher pressure on users” (P32 [133], p. 774). In discussing their virtual reality therapy for online-gaming addiction, Park et al. [173] argue that there could be ethical concerns surrounding the use of aversive stimuli, by highlighting the need to discuss the intensity of these interventions, since they could create “aversive stimuli that have the potential of provoking anxiety and defensive aggression” (p. 106). Kim et al. [103], instead, state that strict interventions like locking smartphones may infringe users’ autonomy, i.e., the state of being independent or self-governing, and argue that a direct social competition may result in a negative outcome. To mitigate these problems, the authors allow the users of their Let’s FOCUS tool, i.e., a DSCT with which students can share a “virtual room” with specific usage blocks, to leave a room at any time, and to self-organize social support groups. Furthermore, the authors abstract limiting records rather than displaying exact limiting records. As reported by some studies in the Evaluated Tools corpus [88, 189], the principle of non-maleficence can also be pursued while recruiting participants [88], as well as while collecting and analyzing data [189]. To avoid harm, in particular, the recruiting process of the study of Park et al. [173] involves the presence of a trained psychiatrist that screened participants through a structured clinical interview. Regarding data collection and analysis, instead, only the work of Rooksby et al. [189] report not having logged a device for ethical reason, since the user was using it while working with data about vulnerable people. Also, some articles in the Qualitative Discussions corpus [64, 80, 165] provide concrete suggestions to take into account the non-maleficence principle in the digital wellbeing context, from taking into account possible unintended outcomes of DSCTs [64] to avoiding privacy-critical designs [165].

Figure 6 summarizes the processes followed by researchers to recruit participants for the 43 studies included in our analysis. The majority of the studies (\(N = 36\)) include the description of a single procedure to recruit participants, ranging from campus-wide campaigns to recruiting on Amazon’s Mechanical Turk, while 7 other studies mention the adoption of 2 of these procedures at the same time.

As Figure 6 shows, the most common recruiting method in DSCTs studies (\(N = 12\), [102, 103, 106, 108, 113, 145, 161, 173], 2 studies in [223], and 2 studies in [223]) is a campus-wide campaign, e.g., by means of promotional posters and announcements in the researchers’ universities [47, 103, 113, 161]. Common strategies also include online posts (\(N = 10\), [33, 47, 88, 104, 105, 108, 114, 145, 171, 229]), e.g., on social networks [108, 145], the usage of mailing lists (\(N = 9\), [88, 108, 112, 145, 149, 150, 159] and 2 studies in [33]), private messages to the researchers’ social circles (\(N = 4\), [62, 127, 133, 159, 205]), and the recruitment of users that were already using an existing DSCT (\(N = 7\), [116] and 3 studies included in [117] and [118], respectively).

Overall, the median number of recruited participants is 36, with a minimum of 4 involved users [186] and a maximum of 11,700 participants in the study of Löchtefeld et al. [134], who analyze the data of all the users who installed the AppDetox app from February to October, 2013. By analyzing the 26 studies that report age information, we can see that the users involved in DSCTs studies are on average 26.23 years old (\(SD = 4.14\)). As shown in Figure 7, they are most often students (\(N = 25\), [33, 103, 104, 105, 106, 108, 113, 114, 133, 145, 159, 161, 171, 172, 173, 177, 186, 189, 205, 214], the 3 studies in [47], and the 2 studies in [223]). Together with the most common recruiting method reported above, i.e., campus-wide campaigns, this highlights a strong selection bias towards young university students. Another recurrent occupation for participants in DSCTs studies is office workers (\(N = 11\), [47, 88, 108, 149, 159, 186, 205, 214, 223] and 2 studies in [33]). Least common occupations are researchers (\(N = 2\), [47, 214]) and a generic “parents and children” (\(N=2\), [81, 112]). Therefore, we can identify another selection bias. Our findings, indeed, highlight that DSCTs are nearly always evaluated with technology savvy users that use devices like PCs and laptops every day for studying and working.

To evaluate a DSCT, researchers collect different measures related to the tool usage, its influence on usage patterns, as well as its influence on users’ character traits and feelings. Figure 8 shows the measures, i.e., the logged and/or collected data, that are mentioned in at least 4 of the 43 studies included in our corpus.

To collect the aforementioned measures, researchers adopt several instruments, i.e., state-of-the-art scales and evaluation tools like questionnaires and interviews. Table 9 reports all the instruments that are mentioned in the 37 articles of the Evaluated Tools corpus.

The most common instruments, mentioned in at least 2 articles, are the following:

The median duration of a study evaluating a DSCT is 21 days, with a minimum of 1 day and a maximum of 528 days. Only 5 studies have lasted more than two months: 2 experiments described in [118], and the studies in [116, 134, 229]. All these studies are based on existing DSCTs, be they research artifacts or commercially available solutions, with researchers that had the opportunity to exploit data coming from users who had already installed the tool in the past.

Figure 9 summarizes the designs and the main characteristics of the 43 studies included in the Evaluated Tools corpus. Overall, 26 studies (those included in [33, 47, 81, 82, 88, 103, 105, 106, 112, 113, 116, 117, 127, 134, 149, 159, 161, 172, 186, 189, 205, 214, 223, 229] and 2 experiments in [118]) follow a within-subject design, in which all participants can use all the functionality of the evaluated DSCTs, i.e., they are exposed to the same interventions. Such evaluations include controlled field studies [47, 81, 82, 105, 106, 112, 116, 117, 118, 149, 159, 186, 205, 214, 223], field deployments [33, 47, 88, 103, 113, 118, 127, 134, 172, 189, 229], and lab studies [161]. During within-subject controlled field studies, researchers maintain some degree of control over the DSCT. Mainly, this is related to the possibility of deciding when to activate the provided interventions [33, 81, 82, 105, 106, 112, 149, 159, 186, 205, 214, 223]. This allows researchers to collect baseline data, i.e., how users behave without the help of the DSCT, and intervention data, i.e., how users interact with their technological sources when using the DSCT. In some cases, researchers even control the kind of interventions that are delivered to the user (see the “rotating” interventions of HabitLab [117]), as well as the frequency of the interventions [47, 116], i.e., how many times on average an intervention like a persuasive message is delivered to the user. Through within-subject field deployments, instead, researchers recruit participants to install and freely use a DSCT for a given amount of time, without enforcing any baseline periods nor changes in the DSCT functioning. Such field deployments are used to test novel DSCTs [33, 88, 103, 113, 118, 127, 134, 172, 189], as well as to evaluate commercial DSCTs (see the third study carried out by Collins et al. [47], who interviewed RescueTime users, and the study of Zhou et al. [229], who evaluated an existing intervention in an online gaming platform). Only Morley et al. [161] report on a within-subject lab study. The authors present a design of a virtual reality user experience to evaluate users’ smartphone and driving behaviors. During the 1-day lab study, participants interact with their smartphones while driving a virtual car, and they experience (virtual) crash events that should serve as a deterrent for smartphone use while driving.

Another set of articles include studies that follow a between-subject design (\(N = 17\), [33, 62, 104, 108, 114, 118, 133, 145, 166, 171, 173, 177, 223], 2 studies in [117], and 2 studies in [47]). Differently from within-subject studies, between-subject evaluations have two or more groups of subjects each being tested by a different study condition. While within-subject designs typically require fewer participants and are cheaper to run, between-subjects reduce learning effects and transfer of knowledge across the different tested conditions. As reported in Table 8, between-subject evaluations include controlled field studies [33, 47, 62, 104, 108, 114, 117, 118, 145, 166, 171, 223], lab studies [133, 177], and prospective trials [173]. Researchers use controlled field studies that follow a between-subject design to test, in-the-wild, different variations of the implemented DSCTs. Such variations can be small, e.g., asking users to provide daily retrospective estimations at noon or midnight [47], or they may involve completely different intervention strategies (see Lyngs et al. [145], who tested two different intervention strategies against Facebook overuse, i.e., goal advancement and removing the newsfeed). Between-subject lab studies, instead, are used both to assess intervention variations [133] or to evaluate the effects of an intervention strategy with respect to a control group [177]. A between-subject design is also followed by Park et al. [173], who carried out a 4-week prospective trial to compare their virtual reality therapy for an online gaming addiction with a state-of-the-art psychological treatment named cognitive behavior therapy [107].

All in all, Table 8 shows that the most common type of evaluation adopted by researchers is a 21-days controlled field study that follow a within-subject design [81, 106, 112, 159, 186, 205, 214]. During such an evaluation, the DSCT is deployed on participants’ smartphones, PCs, or both, and usage data are typically collected for 1 initial week of baseline, i.e., 7 days during which the tool is “transparent” to the participants, and for the subsequent 2 weeks of intervention, i.e., 14 days during which participants can use all the functionality of the DSCT. By comparing usage statistics without and with the DSCT, researchers aim at analyzing the influence of the tool on technology usage. Despite common, we argue that such a design is not sufficient to accurately understand the effectiveness of DSCTs. The same researchers, indeed, agree on the need of longer-term evaluations, since behavior change is a long and complex process (see Section 5). Furthermore, despite the baseline phase, the lack of a “real” control group, i.e., a group of participants who do not receive the experimental treatment, may result in inaccurate evaluations due to problems like confirmation biases, i.e., the tendency for experimenters to give their expected outcome too much weight when measuring results [163]. As shown in Figure 9, the lack of a control group is also common among between-subject studies. Indeed, only 7 studies out of 35 (20%) compare participants using a DSCT with a group of people who do not use any interventions [33, 104, 133, 145, 166, 173, 177]. We can conclude that such a shortcoming profoundly characterizes how contemporary DSCTs are evaluated.

With a few notable exceptions, e.g., the evaluations of HabitLab [116] and AppDetox [134], Table 8 also demonstrate that short-term studies characterize the majority of the articles included in our corpus, independently of the adopted study design. A very limited number of researchers try to overcome such a problem through follow-up interviews and surveys [145, 166, 173], e.g., 5 months after the end of the study [145]. Unfortunately, this does not provide a complete and accurate assessment of the DSCTs long-term effects, since self-report is typically different than actual use of digital devices [57, 169]. Moreover, only 4 studies [108, 145, 166, 205] include (and monitor) a withdrawal phase, i.e., a phase during which the delivered interventions are removed. Consequently, it is hard to understand whether the effects of a DSCT would survive a break in the use of the tool.

We measured the effectiveness of a DSCT by analyzing its influence on the time spent on a target device, website, or mobile application. While time spent on a given online or mobile content does not perfectly correspond to attention or engagement behavior [149], e.g., as users can get distracted by “offline” events, prior work has generally accepted it as an effective estimation of users’ engagement with technology [223]. Furthermore, time spent is one of the most common metrics collected during the studies evaluating DSCTs (see Section 6.1.2). For DSCTs targeting the overall usage of a device, e.g., the first version of Socialize [159], we analyzed the effectiveness of the tool in reducing the overall time spent by the user with the device. For DSCTs targeting specific websites or mobile applications, e.g., the Facebook interventions explored by Lyngs et al. [145], we analyzed the effect of the tool on the overall time spent on the specific service, e.g., Facebook.

Given this definition of effectiveness, we re-analyzed the Evaluated Tools corpus to select the articles to be included in our meta-analysis. We applied, in particular, the following additional criteria:

Furthermore, we excluded articles that did not report sufficient statistical data, e.g., those reporting means without standard deviations. From the initial corpus, we selected 9 articles as eligible to be included in the meta-analysis, each of which included exactly one study investigating whether the evaluated DSCT reduced the time spent by the user on a given device, web site, or mobile application. Since only a small portion of these studies reported an explicit effect size, we decided to run the meta-analysis by leveraging means, standard deviations, and sample sizes. For each of the 9 studies, in particular, we extracted these information from both the “baseline phase” and the “intervention phase.” Overall, 6 studies (S1–6) [112, 114, 159, 171, 214, 223] explicitly reported all the needed information. For two studies (S7–8) [82, 105], instead, we estimated means and standard deviations from the reported figures, while for another study (S9) [145] we calculated these information from the provided dataset. When multiple studies were available, e.g., when a article compared different configurations of the same interventions, we included data for the best-performing intervention strategy, only.

To run the meta-analysis, we exploited R [183], a language and environment for statistical computing. We followed, in particular, the guide “Doing Meta-Analysis in R” by Harrer et al. [72]. Since the analyzed studies involved different types of users, we applied a Random-Effects-Model [32], by estimating the variance of the distribution of true effect sizes (\(\tau ^{2}\)) through the Hartung-Knapp-Sidik-Jonkman (HKSJ) method [89]. We used Hedges’ g as the primary estimate of effect size for each DSCT. Hedges’ g is defined as the difference between the two means (for baseline and intervention phases, respectively) divided by the pooled standard deviation. We used the Cohen’s guidelines [45] to interpret effect sizes. According to Cohen, \(g = 0.2\) suggests a small effect, \(g = 0.5\) suggests a medium effect, whereas \(g = 0.8\) denotes a large effect.

We assessed heterogeneity, i.e., the extent to which effect sizes vary within our meta-analysis, by studying Chochran’s Q [44] and Higgins and Thompson’s \(I^{2}\) [78]. The first one represents the difference between the observed effect sizes and the fixed-effect model estimate of the effect size. When Q is statistically significant, it indicates that the effect sizes are heterogeneous. The latter, instead, represents the percentage of variability in the effect sizes which is not caused by sampling error, and it is not sensitive to changes in the number of studies in the analyses. Furthermore, it can be interpreted according to specific guidelines. According to Higgins et al. [79], \(I^{2} = 25\%\) suggests low heterogeneity, \(I^{2} = 50\%\) suggests moderate heterogeneity, whereas \(I^{2} = 75\%\) denotes substantial heterogeneity. We also analyzed the confidence intervals of the effect sizes of each included study to spot any extreme effect sizes, i.e., outliers, and we performed an influence analysis by exploiting the Leave-One-Out method [219] and the Graphic Display of Heterogeneity (GOSH) analysis [168]. Our aim was to detect and remove influential cases, i.e., studies that exert a high influence on the overall effect size. Finally, we assessed publication bias by investigating small sample biases [32].

We began our analysis looking for outliers and influential cases in our sample. Outliers are studies with extreme effect sizes that may distort the overall effect size. Influential cases, instead, are studies that exert a high influence on the overall effect size. By detecting and eventually removing these studies, we ought to reduce between-study heterogeneity, thus retrieving an estimation of the pooled effect size that is robust enough, i.e., that it does not depend heavily on a small subset of studies.

We defined an outlier as a study whose confidence interval does not overlap with the confidence interval of the pooled effect size. We, therefore, conducted a first meta-analysis on all the identified studies by inspecting the confidence interval of each effect size, looking for studies reporting extremely small or large effects. Such an analysis identified the study of Foulonneau et al. (S8) [82] as an outlier with an extremely large effect size. The lower bound of its 95% confidence interval (\(g = 1.9789\)), indeed, was higher than the upper bound of the confidence interval of the pooled effect size (\(g = 1.3883\)).

After the outlier detection, we conducted a Leave-One-Out analysis [219] to spot influential cases. In this case, we recalculated the same meta-analysis 7 times, each time leaving out one of the 8 studies. Again, we found that the study of Foulonneau et al. (S8) [82] highly influenced the pooled effect size according to different metrics, e.g., the Cook’s distance and the covariance ratio. To further explore patterns of heterogeneity in our sample, we also conducted a GOSH analysis [168], through which the same meta-analysis model is fitted to all possible subsets of the included studies. Such an analysis, in particular, uses supervised machine learning algorithms to detect if the effect sizes in the sample are homogeneous or if there exist specific patterns, e.g., clusters of studies with different effect sizes. The GOSH analysis identified another study with an extreme effect size which might potentially contribute to cluster imbalance, i.e., the study of Park et al. (S6) [171]. The Baujat plot [24] of Figure 10 further confirms the findings of our outliers detection and influence analysis. The plot shows the contribution of each study to the overall heterogeneity (as measured by Cochran’s Q) on the horizontal axis, and its influence on the pooled effect size on the vertical axis. We can see that S8 overly contributes to the heterogeneity of the meta-analysis, while S6 has an extreme effect size.

We, therefore, removed S8 and S6 from the corpus of studies included in our final meta-analysis. The resulting sample (\(N = 7\), [105, 112, 114, 145, 159, 214, 223]) was characterized by a low heterogeneity. The Chochran’s Q value (4.56) was not statistically significant (\(p = 0.6017\)), and the Higgins and Thompson’s \(I^{2}\) was \(0.0\%\), with a 95% confidence interval ranging from \(0.0\%\) to \(61.6\%\).

Table 10 reports the results of the final meta-analysis conducted on the 7 identified studies, while Figure 11 visualize these results by means of a forest plot.

Before analyzing the results, we generated a contour-enhanced funnel plot [175] of the studies included in our meta-analysis in order to assess publication bias (Figure 12). As the chart shows, the studies are partly significant and non-significant, and they lie quite symmetrically around the pooled effect size. The overall pooled effect size across all the studies (Table 10), in particular, is \(g = 0.4734\), with a 95% confidence interval from 0.2657 to 0.6811 and a total of 255 participants. This means that the evaluated DSCTs had, on average, a small to medium effect on reducing the time spent by users on devices, web sites, and/or mobile applications, according to the Cohen’s criteria [45]. As devices such as smartphones and PCs have become an integral part of the daily lives of billions of people, such a relatively small effect of DSCTs may impact a very large population. Furthermore, the cost for reaching such an effectiveness incurs with the design and the implementation of the DSCTs, thereby overcoming the costs associated with delivering individual treatments [1].

Despite the positive aspects, we saw from Section 6.1 that DSCTs are primarily evaluated through short-term studies. The studies involved in our meta-analysis, for instance, had an average duration of 22.57 days (\(SD = 10.43\)). Therefore, the effect of DSCTs on users’ behaviors like time spent on digital devices and services still require a confirmation through longer-term studies, e.g., to understand whether the effectiveness of DSCTs remains constant or varies over time. Furthermore, little is known about whether the new behaviors that users learn by using contemporary DSCTs would survive a break in the use of the tools. A preliminary analysis of the small subset of articles [108, 145, 166] which includes a withdrawal phase, i.e., a phase during which interventions are removed, seems rather to suggest that these behaviors actually do not survive without the help of the tool. According to these studies, in particular, only some significant effects, mainly related to self-perception measures like perceived self-control [145], are likely to persist when the DSCT is removed, at least for the limited amount of time of the withdrawal phase. Quantitatively, instead, users gradually return to their old habits without interventions, with the time-spent metric that increases back to nearly the same level of the baseline phase [145, 166]. The reason for this effectiveness degradation can be found in the self-monitoring nature of contemporary DSCTs [159]: unless other strategies like habit-formation [143] are applied, a discontinued use of tools based on self-monitoring strategies like statistics and self-imposed timers makes a changed behavior to slowly return to its pre-intervention level [110].

While the meta-analysis reported in Section 6.2 shows that implemented DSCTs have a small to medium effect on changing user’s behavior, our work also highlights several gaps and challenges characterizing the design of the proposed interventions. Several works included in our analysis, e.g., [19, 159], in particular, relate the ability to exert control over digital devices to the concepts of self-control and self-regulation [144]. The final aim of developing a tool for digital self-control is precisely to assist users to take advantage of (and preferably improve) such an ability. Unfortunately, our analysis on the strategies adopted by contemporary DSCT implementations (Implemented Tools corpus, Section 4.1) shows that the way these tools support people’s self-regulation is typically based on self-tracking statistics and block/removal strategies, e.g., timers and lockout mechanisms. To be effective, these strategies require the continuous motivation of the user. More importantly, the fact that these interventions are (nearly) always “self-programmed” means that users alone are forced to understand what the causes of their digital wellbeing issues, with the sole help of statistics like screen-time. At the same time, users must also decide what is an appropriate strategy to intervene in their unwanted behaviors, e.g., by selecting a proper time threshold for a usage timer.

We argue that this kind of self-monitoring approaches are not sufficient to effectively improve the impact of digital technologies on people’s lives. In this respect, a recent work of Meier and Reinecke [153] suggests the need of going beyond solutions that use screen time as the sole indication of use. The authors suggest that researchers should further differentiate the usage of technology by taking into account the used device, including its features and the type/brand of the used applications, as well as the type of the interaction. In the context of digital wellbeing. The formative study of Monge Roffarello and De Russis [160], for example, demonstrates that users themselves are aware of the fact that distractions can come from any connected device. Using more than one device at the same time, in particular, is a common situation that can result in either positive or negative digital experiences depending on the underlying performed task [160]. Users may indeed have nuanced goals regarding the usage of their devices and services, e.g., to post more content on social networks rather than using them passively [140], or to avoid using smartphones during mealtimes [162]. We, therefore, call researchers to investigate more “intelligent” DSCTs that are able to analyze and learn from people’s multi-device usage patterns, e.g., through machine learning algorithms, with the aim of adapting interventions to various contextual aspects, from the used device to the underlying usage intention [105]. This would enable future DSCTs to provide a proactive support to define tailored interventions, by helping those users that do not recognize what are the problems that are negatively influencing their digital wellbeing, e.g., when self-perception diverges from actual use of devices [57, 169]. An example of a research-based tool that has started to pursue such a goal, the only one we found in our analysis on DSCTs strategies (RQ2, Section 4.1), is the second version of Socialize [188], which is able to detect and notify context-enriched smartphone habits in real-time, with users that can define personalizable just-in-time reminders to avoid such behaviors in the future. Section 5.1 highlights that adaptability could also be useful to reduce attrition. To balance effectiveness and low attrition, in particular, an interesting research direction proposed by Schwartz et al. [198] could be the investigation of continuously variable interventions, with DSCTs that are able to progressively increase or decrease the intervention intensity or change “on-the-fly” the adopted strategy by scaling-down strategies that seem to be at high risk of abandonment. Taking inspiration from the DSCTs analyzed in our study, for example, DSCTs could selectively remove parts of the Facebook newsfeed [145], use increasingly complex “interaction restraint” tasks [106, 172], or implement timers before instant blocks [159]. We claim that these solutions would also be useful to promote a step-by-step “learning” processes for their users, a fundamental aspect in the context of behavior change and digital wellbeing [160]: an adaptable DSCT, in particular, could progressively reduce (or increment) its degree of support based on user’s achievements, until the user acquires a sufficient level of independence, i.e., it is able to sustain the new behavior without the help of the tool.

Interestingly, our analysis on strategies also reveals that the interventions that researchers are proposing as future works (Section 4.2) are significantly different from those that have been implemented in the last years, thus highlighting a possible new research direction for the field of interventions for digital self-control. While contemporary DSCTs mainly focuses on blocking and/or mitigating interactions, in particular, several formative studies in our Qualitative Discussions corpus explicitly call researchers and developers to explore designs that promote meaningfulness experiences and alternatives, be they online or offline activities. Some notable exceptions of implemented DSCTs that partially follow such a principle are those that can spur the user to perform an alternative physical or virtual task (see Socialize [188] and Aiki [88], respectively), and those that aim at re-designing user interfaces and features of existing services, e.g., to reduce dark-patterns in mobile apps [115] or remove “addictive” features like the auto-play of the next video [81].

All in all, focusing on the design of positive and meaningful experiences, e.g., by integrating well-known principles like Slow Design [70] and Mindfulness Design [230], would surely improve the effectiveness of interventions for digital self-control. Unfortunately, re-designing interfaces and features of existing mobile apps or websites is surely more difficult than implementing timers and lockout mechanisms, especially due to technical limitations/restrictions imposed by vendors and providers [80]. While this problem can be circumvented by using alternative approaches (see the community-driven app modification framework proposed by Kollnig et al. [115], which is based on third-parties patches), we argue that more effort should be put into designing DSCTs that can make internal changes to application designs. Furthermore, we envision a future in which digital services promote users’ digital wellbeing by design, i.e., without the need for an “external” DSCT, with the aim of overcoming the inherent contradiction of designing for digital wellbeing in a business model which currently incentivizes frequent and continuous usage (see Section 7.3.2). This would necessarily require a change in perspective by contemporary tech companies, starting from providing developers with more transparent APIs [80] to finding alternative business models (see Section 7.3.2 for further discussions on this topic).

Grounding the design of behavior change technologies on well-established behavioral theories is fundamental to generate long-lasting results [178]. Yet, such a theoretical-grounded approach still needs to be further established. On the one hand, some systematic and meta-reviews have identified a general poor use of theory in research investigating digital interventions for behavior change [155, 156], and other reviews have shown that such a trend is common in commercially-available apps, too [48]. On the other hand, the number of HCI researchers that draw on behavioral sciences to inform the design of behavior change technologies is continuously growing [77, 178]. Such a contamination between HCI and behavioral science communities characterizes different domains, from supporting healthy diets [184] to promote physical activity [30]. Yet, the plethora of existing theories makes it difficult for HCI researchers to disentangle the right behavioral strategy to be pursued [178], and research findings mainly remain siloed between the two communities [77]. To mitigate such a problem, the article “Mind the Theoretical Gap: Interpreting, Using, and Developing Behavioral Theory in HCI Research” by Hekler et al. [77] provides HCI researchers with guidance on “interpreting, using, and developing behavioral theories” (p. 3,307). In their work, the authors highlight the problems that characterize the use of behavioral theories during the design and the evaluation of behavior change technologies, e.g., gaps between theories and concrete designs and lack of large-scale randomized trials. Furthermore, the authors provide several insightful suggestions to assist HCI researchers to ground their prototypes on behavioral science.

With the theoretical analysis reported in Section 4.3, we analyzed the extent to which drawing on behavioral science is common in the context of DSCTs, and, in particular, whether researchers that are proposing and/or designing tools for digital self-control are taking advantage of the valuable advice from previous studies connecting HCI and behavioral science, e.g., the work of Hecker et al. [77]. Our analysis shows that a large quantity of the analyzed articles (38 out of 62) do not mention any behavioral theory or construct. Despite not using a theoretical-grounded approach, however, these works often mix together different strategies that take inspiration from a known theory. For instance, while several of the DSCTs included in the Implemented Tools corpus allow users to set up conscious behavioral goals, only 3 articles [47, 82, 117] explicitly ground such an intervention in the goal setting theory [135, 136]. Besides goal-advancement features, Section 4.1 shows that a single DSCT often includes several other features, from block/removal to self-tracking. Picking and mixing different strategies without a common theoretical grounding is problematic, since it makes it difficult to understand the effects of particular behavioral theories or techniques [166]. As reported by Hekler et al. [77], moreover, picking only some constructs from a theory makes researchers loose the potential of the full conceptual framework for designing the system, and may lead to methodological flaws in interpreting the validity of the proposed DSCT, e.g., when the tool is inadvertently designed based on constructs that do not work independently but rather relay on other constructs that are not included in the design process.

Despite the advantages of grounding digital interventions on behavioral science, e.g., for achieving higher effectiveness [1], how to best utilize behavioral theories is however challenging. As reported by Hekler et al. [77] and Pindler et al. [178], this is mainly due to the fragmentation that characterizes behavioral science, with an overabundance of different theories, approaches, and techniques that result in redundant constructs differently named depending on the domain of origination [13, 154]. Consequently, many studies in behavior change research that claim to be based on theory do not provide sufficient evidence on how the theory relates to the proposed interventions [156]. Such a confusion also emerges from our DSCTs analysis. In Section 5, indeed, we found that the same intervention, e.g., removing the Facebook newsfeed, is sometimes viewed under different theoretical lenses, e.g., operant conditioning [206] and dual system theory [93, 207], thus making it difficult to understand the real effects of the adopted strategies and understand their long-term impact [178].

All in all, we can conclude that also research on digital self-control tools is subjected to a “theoretical gap [77]” that makes the selection of relevant strategies difficult [164]. To develop better DSCTs and overcome such a gap, we suggest a closer collaboration between HCI researchers and behavioral scientists, as called for by Heckler et al. [77] in the broader context of behavior change research. At the same time, we argue that an effective way to ground DSCTs into behavioral theory is to take advantage of the existing explanatory frameworks that aim at closing the described theoretical gap in behavior change research. The HAM [178] that has been recently proposed by Pinder et al. [178] is a prominent example. The framework synthesizes dual system theory, goal setting theory, and modern habit theory into an integrative model that simplifies the design of digital behavior change interventions able to form and break habits. Focusing on breaking unwanted digital habits and/or substituting them with alternative (wanted) behaviors, in particular, is a promising approach to support behavior change and ensuring its long-term effects [159]. As reported in our analysis, such an approach has been preliminary investigated in the context of smartphones (see the second version of the Socialize tool [188]).

Several studies and experiments have demonstrated that behavior change is a long and complex process. The work of Lally et al. [121], for instance, shows that a new behavior requires from a few weeks to almost a year of repetition to become automatic, with substantial variation at individual level. Similarly, Prochaska and Velicer [182] analyzed previous studies in the context of smoking cessation, showing that an efficient behavior change process may require several years of efforts. Stemming from such findings, Klasnja et al. [111] argued that HCI researchers should not use their short-term studies to make claims about participants changing their habits. Indeed, long-term follow-ups are needed to speculate on more permanent behavior change. More recently, Pinder et al. [178] stated that “short-term studies introducing participants to novel behaviors may provide evidence of a strong intention-behavior link that does not persist over time” (p. 15:42).

Ideally, the usage of a DSCT gives rise to a behavior change process that should lead people to improve their relationships with technology. Such a “learning” process will become particularly important (and explicit) if researchers will explore habit formation strategies in their DSCTs, as called for by recent reviews in the digital wellbeing context [144, 159]. Unfortunately, the articles included in our Evaluated Tools corpus fail to provide sufficient evidence for the long-term effectiveness of the evaluated DSCTs. As Section 6.1 shows, indeed, the median duration of a DSCT study is 21 days, only, with a very limited set of articles that include a follow-up interview and/or survey. Consequently, the majority of the small to medium effects reported in the article under analysis, e.g., those assessing the influence of DSCTs on time-spent (Section 6.2), should be interpreted as short-term results, only. Without accompanying short-term studies with long-term follow ups, in particular, we simply do not know if the described changes in participants’ behavior have become persistent [111] or if they are simply related to the fact of using a novel tool, with the reported effects that would therefore not survive a break in the use of the DSCT [145, 166]. As reported in Section 6.2.3, the small subset of studies [108, 145, 166] that monitor participants’ behavior during a withdrawal phase, i.e., a phase during which interventions are removed, seem rather to suggest that the effects of contemporary DSCTs are strongly related to an ongoing use of the tool. As such, we argue that, in general, researchers evaluating DSCTs should try to adopt long-term studies to make claims on the real impact of their tools on users’ behavior, preferably by using follow-up periods. Even some articles in our corpus, indeed, acknowledge this urgent challenge (see Section 5). As a reference, researchers might consider the work of Marcus et al. [147] in the context of physical activity, which suggests to evaluate interventions through year(s)-long follow-ups. However, as not all the DSCTs are designed to target the same goals (see Section 3.1), also the proper duration of an evaluation may vary. Indeed, while DSCTs to counter compulsive social network usage should ideally be used continuously, other DSCTs, e.g., those aiming at reducing distractions, may be useful in specific periods of the year, only, e.g., during exams periods for students.

Besides duration, another important factor to be considered when evaluating technology for behavior change is the experimental design [111, 178]. Klasnja et al. [111], for example, state that control groups and very large population samples are necessary to unambiguously demonstrate the effect of a technology on behavior change. In this respect, the authors highlight that large Randomized Control Trials (RCTs) could allow researchers to demonstrate whether a technology effectively promote people’s behavior change. RCTs, i.e., the gold standard of efficacy research in health sciences, have also been recently used for assessing digital behavior change interventions, e.g., in the context of promoting physical activity [85].

Unfortunately, the findings reported in Section 6 demonstrate that there is a general lack of control groups in DSCT studies, with a prevalence of within-subject experiments. While allowing participants to try all the functionality of a DSCT may be useful at early stages of development, e.g., to elicit their qualitative feedback [111], we argue that between-subject experiments with control groups are fundamental to assess the effectiveness of DSCTs and avoid problems like confirmation biases [163]. An example of a study included in our corpus that resembles a Randomized Control Trial is the one reported by Lyngs et al. [145] (see Figure 13 for the adopted procedure). Although the study involves a limited number of participants, especially if compared with traditional clinical RCTs, it compares two randomly-selected groups of participants subjected to different Facebook interventions, i.e., goal reminders and remove newsfeed, with a group of users that receive a “placebo” intervention, i.e., turning the Facebook background from light grey to white. In line with RCT guidelines [197], the authors also report on several surveys and interviews that have been conducted after each phases of the study, including a 5-month follow up that partially addresses the need for long-term evaluations.

Finally, the relatively low number of participants that characterize the analyzed studies¹², including the one of Lyngs et al. [145], leads to another important problem characterizing the evaluation of DSCTs. As reported in Section 6.1.1, we found that the recruiting processes described in the articles under analysis suffer from a strong selection bias towards young university students, and, more generally, towards technology savvy users that use devices like PCs and laptops every day, e.g., for studying or working. While considering these categories of users is reasonable, e.g., because they are more exposed to experience firsthand problems like technology overuse, excluding vulnerable categories like teenagers and non-tech savvy users, e.g., middle-aged people that do not use computers as a working tool, creates a gap in the digital wellbeing research context. The former, in particular, need to be involved in the digital wellbeing discourse, since their subjective wellbeing is nowadays undeniably affected, either positively [18] or negatively [220], by the usage of technology. The latter, instead, may experience the same negative consequences of overusing digital devices, e.g., for entertainment purposes, without possessing the necessary skills to proficiently use a DSCT. We, therefore, call researchers that are designing DSCTs to invest more effort in diversifying their target users and/or recruited participants. We argue that such a comprehensive approach is fundamental to assess whether and how the effects of a DSCT, e.g., those reported in Section 6.2, generalize to wider and diverse populations.

To conclude, we must acknowledge that conducting long-term evaluations of DSCTs by including diverse populations and control groups is not an easy task, e.g., because it requires recruiting several participants that are willing to test (sometimes restrictive) tools for several months. Furthermore, previous work in the HCI area [111, 191] rightly points out that these types of studies are hard to reconcile with novel and continuously evolving technologies. Apart from requiring a lot of effort, indeed, they often do not allow researchers to understand why an intervention is effective or not [111], therefore undermining the iterations that characterize the early stages of design. These problems apply to DSCTs as well. Digital wellbeing, indeed, is a recent and evolving research topic, and, as reported in our work, researchers in this area are continuously designing and iterating over a novel and existing interventions, with a lot of effort that goes into overcoming technical constraints (see Section 5.1).

Conducting studies to evaluate the effects of DSCTs on technology use requires first of all to define what the concept of effectiveness means in the context of digital wellbeing. As already discussed, the usage of a DSCT should ideally give rise to a behavior change process leading people to improve their relationships with technology. Obviously, such an improvement can be interpreted in many different ways, as technology may influence several aspects of individual’s life. In this respect, Section 6.1.2 shows that researchers assess DSCTs by collecting a variety of different measures, ranging from direct assessments of the tools’ characteristics to evaluations on how such tools influence users’ character traits and feelings, e.g., focus and stress. However, the lack of a standardized definition of effectiveness, as well as the variety of different adopted measures, makes it difficult to compare the effects of different DSCTs. Given the heterogeneous set of measures collected by the study analyzed in this work, for example, the only way to conduct our meta-analysis (Section 6.2) was to focus on the influence of the analyzed DSCTs on the time spent by the user on a target device, website, or mobile application. Despite the prevalence of this measure, however, we were only able to include 9 studies in the analysis, e.g., because several studies did not include sufficient statistical data. This requires researchers to provide (well-reported) statistics in their articles, thus allowing other researchers to confirm (or refuse) our findings with larger meta-analysis. To this end, we invite researchers to follow established guidelines and policies, like the ones proposed by the Transparent Statistics in Human–Computer Interaction initiative [8].

That being said, using time spent as a measure for people’s digital wellbeing may not be the right choice, and one of the reasons why effects of digital usage on wellbeing are very controversial [123, 124] is because we lack objective measures to gather strong evidence. As already discussed in the previous sections measures like screen time are too coarse, and they do not reflect the variety of goals and different kinds of tech usage of the users. Furthermore, providing users with an indication of their screen time, e.g., for self-regulation purposes, may in turn produce negative reactions [69], thus inducing users to stop using the DSCT [160]. Instead of quantifying their overall screen time, previous work demonstrate that users are more interested in understanding the details of their usage sessions [189]. Unfortunately, our work shows that time spent is still one of the most used metric to evaluate people’s digital wellbeing. There is therefore the urgent need for complementary strategies and tools for measuring people’s digital wellbeing, e.g., through alternative measures, so that researchers could further explore the relationship between wellbeign and technology use. In particular, we argue that HCI researchers should rethink the concept of effectiveness when evaluating their tools for digital self-control. In the broader context of technologies for health behavior change, Klasnja et al. [111] suggest to carry out studies that investigate “how and why a technology works or does not work” (p. 3,065). To this end, a possibility is to specifically focus on the constructs that characterize the behavior change strategy implemented by the evaluated tool [111, 191], e.g., by assessing the key mediator variables of a behavioral theory [109]. For those DSCTs like NUGU [114] and FamilyLink [112] that are informed by the social cognitive theory, for instance, an evaluation revealing an improvement on users’ self-efficacy may suggest that the implemented interventions behave as expected. Such an assessment can be performed by measuring the identified construct, e.g., self-efficacy, right before and after a period of time during which the user adopt the DSCT, e.g., as done by Monge Roffarello and De Russis [159] and Ko et al. [114] through the General Self-Efficacy Scale [92]. Clearly, before rethinking efficacy in the digital wellbeing context and assessing the key mediator variables of the implemented DSCTs, HCI researchers should first focus on closing the theoretical gap discussed in Section 7.2.1.

According to Pinder et al. [178], researchers designing digital interventions for behavior change have a “moral duty” to ensure that their interventions are ethical. Echoing recent reviews in similar domains, e.g., the work of Sanches et al. [191] on HCI and effective health and the work of Thieme et al. [210] on machine learning and mental health, we found a very limited interest in reporting ethical considerations in articles on DSCTs. Section 5.2, in particular, shows that a very limited set of articles explicitly acknowledge having followed ethical guidelines to propose, design, and/or evaluate their interventions for digital self-control. Although not reporting ethical considerations does not mean that ethics was not taken into account, it may however highlight that ethics is sometimes perceived as a secondary aspect. While digital wellbeing cannot be compared, at least in the first instance, with mental health and affective problems, this limited emphasis on ethical considerations is anyway problematic. Indeed, the last few years have seen a growing amount of research attention on the negative aspects of overusing devices like smartphones [129, 217], showing that technology overuse may be associated with negative effects on mental health [122] and social interaction [129]. While the debate is ongoing about whether technology overuse should be considered as a real addiction [123], this makes digital wellbeing, and consequently the design of DSCTs, a sensitive topic that may involve vulnerable people. Furthermore, DSCTs collect, by nature, sensitive information like visited websites and contextual information. As reported by Rooksby et al. [189], “data about everyday device use can reveal much about someone’s life, to the point that it raises privacy concerns” (p. 293). This opens up new questions related to the ethics of use of DSCTs that are still underexplored in the literature, e.g., what happens when workplaces add DSCTs to worker devices or when parents or schools impose DSCTs on children.

All in all, we argue that the first step to bring ethics in the digital wellbeing discourse is therefore to recognize the fact that developing DSCTs countering problematic technology use is, by its nature, a delicate operation that may require researchers to engage with vulnerable users and sensitive information. Recognizing this should encourage designers and researchers to explicitly take into account ethics in the design of interventions and, at the same time, to follow ethical research practices when evaluating DSCTs. In our opinion, designing ethical DSCTs requires researchers to pursue with more effort the core ethical principles of beneficence and non-maleficence, i.e., to balance benefits against risks and not causing harm. As acknowledged by some articles included in our corpus, indeed, interventions adopted by DSCTs may sometimes have a negative impact on users’ wellbeing. The high effectiveness of “punishment” methods, for instance, may in turn provoke high pressure on users [133], since aversive stimuli have the potential of provoking anxiety and defensive aggression [173]. Furthermore, “strict” interventions like locking smartphones may infringe users’ autonomy [103]. As advocated by Sanches et al. [191] in the context of HCI and affective health, we already discussed the need of interventions that go beyond “raw” self-monitoring statistics and lockout mechanisms, thus highlighting the need of more innovative DSCTs that help users understand [192] and reflect [193] on their usage patterns to promote positive behavior change (see Section 7.1). Additionally, we see promise in solutions that try to balance risk of abandonment, i.e., attrition, and effectiveness. According to the work of Schwartz et al. [198], “increasing the short-term efficacy of a DSCT is positively correlated with an increasing risk of DSCT abandonment” (p. 2), e.g., because the tool is perceived as too restrictive. This risk-reward tradeoff, which is considered to be a mental heuristic [179], needs to be solved in favor of the user, e.g., by selecting the lowest-risk interventions that still are effective. Furthermore, another important ethical principle that should be considered when designing DSCTs is to take into account individuals’ differences (justice), e.g., by designing solutions that can adapt to different users’ needs, as well as to different users’ ages, cultural backgrounds, and educations. DSCTs providing several intervention strategies and complex visualizations, for instance, may be suitable for a user with a technical background, but they might result too complex to be used by average users [16]. Furthermore, the ability to detect the current contextual situations of a user, e.g., to understand if she is currently working or not (see Section 5, Implementing Accurate Interventions), may depend on cultural differences in technology adoption [128] and (over)use [225].

As highlighted by our evaluation analysis (Section 6), the need of involving wider and diverse populations is a research practice that also applies to the evaluation of DSCTs, e.g., to avoid selection biases. Furthermore, we also argue that researchers should always support users’ anonymity, consent, and privacy when evaluating DSCTs. As such tools have the potential to collect sensitive information and to over-restrict users [199], indeed, researchers should use proper informed consent to make explicit how their interventions work before recruiting participants, especially when the implemented strategies are based on System 1 processes¹³ [178], e.g., as in [145]. In addition, we believe that researchers should always try to follow (and report) explicit ethical guidelines, e.g., validated by proper Institutional Review Boards, when evaluating DSCTs. In line with previous work on technology to counter affective health problems [191] and self-harm [174], this would require digital wellbeing researchers to take responsibility for recognizing ethical themes before carrying out their evaluations, e.g., to screen the recruited participants by taking into account whether and how the implemented DSCTs might provoke excessive harm on some categories of users.

Finally, some discussions included in a small subset of the analyzed articles (see the introductory part of Section 5.2) suggest that ethics in digital wellbeing is not only a responsibility of researchers designing and evaluating DSCTs, but it involves an increasingly important role for policies and regulations. As DSCTs track their users by collecting sensitive information like device usage, indeed, the risk that these tools are used against users, e.g., by violating their privacy, cannot be excluded. As suggested by Widdicks and Pargman [224], this highlights the need of improving and extending existing regulations to impose well-defined limits to the usage of DSCTs, e.g., through additions to the European GDPR. Furthermore, we also see the development of new policies in this field as an interesting (and at the same time useful) research direction. How much data are users willing to share with a DSCT in order to improve their digital habits? How much control are users willing to give to a DSCT? Developing policies able to answer these questions would be certainly useful to promote the design of ethical DSCTs that respect users’ needs and preferences.

Research works analyzed in this systematic review clearly highlight that an important factor that needs to be considered when designing interventions for digital self-control is the business model followed by contemporary tech companies. This is an important challenge that needs to be addressed in the near future. Indeed, as reported in Section 5, there is an inherent contradiction between designing interventions to promote digital wellbeing and a business model that aims at maximizing users’ attention and engagement to optimize advertising revenue [144], i.e., the so-called Attention Economy [54]. Online services like social networks and video streaming platforms, in particular, are often intentionally designed to maximize frequent and extensive use, to the point of cultivating the idea of promoting automatic and unconscious usage patterns [74, 96], e.g., by exploiting users’ psychological vulnerabilities [38].

Unfortunately, our work highlights that the HCI community has so far had an individualistic view of the problem: through DSCTs, researchers investigated how to help users change individually, without taking into account the larger systemic design factors at work. Only recently, researchers [139, 215] have started to wonder what the role of tech companies is in the current “race for digital wellbeing [159].” While tech companies are often blamed for not doing enough against problems like violence and radicalization on social networks, indeed, avoiding problems related to an excessive use of technology has been traditionally considered as a responsibility that belongs solely to the user [215]. As reported in this work, this one-sided view of the digital wellbeing problem is favored by another underlying contradiction that differentiates the digital wellbeing context from other behavior change domains, i.e., the fact that digital devices and services are, at the same time, the source of the problem and the platform with which the interventions are delivered to the user. This is reflected in the variety of tools for digital self-control that have been recently proposed, and on the difficulty of developing cross-platform solutions (see Section 5, Overcoming Technical Constraints). Recently, the rising research [123, 194] and main-stream media attention [4, 42] on digital wellbeing has finally spurred tech giants like Google and Apple to acknowledge the problem by introducing self-monitoring usage statistics and tools, as highlighted by the two reviews of contemporary DSCTs included in our corpus (Monge Roffarello and De Russis [159] and Lyings et al. [144]). However, we claim that this only the first step towards a real change in the today’s business model: while usage timers and lockout mechanisms may externally apply to many different technological sources, indeed, they do not change the internal attention-capture dark patterns [139] that are often adopted by contemporary websites and mobile applications, e.g., the autoplay of the next video [81, 139].

Just as we believe that the responsibility for digital wellbeing does not lie entirely with users and researchers, we must also acknowledge that a narrative depicting tech companies as the primary source of overuse problems is not useful [215] and mostly speculative [124]. We argue that a real double-sided view of the digital wellbeing topic, one that includes both users, researchers, and tech providers at the same time, is fundamental to promote a positive and meaningful use of technology in the future. Tech companies, for example, could work to minimize the technical constraints that are nowadays hampering the work of HCI researchers and practitioners, e.g., by improving API transparency and by creating permissions to collect usage data [80]. The “Digital Wellbeing Experiments” platform by Google [7] is an example of a promising initiative that goes in this direction: it includes a showcase of guides and open-source experiments to assist designers to kick start new ideas and tools for digital self-control. Furthermore, the HCI community and tech providers could work together to find alternative business models that target users’ digital wellbeing, rather than users’ attention and engagement. A strategy extracted in our review, for example, is to rethink the concept of “relevance” for recommender systems [139], by taking into account when recommendations might promote excessive and compulsive usages of the service. As reported by Lukoff et al. [140] (Qualitative Discussions corpus), designing for users’ digital wellbeing may initially result in a lower user engagement, thus resulting in a lower business profitability in the short term, but it could also increase user loyalty in the long term. To motivate key stakeholders to support “more responsible and sustainable business [224]”, in particular, researchers could engage with policies [224] and standardized definitions: according to Lukoff et al. [139], for example, the design community might encourage tech companies to limit the usage of attention-capture dark patterns by developing “a common language of attention capture dark patterns that recognizes designs that lead to attentional harms” (p.13).