Abstract

1. Introduction

Evidence-based psychosocial treatments for psychosis have strong support for improving symptoms and recovery outcomes among people with serious mental illness (SMI), including schizophrenia spectrum disorders (Bighelli et al., 2021; Frawley et al., 2021; Turner et al., 2018). However, as few as 10% of people with a diagnosed SMI receive these interventions (Dixon et al., 2010). Lack of access leads to poorer outcomes, including higher rates of relapse and hospitalization, and limited community engagement (Lehman et al., 1998). Digital mental health interventions, such as those provided by smartphone applications (apps), show promise as cost-effective approaches to increasing access to evidence-based psychosocial interventions for psychosis (Ben-Zeev et al., 2019).

Limited access to technology is often a barrier to reaping the benefits of digital approaches to mental healthcare in SMI (Fulford and Mote, 2019). Estimates show a large range of ownership of digital technology, including smartphones. For example, fewer than 50% of patients with schizophrenia receiving care in the community report owning a smartphone (Klee et al., 2016; Young et al., 2020), while the rate is as high as 90% in patients with SMI in a private clinic (Torous et al., 2018). The varying rates are tied to socio-demographic factors, including age and economic status. Not surprisingly, older people with SMI report lower rates of smartphone ownership than younger ones (Watson et al., 2021), and receipt of disability payments is a strong predictor of reduced likelihood of smartphone ownership (Young et al., 2020). While there are affordable options for smartphones, the financial resources needed for purchasing data plans can be prohibitive for those living in poverty (Vogels, n.d.).

Even if such technologies can be provided to people with limited resources, such as through a public-funded initiative, it is unclear if general lack of familiarity with the technology could limit the potential benefits. Furthermore, characteristic symptoms and impairments in people with SMI themselves could interfere with capacity to engage in, and thus benefit from, digital interventions. One such characteristic concerns intellectual and cognitive ability. Studies across all stages of psychosis suggest cognitive impairment is a key feature of SMI (Fett et al., 2020; Seidman et al., 2006). Although low educational attainment does not appear to limit gains in traditionally-delivered evidence-based treatments for psychosis, such as CBT (Turner et al., 2020), digital interventions such as those implemented through smartphone apps require active participation, often with limited human support, which could create additional challenges in engagement for those with cognitive impairment. Thus, it is important to evaluate whether intellectual and cognitive functioning is related to digital treatment engagement and outcomes.

A related challenge for people with SMI is in social cognitive impairment, which impacts social outcomes (Couture et al., 2006; Fett et al., 2011) and could potentially limit the gains of interventions designed to improve social relationships. Interventions designed to directly improve social cognition, such as Social Cognition Training or Cognitive Enhancement Therapy, demonstrate positive benefits in social cognitive abilities and functioning (Nijman et al., 2020); however, the extent to which social cognitive abilities impact the benefits of social goal interventions, particularly those delivered digitally, is unknown. That is, difficulties in identifying emotions and thoughts in others—through impaired affect recognition and theory of mind abilities—could limit successful implementation of strategies designed to form and maintaining interpersonal bonds, such as in supporting social goal attainment.

We previously developed a smartphone app to provide guided support in improving social functioning for people with psychosis. This app—Motivation and Skills Support (MASS)—was designed with direct input from people diagnosed with schizophrenia spectrum disorders, with a key aim to minimize the potential impact of cognitive or motivational impairments on treatment engagement. We used e-design tools validated to reduce burden of these impairments in people with serious mental illness (Rotondi et al., 2017), and focused on enhancing engagement through empirical work on the temporal experience of pleasure model (Fulford et al., 2018; Gard et al., 2014). Intervention development occurred in two phases—one in which participants engaged with the app over brief periods of time and provided qualitative feedback, allowing for iterative design (Fulford et al., 2020); in the second phase we examined usability and preliminary efficacy on target outcomes (Fulford et al., 2021). Our ultimate goal was to develop an empirically- and theoretically-informed digital intervention that could be used in daily life without regular clinician or caregiver support.

In the current study we examined the extent to which intellectual and cognitive abilities were associated with treatment engagement and outcome in a pilot evaluation of the MASS app digital intervention. We hypothesized that premorbid cognitive and social cognitive abilities would not be associated with treatment engagement, including responses to smartphone notifications or interest/motivation in app content (i.e., working toward a social goal), given our user-centered approach to design elements and usability. Relatedly, we hypothesized that premorbid cognitive ability would not be significantly associated with changes in the primary treatment outcome of social functioning. Conversely, we hypothesized that higher levels of social cognition would be significantly related to greater gains in social functioning over the intervention, given their role in providing a foundation for effective social skills and functioning.

2. Method

Data presented in this paper were collected as part of an open pilot intervention study that took place in the Boston and San Francisco Bay areas (registered clinical trial NCT03404219; see Fulford et al., 2021). Clinical and outcome data were collected before and immediately following the eight-week intervention, as well as three months following treatment termination. Interested participants who met screening criteria completed an assessment of eligibility and baseline clinical measures, followed by the intervention. Participants had to be fluent in English and between the ages of 18 and 65 years, and receiving current standard care (medication, psychotherapy, or both). Participants were not eligible if they endorsed: 1) current substance use disorder (i.e., over the past six months) or 2) suicidal ideation, or 3) reported a diagnosed neurological disorder. In the current study, we present findings on participants who completed the intervention and had all available data for planned analyses.

Thirty-one individuals diagnosed with schizophrenia (n = 17) or schizoaffective disorder (n = 14) (mean [SD] age = 46 [11] years) were recruited using fliers and in-person presentations at community treatment and rehabilitation centers that serve people with SMI. Nine (29%) graduated from college, 11 (36%) were employed at least part-time, and 21 (75%) received disability payments. Severity of positive and negative symptoms was mild to moderate at baseline (see Fulford et al., 2021).

3. Results

Data were normally distributed and there were no individual univariate or multivariate outliers. Mean WRAT-WR and ER-40 scores were slightly higher but relatively comparable to those documented in previous studies in schizophrenia (i.e., WRAT-WR were 2–3 scaled points higher, and ER-40 scores were 1% higher, than in prior studies; see (Hill et al., 2013; Pinkham et al., 2018)). Bivariate correlations between premorbid cognition (WRAT-WR standard score) and social cognition (ER-40 total accuracy) and demographic variables indicated only one significant association—higher WRAT-WR was associated with higher level of education (rho = 0.40, p = 0.04). Higher ER-40 was also associated with higher level of education, though this was not statistically significant (rho = 0.31, p = 0.09). Indicators of socioeconomic status (education level, employment, receipt of disability payments) were unrelated to aggregated markers of treatment engagement (see Table 1).

Table 1

Socio-demographic variables and associations with cognition.

WRAT-WR standard score
	n	Mean	SD	t-test	Rho/r	p
Age				–	−0.10	0.60
Education level				–	0.40	0.04
Male	15	91.47	11.93	−1.69	–	0.11
Female	12	100.92	16.20
Disability payments	19	95.11	15.84	−1.39	–	0.19
No disability payments	7	104.14	14.69
White	13	94.15	14.67	−0.78	–	0.44
Non-white	15	98.60	15.41
Employed	9	94.11	13.97	0.61	–	0.55
Unemployed	19	97.68	15.65
Total	28	96.50	15.00	–	–	–

Open in a separate window

ER-40 total accuracy
	n	Mean	SD	t-test	Rho/r	p
Age				–	0.01	0.98
Education level				–	0.31	0.09
Male	16	31.56	4.00	−0.56	–	0.58
Female	14	32.36	3.71
Disability payments	21	31.95	3.76	−1.39	–	0.19
No disability payments	7	34.29	1.80
White	14	31.29	4.14	−0.94	–	0.36
Non-white	17	32.59	3.47
Employed	11	33.36	2.25	−1.81	–	0.08
Unemployed	20	31.25	4.27
Total	31	32.00	3.78	–	–	–

Open in a separate window

Note. ER-40 = Penn Emotion Recognition Test; WRAT-WR = Wide Range Achievement Test – Word Reading. For gender, one participant identified as non-binary and was not included in analyses. One participant declined to answer whether they received disability payments, and two others were unsure. Total possible n for WRAT-WR data = 28.

p < 0.05.

WRAT-WR scores showed small to moderate negative associations with aggregated treatment engagement markers, including proportion of notifications responded to (rho = −0.33), proportion of notifications in which the participant elected to work on their goal (rho = −0.12), and degree of interest in working toward the goal (rho = −0.02, see Fig. 1 and Table 2). When examining proportion of notifications responded to as a function of WRAT-WR standard score, those higher in premorbid cognition (i.e., above 100 standard score) did not differ in the proportion of completed data than those lower in premorbid cognition—54% vs. 42%, respectively (t = 1.10, p = 0.28). WRAT-WR scores also showed a small positive association with change in social functioning (i.e., scores at termination, controlling for baseline scores: b [SE] = 0.12 [0.07], t = 1.84, p = 0.08, Δ R² = 0.05). ER-40 total accuracy showed small associations with treatment engagement and outcome (rho values < ±0.20).

Open in a separate window

Fig. 1

Scatterplots of association between WRAT-WR and treatment engagement.

Generally, Bayes factors suggested anecdotal evidence for the hypotheses of no association between WRAT-WR and EMA notifications responded to (B₁₀ = 0.962) and election to work on the social goal (B₁₀ = 0.318), and moderate evidence of no association between WRAT-WR and motivation to work on the social goal (B₁₀ = 0.240). For the association between WRAT-WR and change in social functioning, there was anecdotal evidence of the alternative hypothesis (i.e., a positive correlation; B₁₀ = 1.530). There was also moderate evidence of no associations between ER-40 and treatment engagement and outcome (B₁₀ range: 0.23 to 0.34). Results are presented in Table 2.

4. Discussion

It is unknown to what extent impaired cognitive abilities may reduce potential benefits of standalone digital interventions for psychosis, due to problems such as difficulty navigating software and hardware in the latest mobile technologies. These challenges could limit the initiation and sustained use of technologies in daily life and contribute to a ‘digital divide’ between those who do and do not benefit from such interventions, diminishing the potential for increased access to evidence-based interventions in this hard to reach population (Fulford and Mote, 2019). In the current study we sought to provide a preliminary evaluation of this question.

Our findings suggest that in a middle-aged sample of outpatient participants with a schizophrenia spectrum disorder, premorbid cognitive ability was not associated with indicators of engagement in a digital intervention for social functioning. This preliminary finding is encouraging and quells some concerns that cognitive abilities could diminish the accessibility of standalone mobile interventions designed for this population. Importantly, the MASS intervention was developed for and with people with schizophrenia spectrum disorders. As such, our findings cannot address the extent to which cognitive impairments might impact engagement with mental health apps available to the general public.

There was a small association between higher cognitive ability and greater improvement in social functioning over the intervention—WRAT-WR scores explained 5% of the variance in SFS score improvement, with a Bayes factor that suggested anecdotal evidence for the alternative hypothesis. The clinical significance of this small association may not be especially meaningful, but does indicate that cognitive ability could have some impact on outcomes in digital psychosocial interventions, consistent with findings that people with greater cognitive impairment may be less likely to benefit from community-based psychosocial rehabilitation for schizophrenia (Kurtz et al., 2011). More work is needed to provide more robust tests of such potential associations, in larger samples and with more comprehensive measures of cognition and treatment engagement.

We also examined other potential contributors to the digital divide, including associations between treatment engagement and indicators of socioeconomic status. Education level, employment, and disability payment status were largely unrelated to proportion of notifications responded to, or participation or motivation for social goal support. These findings further suggest that availability of financial and other resources may not be robustly associated with the potential benefits of digital interventions among people with SMI. It is important to note, however, that participants in the study were provided with phones and data plans—thus, we can only speak to the extent to which such limited resources might impact the use (and not availability or access to) such digital interventions.

Finally, social cognitive ability at baseline, as assessed via facial affect recognition, was unrelated to treatment engagement or outcome. Social cognition is a multifaceted construct that includes several related information processing abilities that support basic capacity for interpersonal communication (Frith, 2008). Other facets of social cognition that might support treatment engagement and/or outcome, but that we did not measure in this study, include theory of mind or mentalizing, joint attention, and empathy. Furthermore, other interpersonal factors, such as social skills and motivation, may be more directly related to social functioning benefits (Fulford et al., 2018). Inclusion of measures of these constructs in future studies could provide evidence of the relevance of such factors for digital treatment engagement and outcome.

There are several limitations in the current study that should be mentioned. Our sample size was relatively small, limiting the potential to detect more subtle associations. We were only powered to detect moderate or higher correlations. Indeed, there were several relationships between cognition and intervention engagement and outcome that were positive but small in magnitude, which with a larger sample could have passed thresholds of statistical significance. Also, while the WRAT-WR has demonstrated evidence of convergent validity with more comprehensive measures of neurocognition, this scale is typically considered to be a measure of premorbid cognitive ability or intellectual function. There may be specific cognitive abilities that are more strongly tied to digital intervention engagement and outcomes. For example, executive function may be especially relevant for self-management skills that support sustained use of digital interventions.

In sum, in a standalone digital intervention for social function in psychosis, we did not find strong associations between premorbid cognitive ability or social cognition and treatment engagement and outcome. These findings suggest that while limited resources may certainly impact access to mobile devices, such as smartphones and associated mental healthcare apps, characteristic cognitive impairment among people with SMI may not contribute robustly to such a digital divide.

CRediT authorship contribution statement

Fulford: Conceptualization, Methodology, Writing – Original Draft, Writing – Review and Editing, Supervision, Funding acquisition; Schupbach: Data curation, Writing – Review and Editing; Gard: Conceptualization, Methodology, Writing – Review and Editing, Supervision, Funding acquisition; Mueser: Conceptualization, Methodology, Writing – Review and Editing, Funding acquisition; Mow: Investigation, Data curation; Leung: Investigation, Data curation.

Declaration of competing interest

The authors declare no conflicts of interest associated with the data presented in this manuscript.

Acknowledgements

References