Work Package 1: Stakeholder Co-Design of the EPICare Registry and CDSS

A participatory co-design framework previously established for informatics in mental health [29] was used to engage a diverse network of stakeholders, including clinicians, academics, policy makers, and PPIE members, in a series of 4 co-design workshops. Due to the COVID-19 pandemic, the workshops were convened online. While this enhanced the scope for collaboration between centers in the study, there were also potential drawbacks of this approach, which included PPIE members needing to have access to and know-how of technology. Second, hosting face-to-face meetings on neutral ground in an approachable format may have helped to remove traditional power structures. The workshops were in a facilitator-led, semistructured format, including presentations, whole-group discussions, and themed breakout activities (card-sort tasks and small group discussion) with both mixed (random allocation) and streamed group (by broad stakeholder type) sessions on a per-task basis. Essential materials were circulated to stakeholders in advance of each meeting. We also convened additional online preparatory sessions for PPIE stakeholders, led by our PPIE coordinator, to aid understanding and participation in the main workshops. Registry and CDSS goals were examined by stakeholders, who reviewed, refined, and identified a core set of essential and desirable measures that should be collected in the integrated EPICare registry and CDSS, across 4 domains: sociodemographic measures, treatment measures, patient-reported outcome measures, and clinician-reported outcome measures.

To facilitate this process, stakeholders were provided with a list of data already recorded in electronic health records in EIP services, in addition to information on data relevant to the 8 nationally mandated NICE standards for EIP care [30] and key outcome measures (Health of the Nation Outcomes Scales [HoNOS] on functioning; quality of life and treatment satisfaction [DIALOG]; and patient-reported recovery [QPR]). This was supplemented with a minimal set of other initial measures recognized as potentially relevant by the program management group based on expert knowledge, prior to the first workshop. Examples included symptom ratings, duration of untreated psychosis, and genotyping, among others. In workshops 1 and 2, stakeholders were asked to review, refine, prioritize, and suggest additions or subtractions from this list, with other data that may not currently be routinely collected, but considered by stakeholders to be essential or desirable. The group also explored what barriers to implementation and uptake may be encountered in EPICare (eg, data security and ownership, time for completion, and digitizing of routine data currently collected on paper). Similarly, in workshop 3, stakeholders identified the essential and desirable features of a CDSS to provide timely actionable insights for patients and clinicians, including potential clinician prompts to complete health assessments aligned to NICE standards for EIP care.

After these 3 initial workshops, we synthesized all information gathered via a modified Delphi approach involving all stakeholders and members of the research team, to develop a consensus of data priorities (what, when, and by whom). We shared this with the members of work package 2 to understand technical and governance barriers to implementation to further refine our framework to identify a set of “must have” and “could have” data elements. Finally, in workshop 4, we presented our proposed framework to stakeholders, sought further feedback, and outlined our plans for stage 2 of EPICare development. From our initial stakeholder network, we sought to retain a representative group of stakeholders for our stage 2 activity, who will continue to guide the pilot, testing, and evaluation throughout the project.

Work Package 2: Informatics Architecture, Infrastructure, and Integration—Framework, Protocol, and Tool Kit

Work package 2 aimed to reduce technical and governance challenges in the future full build of EPICare by addressing key questions and unknowns. Based on prior experience and knowledge, the area of biggest technical risk for EPICare is the retrieval of data from electronic health records and the standardization of this into a common data model, while ensuring compliance with information governance and ethical standards. Previous work by the group involved auditing all EIP services that are part of the National Institute for Health Research (NIHR) Mental Health Translational Research Collaboration in Early Psychosis (MHTRC-Early Psychosis), to inform understanding of the existing infrastructure, capacity, capabilities, and limitations around designing and developing the EPICare platform for potential national implementation. This initial scoping work has highlighted several different electronic health records in use as well as different ways of capturing and storing relevant data in each of the trusts.

To build on this knowledge, health informatics specialists within the research team contacted and liaised with 5 NHS Trusts, including Greater Manchester Mental Health NHS Foundation Trust, Birmingham Women’s and Children’s NHS Foundation Trust, Cambridgeshire and Peterborough NHS Foundation Trust, Nottinghamshire Healthcare NHS Foundation Trust, and Camden and Islington NHS Foundation Trust. Information technology team leads and proposed demonstrator sites were identified, gathering information to further understand what electronic health record system is used by each trust; what relevant data are currently captured in the electronic health record (key foci: 8 NICE standards for first-episode psychosis treatment); how that data can be retrieved, such as via application programming interface or through regular exports; how the data can, and should be secured during retrieval, complying to the highest information governance standards; and the data formats used for each type of data.

Once this information was captured from all trusts, it was then used alongside the information gathered from the stakeholders in work package 1 to develop and document key requirements for the EPICare system; software architecture for the registry and CDSS; data model design including standardization of data items into a common format; security threat model including planned treatments for identified threats; proposed infrastructure to include an appropriate hosting solution, such as a secure cloud environment; and an integration plan for retrieval of data from each trust’s electronic health record.

This was drafted into an overall framework document and set of technical infrastructure and information governance protocols and tool kits to inform the future build, pilot, implementation, and evaluation of the EPICare platform. With all of this in place, the technical and governance challenges for the main program grant for applied research application should be significantly reduced.

Work Package 3: Implementation Evaluation Framework

Working in parallel and in collaboration with the members of work packages 1 and 2, the purpose of this work package was to establish the preliminary implementation framework for the subsequent testing and rollout of EPICare. Founded on the idea that implementation research should be integrated throughout all stages of innovation development rather than at “end-stage,” this involved understanding the distinct and interconnected implementation issues within the stages of problem definition; iterative evidence-building, intervention conceptualization, development, and testing; and subsequent rollout, experimentation, and embedding in different service settings. With particular reference to EPICare, this involved understanding how the earlier stages of stakeholder engagement contributed to intervention development and, at the same time, how stakeholders perceived challenges to future adoption and use. With regards to PPIE stakeholders (work package 1), this involved understanding views about (1) current challenges in EIP care; (2) how clinical registries and CDSS might influence care and service improvement; (3) expectations about how interventions might be used in standard practice; and (4) participants’ experiences of the co-design process. We also studied the early stage activities of the health informatics team (work package 2) to understand the explicit and tacit design assumptions; the contingencies presented by current technological parameters; the influence of prevailing governance arrangements; and importantly, to understand and evidence the interaction between the relative influence of multiple stakeholders in the co-design process. This evidence will be brought together with existing implementation science frameworks, such as normalization process theory (NPT) [31], in conjunction with complementary insights drawn from science and technology studies [32,33].

NPT helps understand how service innovations are implemented, embedded, and normalized within organizations, to the point where new practices are no longer regarded as new. It is different from other implementation models because it focuses on the specific “work” undertaken by social actors to implement innovations into everyday practice while taking into consideration the interplay between actions, contexts, and objects. NPT has 4 linked constructs, “coherence,” or the work of making sense of an innovation; “cognitive participation,” or the work involved when engaging with an innovation; “collective action,” or the combined work of integrating new practices into existing skills, relationships, and contexts; and “reflexive monitoring,” or the work of continually appraising and adapting to the introduction of new practices. It has been widely used to explain the factors that shape the implementation of complex interventions [31].

Field researchers directly observed all 4 stakeholder co-design workshops and considered the influence of multiple social, cultural, and organizational factors on the co-design process. They also observed a selection of key design meetings held between the technologists, NHS Trust partners, and other stakeholder groups to map and describe the iterative development of EPICare. Each researcher recorded their observations following an agreed semistructured guide which were then aggregated for analysis.

To clarify the observational data, qualitative semistructured interviews guided by the constructs from NPT were then conducted with all stakeholder groups to understand the factors likely to influence the subsequent implementation of EPICare. An initial set of questions and topics derived from the study objectives were used to systematically code interview transcripts and develop themes. This was piloted on 4 transcripts by 2 researchers, before agreeing to a revised set of codes, followed by further coding of remaining transcripts. Interviews focused on the different cognitive-cultural perspectives of each stakeholder group, their experiences of participating in the co-design process, and their perceptions about their influence on the co-design, together with their recommendations for subsequent development and testing.

Study Participants

Participants were recruited between November and December 2021. We recruited 40 participants across all stakeholder workshops (work package 1). This included at least 10 people with lived experience of psychosis, and ideally, lived experience of early intervention, to form the PPIE stakeholder group. PPIE members were recruited from the Birmingham University Youth Advisory Group, Cambridgeshire and Peterborough Foundation Trust, Bristol Lived Experience Advisory Panel, and PPIE networks at University College London, including those associated with the NIHR Mental Health Policy Research Unit. As an acknowledgment of the time and effort involved in taking part in the study, PPIE participants were reimbursed in line with the Involve payment policy [34], which equates to £25 (US $33.19 at the time of the study) per hour of participation.

The remaining 30 participants were recruited from the breadth of multidisciplinary care in EIP services (psychiatrists, psychologists, occupational therapists, social workers, and nurses), in addition to stakeholders from the charitable sector, NHS England, policy makers, and other academics, for facilitated group meetings. The clinical collaborators were recruited from NHS Trusts serving diverse and underserved areas with a combined population of approximately 3.4 million people (corresponding to 9.1% of the English population eligible for EIP services): Birmingham Women’s and Children’s Trust, Manchester Health and Care NHS Foundation Trust, Camden and Islington NHS Foundation Trust, Avon and Wiltshire Mental Health Partnership Trust, and Cambridgeshire and Peterborough Foundation Trust. Attendance at the stakeholder group meetings was taken as consent for this process and no individual written consent was required from stakeholders (including PPIE).

All stakeholders were also invited to participate in individual qualitative interviews in work package 3 to ensure that we selected a representative subset of each stakeholder group from our work package 1 stakeholder meetings. Written informed consent was obtained and interviewees were given a unique participant identification number, which was used throughout the transcription of interviews to ensure anonymity.

Given the online group format of the stakeholder workshops, individual participants attending these workshops were identifiable to each other and to the authors. However, the identities of participants who consented to an individual qualitative interview were known only to the interviewer, and as noted above, interviewees were assigned a unique participant identification number to ensure their anonymity during the transcription of their interviews.

Work Package 2: Informatics Architecture, Infrastructure, and Integration—Framework, Protocol, and Tool Kit

Work package 2 occurred in parallel with work packages 1 and 3, between December 2021 and September 2022. In this work package, we collaborated with NHS informatics teams to address key questions about informatics architecture, infrastructure, governance, and integration plans to facilitate onward development and testing of EPICare in diverse NHS Trusts.

Early Psychosis Informatics into Care (EPICare) is fully funded by the National Institute for Health Research (NIHR) Programme Development Grants (PDG) Mental Health Call 2021 (grant NIHR203669). Birmingham Women’s and Children’s National Health Service (NHS) Foundation Trust agreed to act as sponsor for this study. This research is supported by the NIHR Oxford Health Biomedical Research Centre, the NIHR University College London Hospitals (UCLH) Biomedical Research Centre, the NIHR Cambridge Biomedical Research Centre (NIHR203312), and the NIHR Applied Research Collaboration East of England. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.