Themes for evaluating the quality of initiatives to engage patients and family caregivers in decision-making in healthcare systems: a scoping review

Open access Original research Clayon B Hamilton ‍ ‍,1,2,3,4,5 Maryam Dehnadi,3,4 M Elizabeth Snow,3,6 Nancy Clark,7 Michelle Lui,4 Janet McLean,8 Hussein Mamdani,4 Allison L Kooijman,9,10 Vikram Bubber,9 Tammy Hoefer,11 Patients as Partners Team,2 Linda C Li ‍ ‍4,5 To cite: Hamilton CB, Dehnadi M, Snow ME, et al. Themes for evaluating the quality of initiatives to engage patients and family caregivers in decision-­making in healthcare systems: a scoping review. BMJ Open 2021;11:e050208. doi:10.1136/ bmjopen-2021-050208 ►► Prepublication history and additional supplemental material for this paper are available online. To view these files, please visit the journal online (http://​dx.​doi.​org/​10.​1136/​ bmjopen-​2021-​050208). Received 13 February 2021 Accepted 06 September 2021 © Author(s) (or their employer(s)) 2021. Re-­use permitted under CC BY-­NC. No commercial re-­use. See rights and permissions. Published by BMJ. For numbered affiliations see end of article. Correspondence to Dr Clayon B Hamilton; ​clayon.​hamilton@​fraserhealth.​ ca ABSTRACT Objective To identify the key themes for evaluating the quality of initiatives to engage patients and family caregivers in decision-­making across the organisation and system domains of healthcare systems. Methods We conducted a scoping review. Seven databases of journal articles were searched from their inception to June 2019. Eligible articles were literature reviews published in English and provided useful information for determining aspects of engaging patients and family caregivers in decision-­making to evaluate. We extracted text under three predetermined categories: structure, process and outcomes that were adapted from the Donabedian conceptual framework. These excerpts were then independently open-­coded among four researchers. The subsequent themes and their corresponding excerpts were summarised to provide a rich description of each theme. Results Of 7747 unique articles identified, 366 were potentially relevant, from which we selected the 42 literature reviews. 18 unique themes were identified across the three predetermined categories. There were six structure themes: engagement plan, level of engagement, time and timing of engagement, format and composition, commitment to support and environment. There were four process themes: objectives, engagement approach, communication and engagement activities. There were eight outcome themes: decision-­making process, stakeholder relationship, capacity development, stakeholder experience, shape policy/service/programme, health status, healthcare quality, and cost-­effectiveness. Conclusions The 18 themes and their descriptions provide a foundation for identifying constructs and selecting measures to evaluate the quality of initiatives for engaging patients and family caregivers in healthcare system decision-­making within the organisation and system domains. The themes can be used to investigate the mechanisms through which relevant initiatives are effective and investigate their effectiveness. INTRODUCTION Healthcare systems have begun to embrace a person-­centred and family-­centred approach Strengths and limitations of this study ►► A comprehensive scoping review that produced a list of themes to evaluate initiatives that engage patients and family caregivers in healthcare system decision-­making within the organisation and system domains. ►► Multiple stakeholders, including patients and family caregivers, were involved in the research process to identify and describe the themes. ►► Experts outside of our research team were not consulted to provide feedback on the identified themes. to better meet the priorities of their users.1–3 This approach is an opportunity for healthcare users and their unpaid caregivers, such as family members, to partner with employees, such as managers and professionals, in shaping the healthcare system.4 5 One way the Ministry of Health in British Columbia, Canada is striving to achieve this is through the Patients as Partners Initiative launched in 2008. This initiative is meant to build capacity for and strengthen the engagement of patients and family caregivers in healthcare system decision-­making exercises.6–8 The Patients as Partners Initiative operates within the three domains of the health system: direct care or individual domain (micro-­ level), organisation or community domain (meso-­ level) and system or policy domain (macro-­ level).8 Ultimately, the Patients as Partners Initiative’s goal is to support achieving the quadruple aims of optimal patient and provider experience, better health outcomes, and better cost-­effectiveness.9 Evaluation is required to determine the extent to which policy-­ driven patient and family caregiver engagement initiatives like the Patients as Partners Initiative are effective. Hamilton CB, et al. BMJ Open 2021;11:e050208. doi:10.1136/bmjopen-2021-050208 1 BMJ Open: first published as 10.1136/bmjopen-2021-050208 on 11 October 2021. Downloaded from http://bmjopen.bmj.com/ on April 1, 2022 by guest. Protected by copyright. Themes for evaluating the quality of initiatives to engage patients and family caregivers in decision-­making in healthcare systems: a scoping review Open access METHODS Patient and public involvement Patient and family caregiver partners engaged as research team members throughout this scoping review.10 16 Patient partners (AK, ALK and VB) joined individually when recruited from a provincial community of volunteers and were offered a CA$35/hour honorarium as reasonable compensation in line with guidelines by regional health research organisations. AK is a retired male with chronic pain for over 10 years. JM, our family caregiver partner, is a Caucasian female in her mid-­60s with experience caring for two patients and works for a provincial non-­profit dedicated to family caregivers. AK is a Caucasian female in her late 30s with a disability, and who is a former licensed practical nurse and current master’s student who is a patient advocate with extensive engagement experience 2 and interactions with the healthcare system. VB is a visible minority long-­term male patient and a survivor of childhood and young adult cancer. He is very active in patient engagement activities, such as making conference presentations and being a community advocate for physical and mental health. ALK and JM joined at the beginning and were actively involved in virtual team discussions and provided feedback on written documents. AK resigned for personal reasons after a year, having participated in three team meetings and copresented the research protocol to the British Columbia Ministry of Health. Two new patient partners (ALK and VB) joined the research team and participated from the fourth team meeting. Patient partners and family caregiver partners contributed to team discussions and manuscript writing by reviewing and commenting on drafts. For example, they influenced the results by suggesting names and descriptions for themes, ways to combine themes and identifying a missing theme. Approach Arksey and O’Malley’s scoping review methodology enhanced by Levac et al guided this review.17 18 The methodology consists of six stages and does not require assessment of the methodological quality of included studies. The current paper is based on the first five stages, from identifying the research question through to reporting the results. Stage 6 is ongoing to consult multiple stakeholders to create a conceptual evaluation framework based on our results. We followed the standards of Preferred Reporting Items for Scoping Reviews and Meta-­Analysis Extension for Scoping Review checklist.19 We revised our published review protocol in an iterative process to refine the scope of our research question.10 Scoping reviews do not require ethics approval.17 18 Stage 1: identified the research questions Our question remained unchanged from our published protocol: ‘What key elements define the quality of patient, family, caregiver, and public engagement in decision-­ making in healthcare systems for use in the evaluation of a provincial engagement initiative?’10 The key elements were framed as themes for evaluation. In contrast to the published protocol, we focused on engagement within the organisation and system domains and excluded individual domain engagement. The individual domain draws on a body of literature pertaining to self-­management and self-­care,5 which is outside of the direct context of the other two domains and the scope of this study. Stage 2: identified relevant studies A university librarian with expertise in systematic reviews helped to develop the search strategy for seven databases: CINAHL, Cochrane Library, Embase, MEDLINE, PsycINFO, Social Work Abstracts and Web of Science from their inception (to capture any foundational study) through to 14 June 2019. See the online supplemental appendix A for our search strategy. Our search was broad because of the semi-­organised and evolving nature of the Hamilton CB, et al. BMJ Open 2021;11:e050208. doi:10.1136/bmjopen-2021-050208 BMJ Open: first published as 10.1136/bmjopen-2021-050208 on 11 October 2021. Downloaded from http://bmjopen.bmj.com/ on April 1, 2022 by guest. Protected by copyright. In this paper, ‘patients and family caregivers’ simply describes the many categories of individuals and groups served by healthcare systems.10 In a 2009 scoping review on public engagement in healthcare priority setting and resource allocation, Mitton et al noted that engagement exercises are rarely evaluated.11 The Manafò et al scoping review a decade later, likewise, noted that limited evaluation has hindered patient and public engagement in decision-­making exercises.12 We, therefore, do not have adequate information on what works and does not work for effective, sustainable and productive patient and family caregiver engagement.13 Challenges to evaluating patients’ and family caregivers’ engagement in healthcare decision-­ making include a lack of consensus on which elements are of importance, reliable and valid measures, and appropriate evaluation tools and methods.14 15 It is not clear the extent to which available questionnaires, checklists and other measurement tools address the key themes of patient and family caregiver engagement since those important themes have not yet been adequately mapped out to define success of these types of initiatives. This is a major gap in enabling evaluation, as these themes would provide a way for thinking about how patient and family caregiver engagement initiatives can be, and should be, evaluated. We propose the existing gaps can be addressed by a conceptual evaluation framework for determining indicators to monitor and evaluate for initiatives engaging patients and family caregivers. Such an evaluation framework could, furthermore, be used to establish an agenda for research, policy and quality improvement activities on patients’ and family caregivers’ engagement in decision-­ making in healthcare systems. As a first step, we sought to identify the key themes for evaluating the quality of initiatives to engage patients and family caregivers in decision-­ making across the organisation and system domains (ie, meso-­level and macro-­level) of healthcare systems. The individual domain (micro-­level) was excluded due to the unique and personal nature of engagement in one’s own healthcare. Open access Flow diagram for study inclusion. literature on patients’ and family caregivers’ engagement in healthcare system decision-­making. Stage 3: selected relevant studies Four authors (CH, MD, ML and HM) were involved in iteratively screening 7747 unique articles within the Covidence online platform (see figure 1).20 CH removed duplicate articles. Each article was screened by CH and either MD or ML. The titles and abstracts screening followed the published eligibility criteria. Selected articles were (1) published in English, (2) described patient and family caregiver engagement within healthcare systems and (3) provided useful information that could inform aspects of patient and family caregiver engagement in decision-­ making to evaluate. Additionally, we selected studies conducted in upper-­ middle-­ income and high-­ income countries to align with the British Columbia healthcare system context. Our study topic required us to read, in most cases, the complete abstract before deciding on the article selection. Two to three reviewers (CH, MD and/or ML) discussed the selection of the first 300 articles in sets of 100 articles as a training exercise to achieve a common understanding of articles to retain for full-­text screening. We refined eligibility to include literature reviews and articles that included models, frameworks, recommendations, guidelines, questionnaires and other knowledge tools, as well as case studies and evaluation studies. An initial round of full-­text screening was done by CH and MD. During the second round of full-­ text screening, training for consistency was done in which MD, ML and HM each reviewed 100 articles in duplication with CH. After resolving conflicting selections, MD, ML and HM provided verbal rating to CH on “How confident are you to proceed with correctly selecting articles” using a 10-­point scale (1—not confident to 10—very confident). Hamilton CB, et al. BMJ Open 2021;11:e050208. doi:10.1136/bmjopen-2021-050208 3 BMJ Open: first published as 10.1136/bmjopen-2021-050208 on 11 October 2021. Downloaded from http://bmjopen.bmj.com/ on April 1, 2022 by guest. Protected by copyright. Figure 1 Open access Stage 4: collected data Full-­ text of 366 relevant articles were uploaded into NVivo software (V.12, QSR International). MD, ML and HM coded the articles first by applying the predetermined overarching categories of ‘structure’, ‘process’ and ‘outcome’ as defined in our protocol paper and adapted from the Donabedian conceptual framework.10 21 The Donabedian conceptual framework was used because it is widely accepted as a standard for guiding systematic evaluation of the quality of healthcare and fits well with the overarching categories of a logic model.21 22 The reviewers read each article and then coded appropriate content starting at the results section through to conclusions. The research team then performed open-­coding of the content captured within the three overarching categories to identify underlying themes. Initially, each reviewer read all of the excerpts and independently open-­ coded the same set of excerpts to an equivalent of five pages. The reviewers then met and discussed the identified codes to create a common understanding of how to label the codes for the purpose of identifying themes appropriate for guiding the evaluation of patients’ and family caregivers’ engagement in health system decision-­making. These discussions led to resolving differences in the sections coded, names given to codes and the definition of each code. This training was repeated once again using several more pages of excerpts. After this training on open-­coding and creating an initial coding scheme, HM, ML and MD each open-­ coded the excerpts for structure, process and outcome, respectively. CH met one-­on-­one with HM, ML and MD to discuss and finalise their codes. They combined and renamed codes as themes by changing terms from the articles to align with concepts that fit the study objective and redistributing some excerpts to more appropriate themes. Our research team of multiple stakeholders, listed as the authors in this article, met and discussed the preliminary themes identified and their corresponding excerpts. The discussions led to combining some themes within their overarching categories, renaming some themes and adding one new theme. The eligibility criteria were refined to specify literature reviews as the basis for identifying and describing the themes. Furthermore, the reviews included had to have used a systematic search of bibliographic databases. Many of the reviews identified had already synthesised much of the content of other types of articles. 4 Stage 5: summarised and synthesised results NC, MD and CH used the themes and corresponding excerpts to draft an initial description for the themes under structure, process and outcome. The summaries were primarily based on literature reviews as higher-­order knowledge to create ‘themes for evaluation’ as a useful tool, and were supplemented by non-­reviews for richer descriptions. After reading all of the relevant articles, 37 non-­review studies (see online supplemental file) were used when they provided additional and unique information for each theme’s description. The research team members further refined the descriptions of the themes through discussions by email, team meetings and one-­ on-­one phone calls. RESULTS Figure 1 shows 7747 unique articles retrieved. Eighty-­six of the excluded articles, including five reviews, were on low-­income or middle-­income countries. We selected 42 literature reviews from the 366 relevant articles. The characteristics of these literature reviews are summarised in tables 1–4. They were published between 2002 and 2019 (29 of 42 in 2015 or after) by lead authors from 10 countries (13 from Canada, 8 from Australia, 7 from UK, 6 from USA, 2 from Netherlands, 2 from Austria and 1 each from Belgium, Germany, Iran and Spain). While the titles and objectives of many of the reviews stated a target population of either patient or public, the methods sections typically described a broader inclusion of both patient and public, including caregiver, carer, citizen, consumer, family and services user. We identified 18 key themes which are summarised in figure 2 and described below. Of the 18 themes, 6 corresponded with the structure category, 4 with process and 8 with outcome. Structure themes Structure comprises the settings or contexts within which engagement occurs, such as organisational structure, materials and human resources. Engagement plan Evaluate engagement initiatives for having clear goals, purpose, guidelines and limits of decision-­making for effective engagement.13 23–34 Engagement plans should include guidance needed for initiatives to be successful, such as identifying the level of engagement, addressing power imbalance and outlining clear mechanisms for cultural safety, listening, inclusivity, non-­discrimination, representation, transparency of organisation leadership and a safe environment for patients and family caregivers to contribute.15 25 28 35–39 There are several best practices, models and frameworks to use in an engagement plan. Cleemput et al’s40 Preferred General Model for Public and Patient Involvement, as an example, can be used to map plans for engaging patients and family caregivers in healthcare policymaking. As another Hamilton CB, et al. BMJ Open 2021;11:e050208. doi:10.1136/bmjopen-2021-050208 BMJ Open: first published as 10.1136/bmjopen-2021-050208 on 11 October 2021. Downloaded from http://bmjopen.bmj.com/ on April 1, 2022 by guest. Protected by copyright. When each reviewer was below 9 in confidence, each screened another 100 articles together with CH and discussed conflicts to obtain 100% negotiated agreement. A high level of confidence among reviewers provided assurance that each understood the factors contributing to the negotiated agreement and would lead to the appropriate selection of articles. Following the standards for scoping reviews, we did not assess the methodological quality of the included studies. Open access First author (year)* Country† Population Type of article Specific focus on decision-­making Focus of the paper Djellouli (2019)50 UK Public (patient, carers, public, service user, public representative) Scoping review ►► Described the evidence about how ►► Large-­scale change, public involvement is conceptualised policy, health services, and conducted in large-­scale changes, reconfiguration how different stakeholders perceived the involvement process and its impact Scott (2019)23 Australia Patient, public, citizen, community Literature ►► Developed a deductive coding search followed framework and explored its alignment by framework with the key goals of Citizen/ analysis Community Jury (CJ) by analysing the transcripts of CJ deliberation on dementia Terao (2019)64 USA Patient, caregiver Review ►► Discussed common success factors and barriers encountered by paediatric quality improvement collaboratives Bombard (2018)69 Canada Patient, caregiver/ family, client, user Systematic review ►► Explored the strategies used to engage ►► Quality of care, quality patients in service planning, design improvement, health and evaluation of health services. In services, health delivery addition, it identified the outcomes and contextual factors that shaping optimal patient engagement to improve quality care De Weger (2018)35 The Netherlands Community, citizen Review ►► Developed eight guiding principles and highlighted the contextual factors and mechanisms that could lead to successful community engagement interventions ►► Community engagement, citizen engagement, community participation, healthcare Dukhanin (2018)15 USA Patient, public, consumer, community Systematic review ►► Developed a taxonomy of metrics for evaluating patient, public, consumer and community engagement in healthcare organization-­level, community-­level and system level decision-­making ►► Health systems, health planning, organisational decision-­making Liang (2018)42 Canada Patient Scoping review ►► Explored the literature regarding evaluation of patient engagement in hospital health service improvement Manafò (2018)12 Canada Patients and public Systematic rapid review ►► Described the existing evidence in relation to patient and public engagement priority setting in health ecosystem and health research ►► Priority setting processes, health ecosystem Oldfield (2018)65 USA Patient, family, community Systematic review ►► Explored the impact of patient/family advisory board or councils on health systems ►► Patient-­centred outcome research, evaluation, patient activation Snow (2018)43 Canada Patient Literature ►► Created a model for facilitating ►► Healthcare planning, review followed meaningful engagement of marginalised marginalised populations, by interviews populations in health service planning equity, gender Wieczorek (2018)62 Austria Patient, family, citizen Literature ►► Shed light on International Network of ►► Patient-­centredness and review followed Health Promoting Hospitals and Health family-­centredness, health by a Delphi Services Network’s work to improve promoting hospitals and procedure engagement of patient and family health services, quality in care and highlighted a multilevel improvement, multilevel approach to enable patient, family and approach citizens involvement in care ►► Citizens jury, community jury, health, public deliberation ►► Paediatric quality improvement collaboratives ►► Hospital health services, service planning *Year corresponds to when the article was first published online, which may be earlier than the date included in the citation. †The affiliation of the first author. example, Kohler et al used the co-­design methodology to guide patients and family members to partner with employees to co-­design a patient engagement strategy for primary healthcare.41 Hamilton CB, et al. BMJ Open 2021;11:e050208. doi:10.1136/bmjopen-2021-050208 Level of engagement Assess for the use of appropriate levels of engagement. Many frameworks have varying terminologies to describe levels of engagement. Common among the frameworks 5 BMJ Open: first published as 10.1136/bmjopen-2021-050208 on 11 October 2021. Downloaded from http://bmjopen.bmj.com/ on April 1, 2022 by guest. Protected by copyright. Table 1 Characteristics of the included literature review articles dated between 2019 and 2018 (n=11) Open access First author (year)* Country† Population Type of article Specific focus on decision-­making Focus of the paper Grant (2017)61 USA Patient, public Rapid systematic review ►► Discussed practical considerations ►► Online clinical guideline Harris (2017)24 Australia Consumer, community Literature review ►► Proposed a model to integrate ►► Consumer, community, followed by interviews consumer views and preferences into engagement, health technology, resource allocation decisions technology and clinical practice, resource allocation Jiang (2017)13 Canada Patient, family, advisory council Qualitative review Kohler (2017)41 Canada Public Rosenberg-­ Canada Yunger (2017)80 Patient, public Spain Sharma (2017)86 USA Young (2017)70 Canada ►► Summarised the literature regarding the implementation and organisational structures of Patient and Family Advisory Councils in cancer care centres in North America Patient and family Literature review ►► Developed a system-­level advisors followed by framework implementation strategy to development enhance inclusion of patient and family advisors in healthcare decision-­making Pagatpatan Australia (2017)66 Selva (2017)51 for using online methods to engage patients in clinical guideline development Realist synthesis Systematic review Patient, family, consumer and community Systematic review ►► Patient and family decision-­ making, primary healthcare ►► Identified nine evaluation criteria that ►► Resource allocation, decision-­ ►► Discussed the methodology ►► Clinical guideline development, ►► Investigated the impact of ►► Patient advisory councils, ►► Explored the roles and opportunities ►► Rare diseases, orphan drugs provided in guidance documents for incorporating patients or representatives and patients’ views in the clinical guideline development process Patient and family/ Scoping review caregiver council, cancer care, patient and family centred care ►► Context-­mechanism-­outcome, characterise optimum patient and public involvement in drug resource allocation decisions Patient, family member, caregiver or their representatives ►► Patient and family advisory ►► Identified and explained the underlying mechanisms and contextual factors that contribute to effective public participation in health policy and planning Literature Review development interventions involving patient advisory councils on clinical care outcomes, patient safety and patient satisfaction and the influence of patient advisors on healthcare changes for involving patients with rare diseases, families and patient organisations in the lifecycle of an orphan drug context of political commitment making, consumer and patient participation patient preference, patient views experience of care, practice improvement *Year corresponds to when the article was first published online, which may be earlier than the date included in the citation. †The affiliation of the first author. is a spectrum of control or influence that patients and family caregivers have on the decision-­making process and outcomes.11 15 40 42–47 Typical levels in order of increasing control include inform, consultation, involvement, collaboration, and empowerment or lead, which are suitable for different purposes and strategies for engagement to be meaningful.35 43 48 Time and timing of engagement Evaluate the adequacy of the amount of time spent engaging patients and family caregivers and appropriateness of the timing of engagement, including scheduling of decision-­making exercises.13 35 37 43 49 One can report 6 if sufficient time was available for engagement.29 37 43 49 In terms of timing, evaluation should take into account the frequency and whether or not engagement was done early, one time, recurring, ongoing, or in stages of a decision-­making process.11 13 31 35 37 40 43 50–52 Format and composition Format pertains to decision-­making groups within which patients and family caregivers engage. Some examples include advisory panels, governance boards, citizen councils, citizen juries, community forums, community health councils, patient and family advisory councils and boards, mixed advisory committees, patient organisations and Hamilton CB, et al. BMJ Open 2021;11:e050208. doi:10.1136/bmjopen-2021-050208 BMJ Open: first published as 10.1136/bmjopen-2021-050208 on 11 October 2021. Downloaded from http://bmjopen.bmj.com/ on April 1, 2022 by guest. Protected by copyright. Table 2 Characteristics of the included literature review articles dated 2017 (n=9) Open access First author (year)* Country† Population Type of article Specific focus on decision-­making Focus of the paper Abelson (2016)92 Canada Patient, public Review followed by designing an evaluation tool ►► Designed a public and patient engagement ►► Public and patient engagement Azmal (2016)59 Iran Patient, public or representative Literature review then qualitative study ►► Designed a conceptual model that can provide a ►► Patient and public involvement, Ocloo (2016)25 UK Patient and public Narrative Review ►► Explored the purpose and value of patient and ►► Patient and public involvement Scholz (2016)48 Australia Consumers, user Systematic review ►► Highlighted current understandings of ►► Consumer leadership, consumer Wortley (2016)44 Australia Patient, public, consumer, advocate Review ►► Provided a framework that could be adopted ►► Decision-­making in healthcare, Dalton (2015)26 UK Patient and public, service user Rapid systematic review ►► Discussed the different methods and ►► Public engagement, Ivanova (2015)60 Belgium Public, disadvantaged groups, community Literature review, policy analysis and qualitative study ►► Discussed the involvement of vulnerable ►► Sexual and reproductive health, Li (2015)36 Canada Public (excluded patient) Literature ►► Examined the concepts of ‘use’ where the review followed literature focus is on public involvement use in by a model health policy decision-­making and how it could of concept be understood, interpreted and operationalised development by different actors Conklin (2015)27 UK Public Systematic scoping review evaluation tool informed by a literature review and input from researchers and practitioners for use in a wide range of organisations practical guide for managers and policymakers to involve public in decision-­making public involvement (PPI) in healthcare decision-­ making and the role of power in involvement. In addition, it suggested the use of more inclusive frameworks such as 4Pi in evaluating PPI processes organisational resources and structures in consumer-­led organisations, determinants of leadership involvement and how consumer leadership interacts with traditional mental health service provision by health technology assessment organisations to examine when and where they may engage public in decision-­making processes approaches used to engage service users in decisions regarding health service reconfiguration and the contextual variables that may affect engagement populations in developing sexual and reproductive health policy processes ►► Discussed how the literature define ‘the public’, approaches to involvement in healthcare policy and outcomes of public involvement in health system decision-­ making, public and patient engagement evaluation healthcare system in healthcare, shared decision-­making participation, mental health health technology assessment reconfiguration, user engagement in health services health policy, vulnerable groups, policy development ►► Health policy research, policy decision-­making policymaking ►► Health policy, impact, outcomes, priority-­setting *Year corresponds to when the article was first published online, which may be earlier than the date included in the citation. †The affiliation of the first author. other organisation committees, such as steering committees.13 33 42 53–58 Composition pertains to representation based on the format employed, the number of patients and family caregivers engaged, and demographic factors relevant to initiatives.11 13 15 32 37 43 48 57 59–61 Support This theme involves evaluation of how organisations support patients and family caregivers in their roles.26 35 43 62 63 Support may include training and resources, such as funding, incentives, time, support staff, facilitative leadership, information, information technology infrastructure and tools that enable contributions to decision-­making.15 24 29 34 35 41 48 59 63–67 Support processes and strategies established through senior leaders’ buy-­ in can improve the sustainability of engagement Hamilton CB, et al. BMJ Open 2021;11:e050208. doi:10.1136/bmjopen-2021-050208 initiatives.34 41 62 Success of initiatives may depend on the training of all stakeholders—both patients/family caregivers and staff/professionals.15 30 37 40 65 Environment This theme pertains to the physical or virtual location and socio-­cultural conditions, which create the spaces of power within which engagement takes place.35 43 68 Environment includes how welcoming the spaces are to input from patients and family caregivers.35 68 69 This could include cleanliness, physical props, accessibility to marginalised groups, such as people with disabilities, and the extent to which stakeholders promote reciprocity, cultural safety and open communication.25 30 36 39 69 Engagement within virtual spaces have benefits and challenges.61 70–72 An enabling or engagement-­capable environment that 7 BMJ Open: first published as 10.1136/bmjopen-2021-050208 on 11 October 2021. Downloaded from http://bmjopen.bmj.com/ on April 1, 2022 by guest. Protected by copyright. Table 3 Characteristics of the included literature review articles dated between 2016 and 2016 (n=9) Open access Specific focus on decision-­ making Focus of the paper First author (year)* Country† Population Type of article Kenny (2014)53 Community, consumer, citizen Critical literature review ►► Discussed the challenges ►► Rural health, community for enacting rural participation community participation and strategies that may mitigate these challenges Australia SarramiForoushani (2014)67 Consumer, community Scoping meta-­ review ►► Identified various dimensions of consumer and community engagement-­related strategies and offered an eight stage model consisting of key elements of consumer and community engagement Street (2014)37 Australia Citizen, public, Systematic review community, patient, caregiver ►► Investigated the ►► Citizens’ juries, methodological aspects deliberative democracy, and characteristics health policy of citizens’ juries and their influence on jury processes and outcomes Franx (2013)54 USA Patient Review ►► Explored the ►► Collaborative models, implementation strategies implementation strategy, for building collaborative mental health, primary primary care mental care health Kreis (2013)28 Germany Public, consumer, patient, citizen, user Review ►► Explored operational ►► Health technology processes and underlying assessment, operational rationales of public processes, health policy engagement at health technology assessment agencies in France, Germany and the UK Ti (2012)81 Canada Drug users Narrative literature review ►► Explored the available evidence on peer engagement among people who use drugs in policy and programme development Tempfer (2011)87 Austria User, consumer, Systematic review community, patient ►► Provided an overview ►► Participatory of published data on approach, organisation consumer involvement in development, healthcare organisational organisational change development projects Mitton (2009)11 Canada Public Scoping review ►► Examined public engagement in priority setting and resource allocation ►► Priority setting, resource allocation, public engagement van de Bovenkamp The Netherlands (2008)83 Patient, consumer Review ►► Examined various dimensions of patient participation in guideline development ►► Patient participation, medical guidelines, patient-­centred medicine Hubbard (2007)29 UK People affected by Systematic review cancer ►► Explored involvement of ►► Cancer, community-­ people affected by cancer based research in healthcare research, and or services, policy, planning and decision-­making practice and its outcomes Cavet (2004)30 UK Children and young Systematic review people ►► Reviewed evidence about ►► Children, young people, engagement of children consultation and young people in public decision-­making Australia ►► Shared decision-­ making, consumer representation, patient involvement, implementation ►► Peer engagement, policy development, programme development Continued 8 Hamilton CB, et al. BMJ Open 2021;11:e050208. doi:10.1136/bmjopen-2021-050208 BMJ Open: first published as 10.1136/bmjopen-2021-050208 on 11 October 2021. Downloaded from http://bmjopen.bmj.com/ on April 1, 2022 by guest. Protected by copyright. Table 4 Characteristics of the included literature review articles data between 2014 and 2002 (n=13) Open access Specific focus on decision-­ making Focus of the paper First author (year)* Country† Population Type of article Abelson (2003)31 Canada Public/citizen Systematic review ►► Explored deliberative ►► Public participation, methods of public healthcare involvement and how decision-­making they contribute to health systems decision-­making Pickin (2002)49 UK Community Literature review followed by interviews and a workshop ►► Explored factors that contribute to the effectiveness of partnership between lay communities and statutory organisations, and developed a model to identify and address constraints to effective partnership between lay communities and statutory organisations ►► Lay communities, statutory organisations, partnership *Year corresponds to when the article was first published online, which may be earlier than the date included in the citation. †The affiliation of the first author. supports teams and removes barriers in engaging patients is considered essential for acting on issues of equity.73 74 Process themes Process denotes the methods or mechanisms by which engagement occurs. Objectives Evaluation should assess the extent to which patients and family caregivers contributed to setting the objectives and agenda of engagement events with respect to the level of engagement.15 43 69 This is different from goal and purpose, which focus on the overall initiative, rather than the engagement process for specific issues. The literature highlights the importance of stakeholder collaboration in developing the objectives and agenda for an engagement event based on the shared values and vision of all partners.43 69 Engagement approach Engagement approaches range from top-­down to bottom-­up models and involve a variety of theories and methods used in dynamic ways to engage stakeholders.43 46 48 75–77 Top-­ down approaches typically promote decision-­ making led by healthcare system organisations, and are considered less likely to completely represent patients’ and family caregivers’ perspectives.46 75 78 Partnership approaches have multiple loci of power and learning cycles that promote stakeholders’ insights.76 77 Bottom-­up approaches seek to promote collaboration and are often led by individuals or communities, such as representatives of advocacy organisations who demand health policy changes.50 76 78 Communication Information flows through deliberative or non-­deliberative processes.11 12 38 59 63 Deliberation is iterative discussions that enable participants to think, reflect, question and provide Hamilton CB, et al. BMJ Open 2021;11:e050208. doi:10.1136/bmjopen-2021-050208 points of views to uncover knowledge gaps and make consensus-­based decisions.38 56 A timely two-­ way dialogue facilitates problem-­solving and allows stakeholders to find common ground.67 69 Respectful and reciprocal discussions among parties occur when they actively interact and present their interests and values equitably in easily understood language.23 43 Characteristics of communication to evaluate include openness, respectful and reciprocal discussions, transparency about the use of stakeholders’ input, timely sharing of information on decisions made and inclusiveness of varying points of view.15 23 36 43 79 Organisations may exhibit a spectrum of responses to engagement that would indicate the extent to which communication was authentic and transparent from basic acknowledgement to detailing the use of stakeholders’ input in decision-­making.36 66 Engagement activities Evaluation should account for types and dynamics of the broad range of engagement activities by patients and family caregivers. Activities may include, but are not limited to, planning, designing, governing and evaluating health services; developing guidelines; allocating resources; reconfiguring health services; setting priorities; voting; completing surveys and doing key informant interviews; and providing feedback on documents or processes.11 12 15 26 27 29–31 33 35 36 41 51 52 57 60 61 66 69 71 80–82 During these activities, they may take on different roles such as being advisors, committee members, jurors or leads.23 34 41 48 81 Outcome themes Outcome is the effect of engagement activities. Decision-making process Engagement may contribute to a more transparent decision-­ making process. Engagement provides the 9 BMJ Open: first published as 10.1136/bmjopen-2021-050208 on 11 October 2021. Downloaded from http://bmjopen.bmj.com/ on April 1, 2022 by guest. Protected by copyright. Table 4 Continued Open access Infographic of themes for evaluating initiative to engage patients and family caregivers in healthcare system decision-­ opportunity for patients and family caregivers to have a substantive influence on decisions. Key elements to evaluate are the establishment of mutual trust, broadened understanding by stakeholders on relevant issues and the identification of trade-­offs; all of these make for a more legitimate process.42 50 80 83–85 Stakeholder relationship Engagement activities could strengthen the relationship between patients/family caregivers and other stakeholders. Establishing good relationships may involve effective interactions among stakeholders, stemming from enhanced communication, mutual understanding and trust.50 59 Improvement in stakeholder relationships affect those directly involved in the engagement initiatives and individuals in the larger population who have vested interested in decisions made.13 45 50 71 84 10 Capacity development Engagement can build on the capacity of stakeholders, particularly patients and family caregivers, by reducing their perceived stigma and enhancing their skills, competencies, confidence, knowledge, awareness, self-­efficacy (eg, feeling useful), self-­sufficiency and social capital as both individuals and groups.15 25 27 31 48 66 69 74 83 86–90 It can enhance their sense of control over decisions, expand their community network and advance career prospects through improved employability.43 86 90 91 Stakeholders (patient, family caregiver and others) experience An enjoyable and positive experience in which stakeholders feel valued is important. Such benefits could be reflective of positive characteristics in the structure and process of engagement, and other outcomes, such as mutual learning, skills development, feeling a sense of Hamilton CB, et al. BMJ Open 2021;11:e050208. doi:10.1136/bmjopen-2021-050208 BMJ Open: first published as 10.1136/bmjopen-2021-050208 on 11 October 2021. Downloaded from http://bmjopen.bmj.com/ on April 1, 2022 by guest. Protected by copyright. Figure 2 making. Open access Hamilton CB, et al. BMJ Open 2021;11:e050208. doi:10.1136/bmjopen-2021-050208 11 BMJ Open: first published as 10.1136/bmjopen-2021-050208 on 11 October 2021. Downloaded from http://bmjopen.bmj.com/ on April 1, 2022 by guest. Protected by copyright. Generic frameworks used for evaluation could be paired with the identified themes. Structure and process themes could be mapped onto constructs of the ConsolShape policy/service/programme idated Framework for Implementation Research (CFIR) Engagement enables and is a critical tool for shaping to investigate facilitators and barriers to achieving effec(reconfiguring, redesigning, restructuring or developing) tive engagement of patients and family caregivers.98 99 policies, practices, priorities, processes, programmes, The CFIR is based on the synthesis of multiple impleservices, strategic plans, guidelines, initiatives and other mentation science frameworks and provides a taxonomy discrete products that are responsive to patients’ and of constructs important to the implementation of interfamily caregivers’ needs, insights and firsthand experiventions.98 For example, ‘Support’ could be investigated 15 27 37 42 43 45 48 69 82 89 ences. using underlying themes from CFIR such as ‘Patients needs and resources’, ‘Implementation climate’, ‘LeadHealth status ership engagement’ and ‘Tension for change’ to identify Health status could be identified for populations or barriers and enablers to patients’ and family caregivers’ individuals within healthcare systems.15 59 60 62 86 When engagement in an initiative.98 Similarly, the generic health status is investigated for individuals, the patients evaluation framework called Reach, Effectiveness, and family caregivers who engage directly may benefit, Adoption, Implementation and Maintenance could such as having improved mental health.86 Studies have complement our outcome themes by investigating, for found improvement in the health status of populations, example, the degree of ‘Capacity development’ and for example, after consumer engagement in policy and ‘Stakeholder relationship’ for achieving the ‘Mainteadvocacy in primary care.69 72 89 nance’ of patients’ and family caregivers’ engagement in an initiative.100 Healthcare quality The themes are relevant for evaluation at different Improvements in the quality of healthcare received would points during and after engagement initiatives. The be consequent to engagement having positive effects on outcome themes include all four dimensions of the earlier outcomes, such as better accountability, decision-­ 13 42 51 69 83 89 95 ‘Quadruple Aim’ (ie, health outcome, experience of making, programmes and knowledge tools. patients, experience of professionals and cost of care), Many articles suggest that improvements in healthcare but changes in those dimensions may be consequent quality are realised through instrumental outcomes, to other outcomes such as ‘shape policy/service/ such as increased acceptability, accessibility, credibility, 9 programme’. Outcomes themes were typically aspiraeffectiveness, legitimacy, trustworthiness and usability 13 51 81 84 tional in the literature. The significance of outcome of policies, programmes and services. For themes could vary over time and relate to each other example, Sharma et al noted that improved accessibility in dynamic ways, for example, as mediators and moderof programmes developed with advisors within healthators. The use of outcome themes and their underlying care organisations could translate more effectively into elements must therefore be driven by theory about primary care outcomes.86 the mechanisms for the effectiveness of engagement initiatives. Cost-effectiveness Engagement may lead to greater cost-­ A systematic review by Dukhanin et al with search 15 42 59 69 82 96 conducted in 2015 reported a ‘Taxonomy of Metrics’ as effectiveness. This theme pertains to cost of a comprehensive map of process and outcome metrics engagement and engagement producing tangible returns with domains and subdomains for evaluating patient on investment in a way that is responsive to the healthcare 55 89 97 engagement initiatives at the organisation-­ level and needs of a population. system-­level of healthcare systems.15 The Taxonomy of Metrics complements the themes we identified. The DISCUSSIONS arrangement of our themes extends their findings by This scoping review identified a comprehensive set mapping themes not only for process and outcome, but of themes and their descriptions as key elements of for structure as well. Specifying structure themes adds patient and family caregiver engagement initiatives that clarity for evaluating the context in which engagement should be considered for evaluation. Each theme is initiatives occur. Mapping the metrics identified by useful when assessing the structure, process or outcome Dukhanin et al onto the 18 themes would provide some of engagement initiatives within the organisation and precise elements of engagement to assess. Themes not system domains of healthcare systems. These themes covered by the taxonomy, such as outcomes in shaping organise more precise underlying elements of engagepolicies as opposed to shaping services identified in the ment, and are not prescriptive in defining best practices taxonomy, would need other sources for the precise for any initiative. Rather, they provide a basic underelements.15 standing of key elements of engagement that should be Our study is limited by not including the grey literaevaluated regardless of the current availability of valiture or contacting experts to identify any missed review dated measurement tools. articles. The systematic search produced a relatively agency (or empowerment) among stakeholders and therapeutic effects among patients.15 30 31 59 69 86 92–94 Open access Collaborators Patients as Partners Team: Kristen Grant, Shannon Gibson, Shannon Holms, Sherry Bar and Sirisha Asuri. Contributors CH obtained funding, conceptualised the research and drafted this review. Several authors (CH, MD, MES, NC, ML, JM, HM, TH, PasP and LCL) made intellectual contribution to the development of the protocol and the acquisition of project funding. Authors (CH, MD, MES, NC, ML, JM, HM, AK, VB, TH, PasP and LCL) wrote the manuscript by reviewing and editing early drafts by CH, and by providing input through team discussions, one-­on-­one meetings and email. Funding This work is supported by SPOR Evidence Alliance through funding from the Canadian Institutes of Health Research (CIHR) under the Strategy for Patient Oriented-­Research (SPOR) initiative grant number GSR-­154442. CH was supported by a CIHR-­MSFHR Health System Impact Fellowship from the BC Ministry of Health, Canadian Institutes of Health Research, and Michael Smith Foundation for Health Research (MSFHR). CH was also supported by a MSFHR Research Trainee award. LCL holds the Harold Robinson/Arthritis Society Chair in Arthritic Diseases, and is supported by the Canada Research Chair Programme. She was also supported by a MSFHR Scholar Award. Competing interests JM is employed by Family Caregivers of British Columbia, a paid partner of the Patients as Partners Initiative and a non-­profit organisation that provides support and advocates for family caregivers within British Columbia, Canada. The Patient as Partners Team members are employed by the British Columbia Ministry of Health that provide partial support for this project. Patient consent for publication Consent obtained directly from patient(s). CONCLUSION The 18 themes and their descriptions provide a foundation for identifying constructs and selecting measures to evaluate the quality of initiatives for engaging patients and family caregivers in healthcare system decision-­ making within the organisation and system domains. The themes can be used to investigate the mechanisms through which relevant initiatives are effective and investigate their effectiveness. Consultation on the identified themes with representatives from groups interested in and affected by healthcare decision-­ making should refine the themes, build consensus on their level of importance for evaluation and create a conceptual evaluation framework. Author affiliations 1 Evaluation and Research Services, Fraser Health Authority, Surrey, British Columbia, Canada 2 Primary Care Division, Ministry of Health, Victoria, British Columbia, Canada 3 Faculty of Health Sciences, Simon Fraser University, Burnaby, British Columbia, Canada 4 Arthritis Research Canada, Vancouver, British Columbia, Canada 5 Department of Physical Therapy, University of British Columbia, Vancouver, British Columbia, Canada 6 Centre for Health Evaluation and Outcomes Sciences, Vancouver, British Columbia, Canada 7 Faculty of Human and Social Development, University of Victoria, Victoria, British Columbia, Canada 8 Family Caregivers of British Columbia, Victoria, British Columbia, Canada 9 Patient Voices Network, BC Patient Safety and Quality Council, Vancouver, British Columbia, Canada 10 School of Interdisciplinary Studies, University of British Columbia, Vancouver, British Columbia, Canada 11 BC Patient Safety and Quality Council, Vancouver, British Columbia, Canada Twitter Clayon B Hamilton @Dr_Clayon and Linda C Li @LLi_1 Acknowledgements The authors thank Ursula Ellis, health sciences reference librarian, at the University of British Columbia for help with developing the search strategy and for peer reviewing the search strategy using the PRESS checklist. Thanks to Shannon Holms, the former director of the Patients as Partners Initiative, for contributions to the initial conceptualisation of this study. Thanks to Andrew Koster for his contributions in the early stages of this study. 12 Provenance and peer review Not commissioned; externally peer reviewed. Data availability statement Data are available upon reasonable request. Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-­reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise. Open access This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-­NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-­commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-­commercial. See: http://​creativecommons.​org/​licenses/​by-​nc/​4.​0/. ORCID iDs Clayon B Hamilton http://​orcid.​org/​0000-​0002-​6852-​3436 Linda C Li http://​orcid.​org/​0000-​0001-​6280-​0511 REFERENCES 1 Entwistle VA, Watt IS. Treating patients as persons: a capabilities approach to support delivery of person-­centered care. Am J Bioeth 2013;13:29–39. 2 Sharma T, Bamford M, Dodman D. Person-­Centred care: an overview of reviews. Contemp Nurse 2015;51:107–20. 3 McCormack B, Borg M, Cardiff S, et al. Person-­centredness – the ‘state’ of the art. International Practice Development Journal 2015;5:1–15. 4 Carman KL, Dardess P, Maurer M, et al. Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Aff 2013;32:223–31. 5 Graffigna G, Barello S. Spotlight on the Patient Health Engagement model (PHE model): a psychosocial theory to understand people’s meaningful engagement in their own health care. Patient Prefer Adherence 2018;12:1261–71. 6 McQuillen K, Davis C, Ho K. Challenges in measuring patient participation: case studies from British Columbia’s Patients as Partners initiative. J Participat Med 2013;5:e10. 7 Maika C, Harper C, Maria J. Patients as partners. in patient engagement: catalyzing improvement and innovation in health care, 2016: 75–80. 8 Bar S, Grant K, Asuri S, et al. British Columbia Ministry of health patients as partners: a transformational approach. Healthc Manage Forum 2018;31:51–6. Hamilton CB, et al. BMJ Open 2021;11:e050208. doi:10.1136/bmjopen-2021-050208 BMJ Open: first published as 10.1136/bmjopen-2021-050208 on 11 October 2021. Downloaded from http://bmjopen.bmj.com/ on April 1, 2022 by guest. Protected by copyright. large sample of literature reviews, which was effective in identifying a comprehensive set of themes for evaluation. Future consultation with experts who study and stakeholders who participate in initiatives for engaging patients and family caregivers in healthcare system decision-­ making could refine the identified themes and their descriptions. By limiting included studies to those about upper-­ middle-­ income and high-­ income countries, our findings need to be confirmed for generalisability to low-­ income and middle-­ income countries. A future study could also investigate the levels of importance and feasibility for conducting evaluation within each theme. An international consensus study of experts could determine the precise elements to evaluate within each theme and possibly subthemes. Finally, given the limited availability of measures for themes specific to patients’ and caregivers’ engagement in healthcare system,14 research should fill this gap. Open access Hamilton CB, et al. BMJ Open 2021;11:e050208. doi:10.1136/bmjopen-2021-050208 36 Li KK, Abelson J, Giacomini M, et al. Conceptualizing the use of public involvement in health policy decision-­making. Soc Sci Med 2015;138:14–21. 37 Street J, Duszynski K, Krawczyk S, et al. The use of citizens’ juries in health policy decision-­making: A systematic review. Soc Sci Med 2014;109:1–9. 38 Gregory J, Hartz-­Karp J, Watson R. Using deliberative techniques to engage the community in policy development. Aust New Zealand Health Policy 2008;5:16. 39 Kelaher M, Sabanovic H, La Brooy C, et al. Does more equitable governance lead to more equitable health care? A case study based on the implementation of health reform in Aboriginal health Australia. Soc Sci Med 2014;123:278–86. 40 Cleemput I, Christiaens W, Kohn L, et al. Acceptability and perceived benefits and risks of public and patient involvement in health care policy: a Delphi survey in Belgian stakeholders. Value Health 2015;18:477–83. 41 Kohler G, Sampalli T, Ryer A, et al. Bringing value-­based perspectives to care: including patient and family members in decision-­making processes. Int J Health Policy Manag 2017;6:661–8. 42 Liang L, Cako A, Urquhart R, et al. Patient engagement in hospital health service planning and improvement: a scoping review. BMJ Open 2018;8:e018263. 43 Snow ME, Tweedie K, Pederson A. Heard and valued: the development of a model to meaningfully engage marginalized populations in health services planning. BMC Health Serv Res 2018;18:1–13. 44 Wortley S, Street J, Lipworth W, et al. What factors determine the choice of public engagement undertaken by health technology assessment decision-­making organizations? J Health Organ Manag 2016;30:872–90. 45 Born K, Laupacis A. Public engagement in Ontario's hospitals-opportunities and challenges. Healthc Q 2012;15 Spec No:16–20. 46 Wait S, Nolte E. Public involvement policies in health: exploring their conceptual basis. Health Econ Policy Law 2006;1:149–62. 47 Celenza JF, Zayack D, Buus-­Frank ME, et al. Family involvement in quality improvement: from bedside advocate to system advisor. Clin Perinatol 2017;44:553–66. 48 Scholz B, Gordon S, Happell B. Consumers in mental health service leadership: a systematic review. Int J Ment Health Nurs 2017;26:20–31. 49 Pickin C, Popay J, Staley K, et al. Developing a model to enhance the capacity of statutory organisations to engage with lay communities. J Health Serv Res Policy 2002;7:34–42. 50 Djellouli N, Jones L, Barratt H, et al. Involving the public in decision-­ making about large-­scale changes to health services: a scoping review. Health Policy 2019;123:635–45. 51 Selva A, Sanabria AJ, Pequeño S, et al. Incorporating patients’ views in guideline development: a systematic review of guidance documents. J Clin Epidemiol 2017;88:102–12. 52 Armstrong MJ, Mullins CD. Value assessment at the point of care: incorporating patient values throughout care delivery and a draft taxonomy of patient values. Value Health 2017;20:292–5. 53 Kenny A, Farmer J, Dickson-­Swift V, et al. Community participation for rural health: a review of challenges. Health Expect 2015;18:1906–17. 54 Franx G, Dixon L, Wensing M, et al. Implementation strategies for collaborative primary care-­mental health models. Curr Opin Psychiatry 2013;26:502–10. 55 Kuhn KJ, Mickelsen LJ, Morimoto LN, et al. The use of patient and family Advisory councils to improve patient experience in radiology. AJR Am J Roentgenol 2016;207:965–70. 56 Scutchfield FD, Hall L, Ireson CL. The public and public health organizations: issues for community engagement in public health. Health Policy 2006;77:76–85. 57 Serapioni M, Duxbury N. Citizens' participation in the Italian health-­ care system: the experience of the mixed Advisory committees. Health Expect 2014;17:488–99. 58 Wright B. Voices of the vulnerable: community health centres and the promise and peril of consumer governance. Public Management Review 2015;17:57–71. 59 Azmal M, Sari AA, Foroushani AR, et al. Developing a conceptual model for the application of patient and public involvement in the healthcare system in Iran. Electron Physician 2016;8:2506–14. 60 Ivanova O, Dræbel T, Tellier S. Are sexual and reproductive health policies designed for all? vulnerable groups in policy documents of four European countries and their involvement in policy development. Int J Health Policy Manag 2015;4:663–71. 13 BMJ Open: first published as 10.1136/bmjopen-2021-050208 on 11 October 2021. Downloaded from http://bmjopen.bmj.com/ on April 1, 2022 by guest. Protected by copyright. 9 Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the patient requires care of the provider. Ann Fam Med 2014;12:573–6. 10 Hamilton C, Snow ME, Clark N, et al. Quality of patient, family, caregiver and public engagement in decision-­making in healthcare systems: a scoping review protocol. BMJ Open 2019;9:e032788. 11 Mitton C, Smith N, Peacock S, et al. Public participation in health care priority setting: a scoping review. Health Policy 2009;91:219–28. 12 Manafò E, Petermann L, Vandall-­Walker V, et al. Patient and public engagement in priority setting: a systematic rapid review of the literature. PLoS One 2018;13:e0193579. 13 Jiang S, Wan B, Lam M. A qualitative review of patient and family Advisory councils in cancer care. J Pain Manag 2017;10:59. 14 Boivin A, L'Espérance A, Gauvin F-­P, et al. Patient and public engagement in research and health system decision making: a systematic review of evaluation tools. Health Expect 2018;21:1075–84. 15 Dukhanin V, Topazian R, DeCamp M. Metrics and evaluation tools for patient engagement in healthcare Organization- and system-­ level DecisionMaking: a systematic review. Int J Health Policy Manag 2018;7:889–903. 16 Hamilton CB, Leese JC, Hoens AM, et al. Framework for advancing the reporting of patient engagement in rheumatology research projects. Curr Rheumatol Rep 2017;19:38. 17 Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2005;8:19–32. 18 Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. Implement Sci 2010;5:69. 19 Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-­ScR): checklist and explanation. Ann Intern Med 2018;169:467–73. 20 Covidence systematic review software. 2019 veritas health innovation. Melborne, Australia. Available: https://www.​covidence.​ org 21 Donabedian A. The quality of care. How can it be assessed? JAMA 1988;260:1743–8. 22 Hoenig H, Lee J, Stineman M. Conceptual overview of frameworks for measuring quality in rehabilitation. Top Stroke Rehabil 2010;17:239–51. 23 Scott AM, Sims R, Degeling C, et al. Developing and applying a deductive coding framework to assess the goals of Citizen/ Community jury deliberations. Health Expect 2019;22:537–46. 24 Harris C, Ko H, Waller C, et al. Sustainability in health care by allocating resources effectively (share) 4: exploring opportunities and methods for consumer engagement in resource allocation in a local healthcare setting. BMC Health Serv Res 2017;17:329. 25 Ocloo J, Matthews R. From tokenism to empowerment: progressing patient and public involvement in healthcare improvement. BMJ Qual Saf 2016;25:626–32. 26 Dalton J, Chambers D, Harden M, et al. Service user engagement in health service reconfiguration: a rapid evidence synthesis. J Health Serv Res Policy 2016;21:195–205. 27 Conklin A, Morris Z, Nolte E. What is the evidence base for public involvement in health-­care policy?: results of a systematic scoping review. Health Expect 2015;18:153–65. 28 Kreis J, Schmidt H. Public engagement in health technology assessment and coverage decisions: a study of experiences in France, Germany, and the United Kingdom. J Health Polit Policy Law 2013;38:89–122. 29 Hubbard G, Kidd L, Donaghy E, et al. A review of literature about involving people affected by cancer in research, policy and planning and practice. Patient Educ Couns 2007;65:21–33. 30 Cavet J, Sloper P. The participation of children and young people in decisions about UK service development. Child Care Health Dev 2004;30:613–21. 31 Abelson J, Forest P-­G, Eyles J, et al. Deliberations about deliberative methods: issues in the design and evaluation of public participation processes. Soc Sci Med 2003;57:239–51. 32 Malfait S, Van Hecke A, Hellings J, et al. The impact of stakeholder involvement in hospital policy decision-­making: a study of the hospital's business processes. Acta Clin Belg 2017;72:63–71. 33 Zakus JD, Lysack CL. Revisiting community participation. Health Policy Plan 1998;13:1–12. 34 Kemper C, Blackburn C, Doyle JA, et al. Engaging patients and families in system-­level improvement: a safety imperative. Nurs Adm Q 2013;37:203–15. 35 De Weger E, Van Vooren N, Luijkx KG, et al. Achieving successful community engagement: a rapid realist review. BMC Health Serv Res 2018;18:285. Open access 14 81 Ti L, Tzemis D, Buxton JA. Engaging people who use drugs in policy and program development: a review of the literature. Subst Abuse Treat Prev Policy 2012;7:47. 82 Carman KL, Mallery C, Maurer M, et al. Effectiveness of public deliberation methods for gathering input on issues in healthcare: results from a randomized trial. Soc Sci Med 2015;133:11–20. 83 van de Bovenkamp HM, Trappenburg MJ. Reconsidering patient participation in Guideline development. Health Care Anal 2009;17:198–216. 84 Restall G. Conceptualizing the outcomes of involving people who use mental health services in policy development. Health Expect 2015;18:1167–79. 85 Meetoo D. The setting of healthcare priorities through public engagement. Br J Nurs 2013;22:372–6. 86 Sharma AE, Knox M, Mleczko VL, et al. The impact of patient advisors on healthcare outcomes: a systematic review. BMC Health Serv Res 2017;17:693. 87 Tempfer CB, Nowak P. Consumer participation and organizational development in health care: a systematic review. Wien Klin Wochenschr 2011;123:408–14. 88 Thurston WE, MacKean G, Vollman A, et al. Public participation in regional health policy: a theoretical framework. Health Policy 2005;73:237–52. 89 Ferreira K, Hodges S, Slaton E. The Promise of Family Engagement: An Action Plan for System-­Level Policy and Advocacy. In: Culp AM, ed. Child and family advocacy: bridging the gaps between research, practice, and policy. New York, NY: Springer New York, 2013: 253–68. 90 Aceves-­Martins M, Aleman-­Diaz AY, Giralt M, et al. Involving young people in health promotion, research and policy-­making: practical recommendations. Int J Qual Health Care 2019;31:147–53. 91 Tierney E, McEvoy R, Hannigan A, et al. Implementing community participation via interdisciplinary teams in primary care: an Irish case study in practice. Health Expect 2018;21:990–1001. 92 Abelson J, Li K, Wilson G, et al. Supporting quality public and patient engagement in health system organizations: development and usability testing of the public and patient engagement evaluation tool. Health Expect 2016;19:817–27. 93 Boothroyd RI, Flint AY, Lapiz AM, et al. Active involved community partnerships: co-­creating implementation infrastructure for getting to and sustaining social impact. Transl Behav Med 2017;7:467–77. 94 Abelson J, Tripp L, Kandasamy S, et al. Supporting the evaluation of public and patient engagement in health system organizations: results from an implementation research study. Health Expect 2019;22:1132–43. 95 Salzer MS. Consumer empowerment in mental health organizations: concept, benefits, and impediments. Adm Policy Ment Health 1997;24:425–34. 96 Wakefield PA, Randall GE, Richards DA. Identifying barriers to mental health system improvements: an examination of community participation in assertive community treatment programs. Int J Ment Health Syst 2011;5:27. 97 Pizzo E, Doyle C, Matthews R, et al. Patient and public involvement: how much do we spend and what are the benefits? Health Expect 2015;18:1918–26. 98 Damschroder LJ, Aron DC, Keith RE, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Sci 2009;4:1–15. 99 Damschroder LJ. Clarity out of chaos: use of theory in implementation research. Psychiatry Res 2020;283:112461. 100 Glasgow RE, Harden SM, Gaglio B, et al. RE-­AIM planning and evaluation framework: adapting to new science and practice with a 20-­year review. Front Public Health 2019;7:64. Hamilton CB, et al. BMJ Open 2021;11:e050208. doi:10.1136/bmjopen-2021-050208 BMJ Open: first published as 10.1136/bmjopen-2021-050208 on 11 October 2021. Downloaded from http://bmjopen.bmj.com/ on April 1, 2022 by guest. Protected by copyright. 61 Grant S, Hazlewood GS, Peay HL, et al. Practical considerations for using online methods to engage patients in Guideline development. Patient 2018;11:155–66. 62 Wieczorek CC, Nowak P, Frampton SB, et al. Strengthening patient and family engagement in healthcare - The New Haven Recommendations. Patient Educ Couns 2018;101:1508–13. 63 South J, Fairfax P, Green E. Developing an assessment tool for evaluating community involvement. Health Expect 2005;8:64–73. 64 Terao M, Hoffman JM, Brilli RJ, et al. Accelerating improvement in children's healthcare through quality improvement Collaboratives: a synthesis of recent efforts. Curr Treat Options Pediatr 2019;5:111–30. 65 Oldfield BJ, Harrison MA, Genao I, et al. Patient, family, and community Advisory councils in health care and research: a systematic review. J Gen Intern Med 2019;34:1292–303. 66 Pagatpatan CP, Ward PR. Understanding the factors that make public participation effective in health policy and planning: a realist synthesis. Aust J Prim Health 2017;23:516–30. 67 Sarrami-­Foroushani P, Travaglia J, Debono D, et al. Implementing strategies in consumer and community engagement in health care: results of a large-­scale, scoping meta-­review. BMC Health Serv Res 2014;14:402. 68 Goodridge D, Isinger T, Rotter T. Patient family advisors’ perspectives on engagement in health-­care quality improvement initiatives: Power and partnership. Health Expect 2018;21:379–86. 69 Bombard Y, Baker GR, Orlando E, et al. Engaging patients to improve quality of care: a systematic review. Implement Sci 2018;13:98. 70 Young A, Menon D, Street J, et al. Exploring patient and family involvement in the lifecycle of an orphan drug: a scoping review. Orphanet J Rare Dis 2017;12:188. 71 Mulvale G, Chodos H, Bartram M, et al. Engaging civil society through deliberative dialogue to create the first mental health strategy for Canada: changing directions, changing lives. Soc Sci Med 2014;123:262–8. 72 Walsh E, Sheridan A. Factors affecting patient participation in clinical trials in Ireland: a narrative review. Contemp Clin Trials Commun 2016;3:23–31. 73 Fancott C, Baker GR, Judd M, et al. Supporting Patient and Family Engagement for Healthcare Improvement: Reflections on "Engagement-­Capable Environments" in Pan-­Canadian Learning Collaboratives. Healthc Q 2018;21:12–30. 74 Rifkin SB. A framework linking community empowerment and health equity: it is a matter of choice. J Health Popul Nutr 2003;21:168–80. 75 Josewski V. Analysing ‘cultural safety’ in mental health policy reform: lessons from British Columbia, Canada. Crit Public Health 2012;22:223–34. 76 Hanks CA. Community empowerment: a partnership approach to public health program implementation. Policy Polit Nurs Pract 2006;7:297–306. 77 Courtney R, Ballard E, Fauver S, et al. The partnership model: working with individuals, families, and communities toward a new vision of health. Public Health Nurs 1996;13:177–86. 78 Restall G, Cooper JE, Kaufert JM. Pathways to translating experiential knowledge into mental health policy. Psychiatr Rehabil J 2011;35:29–36. 79 Jansen MPM, Baltussen R, Bærøe K. Stakeholder participation for legitimate priority setting: a checklist. Int J Health Policy Manag 2018;7:973–6. 80 Rosenberg-­Yunger ZRS, Bayoumi AM. Evaluation criteria of patient and public involvement in resource allocation decisions: a literature review and qualitative study. Int J Technol Assess Health Care 2017;33:270–8.