What is a realist synthesis?

In this section we briefly describe the realist synthesis method. The realist research question is often summarized as "What works for whom under what circumstances, how and why?" Realist inquiry is based on a realist philosophy of science and considers the interaction between context, mechanism and outcome. From a realist perspective, intervention X is not thought of as having effect size Y with confidence interval Z. Rather, intervention X (for example, a program introduced by policymakers who seek to create a particular outcome) alters context (for example, by making new resources available), which then triggers mechanism(s), which produce both intended and unintended outcomes. Intervention X may work well in one context but poorly or not at all in another context.

Realist inquiry seeks to unpack the context - mechanism - outcome relationship, thereby explaining examples of success, failure and various eventualities in between. Theoretical explanations of this kind are referred to as "middle-range theories" (that is, ones which "...involve abstraction... but [are] close enough to observed data to be incorporated in propositions that permit empirical testing" [8].

The basis of realist inquiry is a realist philosophy, whose key tenets are as follows:

1. There is a [social] reality that cannot be measured directly (because it is processed through our brains, language, culture and so on), but can be known indirectly.

Realism thus sits, broadly speaking, between positivism (‘there is a real world which we can apprehend directly through observation') and constructivism (‘given that all we can know has been interpreted through human senses and the human brain, we cannot know for sure what the nature of reality is').

2. Social programs (including complex interventions) may change the macro social context (for example, by introducing legislation). They may also change the resources or opportunities available to participants and, in that sense, change the meso- or micro-level context for those participants.

3. To understand the relationship between context and outcome, realism uses the concept of mechanisms, one definition of which is "...underlying entities, processes, or [social] structures which operate in particular contexts to generate outcomes of interest" [9].

In common with other theory-driven review methods, the realist approach offers the potential for insights that go beyond the narrowly experimental paradigm of the randomized controlled trial [10-12]. It can do so in relation to complex, complicated or simpler interventions (for example, even a simple intervention, such as a drug, is prescribed, dispensed and taken - or not - in a particular social, cultural and economic context).

"Realist synthesis" was first described by Ray Pawson in 2002 [13], updated in an ESRC (Economic and Social Research Council) commissioned monograph in 2004 [14], published as a book in 2006 [1] and summarizsed in a short methods paper in 2005 [15]. Since this paper is deliberately focused on publication standards, we strongly recommend that those unfamiliar with the realist approach consult these or other relevant methodological sources.

A realist synthesis (or realist review - these terms are synonymous) applies realist philosophy to the synthesis of findings from primary studies that have a bearing on a single research question or set of questions. Methodologically, reviewers may begin by eliciting from the literature the main ideas that went into the making of a class of interventions (the program theory). This program theory sets out how and why a class of intervention is thought to 'work' to generate the outcome(s) of interest. The pertinence and effectiveness of each constituent idea is then tested using relevant evidence (qualitative, quantitative, comparative, administrative and so on) from the primary literature on that class of programs. In this testing, the ideas within a program theory are re-cast and conceptualized in realist terms.

For each idea, reviewers seek out the contextual (C) influences that are hypothesized to have triggered the relevant mechanism(s) (M) to generate the outcome(s) (O) of interest. Synthesis consists of comparing 'how the programme was supposed to operate' to the 'empirical evidence on the actuality in different situations' - all along C-M-O lines. Analytic purchase comes from the ability to describe and understand the many contingencies that affect the likelihood of such interventions generating their intended outcomes. This in turn provides guidance about what policy makers or practitioners might put in place to change the context or provide resources in such a way as to most likely trigger the right mechanism(s) to produce the desired outcome.

Description of panel and items

In all, we recruited 37 individuals from 27 organizations in 6 countries. These comprised: researchers in public or population health researchers (8); evidence synthesis (6); health services research (8); international development (2); education (2); and also research methodologists (6), publishing (1), nursing (2) and policy and decision making (2). In round 1, 22 Delphi panel members provided suggestions of items that should be included in the publication standards. In rounds 2 and 3 our panel members were asked to rate each potential item for relevance and clarity. The response rates across all items for rounds 2 and 3 were 93% and 89%, respectively. Consensus was reached within three rounds on both the content and wording of 19 items within the publication standards. Table ​Table11 provides an overview of these items.

Scope of the publication standards

These publication standards are intended to help researchers, authors, journal editors, and policy and decision makers to know and understand what should be reported in the write-up of a realist synthesis. They are not intended to provide detailed guidance on how to conduct such a synthesis; for this, we direct interested readers to summary articles [15,22] or various publications on methods [1,11,14,23]. This publication standard applies only to realist syntheses. A list of publication guidelines for other review methods can be found on the EQUATOR Network's website [24], but at present none of these relate specifically to realist syntheses. As part of the RAMESES project we are also developing quality standards and training materials for realist syntheses, which will be submitted as a separate publication. Publication standards for meta-narrative reviews (also covered in the RAMESES project) have been addressed in a separate article.

How to use these publication standards

The layout of this document has drawn on previous methodological publications and, in particular, on the 'Explanations and Elaborations' document of the PRISMA statement [18]. Each item is followed by an example drawn from published reviews and a rationale for its inclusion. The purpose of the example text is to illustrate how an item might be reported in a write up. However, potentially relevant contextual information may have been omitted, so it may be necessary to consult the original paper from which the example text was drawn. The standards set out what might be expected for each item, but authors will still need to exercise judgement about how much information to include. The purpose of the details reported should be to ensure that the description and explanation provided is coherent and plausible, both against the guidance set out within an item and for the overall purpose of the realist synthesis.

While this publication standard is modeled on the PRISMA statement, the items within are not identical. This publication standard, developed to apply only to realist syntheses, has some overlap with the PRISMA statement. Items 1 to 3, 15, 16 and 19 in this statement broadly match the purpose of items 1 to 3, 24, 25 and 27 in the PRISMA statement. For items 4 to 14, while there is some overlap in purpose with some PRISMA statement items, different or additional reporting is needed due to the nature of realist syntheses. Other items (5, 12, 13, 15, 16, 19 and 23) in the PRIMSA statement have no equivalent in the RAMESES publication standards for realist reviews.

The order in which items are reported may vary. Realist syntheses are not 'linear' reviews. Some of the processes that are listed may legitimately take place in parallel or have to be revisited at a later date as a review progresses. As a general rule, if a recommended item is excluded from the write-up of a realist synthesis, a justification should be provided.

The RAMESES publication standards for realist syntheses

Example

"Human resource management interventions to improve health workers' performance in low and middle income countries: a realist review." [25]

Explanation

Our background searching has shown that some realist reviews are not flagged as such in the title and may also be inconsistently indexed and, hence, are more difficult to locate during searching. The terms 'realist synthesis' and 'realist review' are both in widespread use. We asked our Delphi panel if they had a preferred term - 'realist synthesis' or 'review'. No consensus was reached by our Delphi panel on whether 'review' or 'synthesis' should be the preferred term, and there seemed no good reason to impose one or other term.

Example

Explanation

Apart from the title, an abstract is the only source of information accessible to searchers unless the full paper is obtained. The information in it must allow reviewers and/or users to decide if the review is relevant to their needs.

Introduction section

The following items should be reported in the introduction section.

Example

"A number of reviews on the subject have tried to examine evidence to improve the operationalization of interventions by CHWs [community health workers], including for child health. Lehmann et al. (Reference x1) and Lewin et al. (Reference x1) have reviewed evidence on CHW interventions in LMIC [low-middle income countries] and Haines et al. (Reference x1) have particularly so for child health. Lewin et al. (Reference x1) found lay health workers to be effective in specific areas in child health, when compared to usual care. Haines et al. (Reference x1) highlight the contextual nature of CHW's performance. Both caution that CHW interventions are not the panacea for all that ails the health systems in LMIC and that large scale CHW programmes should be initiated with great caution. Both raise questions about the applicability of findings to different settings and about the conditions under which CHW interventions should be implemented." [27]

Explanation

As with all research, a background section explaining what is already known and what the researchers considered to be the 'knowledge gaps' is a helpful orientation.

Example

"The overriding question for the review was: Does moving from high-poverty neighborhoods to lower-poverty neighborhoods improve health? More specifically: What were the key health outcomes? Who experienced these outcomes? What appeared to be the mechanisms and associated context leading to the outcomes? As the review proceeded, it became clear that one of the only relatively consistent and statistically significant positive health outcomes was an improvement in mental health for adult women, children and adolescent girls. In this paper a review of mental health outcomes of MTO [Moving To Opportunity] is presented, along with some insights about the mechanisms and contexts through which the intervention appears to have impacted mental health." [28]

Explanation

A realist research question contains some or all of the elements of 'What works, how, why, for whom, to what extent and in what circumstances, in what respect and over what duration?' and applies realist logic to address the question (see Item 11).

Because a realist synthesis may generate a large number of avenues that might be explored and explained, and because resources and timescale are invariably finite, the expectation is that the review must be 'contained' by progressively focusing both its breadth (how wide an area?) and depth (how much detail?). This important process may involve discussion and negotiation with, for example, content experts, funders and/or users. It is typical and legitimate for the synthesis' objectives, question and/or the breadth and depth of the review to evolve as the review progresses. How and why it evolved is usually worth reporting.

Methods section

The following items should be reported in the methods section.

Example

"As the review progressed we became aware of various data suitability limitations (see Discussion) and the emergence of two prominent demi-regularities prompted us to narrow our review focus to the two candidate theories discussed below." [29]

Explanation

A realist synthesis can (and, in general, should) evolve over the course of the review. For example, changes to the research question or its scope are likely to have an impact on many of the synthesis' subsequent processes. However, this does not mean the synthesis can meander uncontained. An accessible summary of what was originally planned (for example, as described in an initial protocol) and how and why this differed from what was done should be provided as this may assist interpretation.

Example

"Previous reviews sought to understand PR [participatory research] and provide practical recommendations (References x6) and to assess the value of PR to research goals, health status, and systems change (References x6). Nonetheless, the assessment of outcomes remains weak (Reference x4), partly because the methodologies used have generally failed to embrace the complexity of programs or address mechanisms of change (Reference x1). ...

To handle such complexity, we chose a realist approach (Reference x1) because it provides a rationale and tools for synthesizing complex, difficult-to-interpret evidence from community-based programs." [30]

Explanation

Realist synthesis is a theory-driven method that is firmly rooted in a realist philosophy of science. It places particular emphasis on understanding causation (in this case, understanding how programs and policies generate outcomes through human decisions) and how causal mechanisms are shaped and constrained by social context. This makes it particularly suitable for reviews of certain topics and questions - for example, complex social programs that involve human decisions and actions. It also makes realist synthesis less suitable than other review methods for certain topics and questions - for example, those which seek primarily to determine the average effect size of a simpler intervention administered in a single or limited range of conditions. In our analysis of 37 published realist syntheses, the most common limitation was inadequate engagement with realist explanatory principles and the implications these have, first, for understanding programs and how they work, and second, for cumulating evidence and explanation.

Some realist syntheses published to date have deliberately adapted the method as first described by Pawson. Sometimes, adaptations may be entirely justifiable, but at other times they may indicate a poor grasp of realist methodology. To enable judgement to be made on adaptations, the description and rationale for adaptations should be provided. Such information will allow criticism, debate and counter criticism among review teams and users on the suitability of such adaptations, and may well facilitate methodological development.

Example

"To develop our framework on the threats to the programme theory of public health legislation we started out by conducting a rapid review of broad areas of public health legislation (covering everything from gun amnesties to food labelling) trying to uncover what had been the sticking points in legislation and how (if at all) they had been circumvented. This outline review led to the construction of a provisional framework for reviewing the family of legislative interventions (as described in Figure ​Figure1).1). .... . Beginning with this framework and through discussions (and with reference to other interested stakeholders) we focused on a subset of themes that seemed most relevant in respect to the intervention in question. In our case, we deliberately sought input from the NICE officer seconded to our project." [26]

Open in a separate window

Figure 1

Flow diagram illustrating search process and article disposition [26].

Explanation

This step is used to build an understanding of the topic area. For example, this step may be used to identify provisional program theories, the names/titles of programs within scope and key authors in the area. Initial attempts to make sense of a topic area may involve informal 'browsing' of the literature and also consulting with experts and stakeholders.

Example

"...the literature search was iterative and ongoing throughout the project. An initial search was conducted of various academic databases, such as Academic Search Premier, Arts and Humanities Citation Index, Canadian Research Index, as well as through various search engines, such as Prowler, Novanet, Google and Google Scholar. Search terms included: Moving to Opportunity [MTO]; housing intervention; housing mobility; housing health effects; low-poverty neighborhood/community; high-poverty neighborhood/community; neighborhood/community health; poverty neighborhood/community; poverty community effects; poverty housing; poverty health; and housing health. A "snowball" approach was used in which one reference led to others. Other evaluations were revealed through correspondence with Dr. Jeffrey Kling, one of the principal MTO researchers." [28]

Explanation

Searching should be guided by the objectives and focus of the synthesis, and revised iteratively in the light of emerging data. Data relevant to a realist synthesis may lie in a broad range of sources that may cross traditional disciplinary, program and sector boundaries. The search phase is thus likely to involve searching for different sorts of data, or studies from different domains, with which to test different aspects of any provisional theory.

Search methods using forward and backward citation tracking may be particularly valuable in finding the documents necessary to develop and then test provisional theories. Realist syntheses do not exclude sources solely on the basis of their study design; hence, 'methodological filters' (for example, to identify randomized controlled trials) may add little to the search and could potentially miss relevant papers.

Searching is likely to be iterative because, as the synthesis progresses, new or refined elements of theory may be required to explain particular findings, or to examine specific aspects of particular processes. As new elements of theory are included, searches for evidence to support, refute or refine those elements may be required. If undertaken, the process used for any such additional searches should be clearly documented. A single pre-defined search is unlikely to be sufficient and may suggest insufficient reflection on emerging findings.

Sufficient detail should be given to enable the reader to judge whether searching was likely to have located sources needed for theory building and/or testing.

Example

"Three tools were developed (for identification, selection, and appraisal) in March, June, and October 2009, respectively. Modifications were made during each stage after piloting. Each stage processed a different type of data: citations in identification; full-text papers in selection; and sets of publications in appraisal.

...The identification tool consisted of three questions. This step funnelled the number of citations from 7,167 to 594.

The librarian (JH) retrieved the 594 full-text papers, which were read by two independent reviewers, using a selection tool initially comprised of six questions in June 2009, with an additional two questions added in October 2009. ...

Two hundred articles remained from 594 after filtering them through the selection tool. Due to the complexity of the dataset, we decided at this stage to further limit the scope of our review to community-based settings, and to participatory interventions. Our rationale was that: PR in all forms (community-based PR, organizational PR, action research) was too diverse to be assessed within one review; the complexity of PR benefits from community-based research provided a manageable set of studies; intervention research demonstrated more complexity of outcomes than non-intervention research, and would be best suited for analysis using realist review methods; and the pool of studies needed to be reduced to a manageable size for an in depth realist synthesis (analysis). Adding two questions reduced the pool to 83 studies.....

Contact with principal investigators of all full-text papers retained after selection was undertaken because descriptions of programs, methods and findings of PR interventions were found to be commonly described across a number of publications pertaining to the same intervention. It was thus necessary to confirm that we had complete sets of papers in order to fairly appraise projects according to the realist review approach. ... For each study, we then sent our list of papers to the corresponding author or PI, and asked them to confirm that we had the complete set, or to send us additional documents. ... Only those sets of studies in which the contacted researcher responded to our request were retained for appraisal.

.... The appraisal tool consisted of three questions. An additional 11 sets were eliminated after screening with the tool below, which left a total of 23 sets, comprising 276 documents that were retained for synthesis. See Appendix 4: ... for a complete breakdown of the number of cases retained at each stage." [30]

Explanation

Realist synthesis is not a technical process - that is, following a set protocol will not guarantee that a review will be robust. Rather, it requires a series of judgements about the relevance and robustness of particular data for the purposes of answering a specific question.

Within any document, there may be several pieces of data that serve different purposes, such as helping to build one theory, refining another theory and so on. Therefore, the selection (for inclusion or exclusion) and appraisal of the contribution of pieces of data within a document cannot be based on an overall assessment of study or document quality. An appraisal of the contribution of any section of data (within a document) should be made on two criteria:

• Relevance - whether it can contribute to theory building and/or testing; and

• Rigor - whether the method used to generate that particular piece of data is credible and trustworthy.

A wide range of documents can potentially contribute to a realist synthesis. For example, outcome and impact studies, qualitative interviews, ethnography, questionnaire surveys, mixed-method case studies, and close reading of policies, business plans, websites, project initiation documents and 'gray literature' write-ups of programs may all contribute in different ways of identifying and elucidating program theories. Because of this range and realist review's focus on relevance and rigor, it can initially be difficult to 'whittle down' the number of documents that are potentially eligible for inclusion in a review. This process can only occur as the data sources are analyzed in detail. Thus, in practice, the selection and appraisal stage may need to run in parallel with the analysis stage.

It is unlikely that authors will be able to provide an in-depth description of each decision involved, but the broad processes used to determine relevance and assess rigor (for example, using quality standards appropriate to particular kinds of research to appraise documents or sections of documents; discussion and/or debate within a review team of a document's findings; or consulting experts about technical aspects of methods or findings) should be described. While the description of the processes followed will not allow the reader to draw firm conclusions about judgements made, it will give an indication of the coherence, plausibility and appropriateness of the processes used to inform those judgements.

Example

"In order to identify key elements of importance to the success or failure of an intervention in a certain context using a realist perspective, information was gathered on the intervention, the context and the actual "working of the intervention" or the mechanisms. As we intended to discuss the strength of the evidence and the usefulness of the application of realist principles to already published studies, we developed a process of data analysis that was comprehensive and as objective and transparent as possible. Therefore, a data analysis matrix was developed by the team of authors (see Annex 2). During the development of this matrix, the team extensively discussed and defined terms (such as context, mechanisms and outcome) and evaluation levels (such as process, output and outcome)." [31]

Explanation

In a realist synthesis, data extraction assists analysis and synthesis. Reporting on what was extracted and why can add to the transparency of the synthesis process.

The extracted data may consist of descriptions (for example, of the detail of what was done in a program), findings (for example, cure rates, mortality) or explanations about how and why the program may have worked in particular contexts. Of particular interest to the realist reviewer are data that support the use of realist logic to answer the review's question(s) - for example, data on context, mechanisms and outcome configurations, demi-regularities, middle-range and/or program theories. Realist synthesis is used for a wide range of research questions, so it is impossible to be prescriptive about what data should be extracted. However, the link between the research question and the category of data extracted should be clear.

Example

"Data synthesis was undertaken either by RP and/or GW and synthesis results were regularly shared and discussed within the review team to ensure validity and consistency in the inferences made. Specifically (where relevant), we attempted to identify prominent recurrent patterns of contexts and outcomes (demi-regularities) in the data and then sought to explain these through the means (mechanisms) by which they occurred. For example, we noted that in our included articles self-reported public support for a ban on smoking in vehicles carrying children was often found to be high amongst smokers. During data synthesis we would then aim to provide an explanation of this demi-regularity through the identification of mechanism(s). As we delved further into our included articles and beyond (through our aforementioned purposive searching) for an explanation, data emerged that smokers harboured within them the wish to want to protect children from harm and also regret at having started smoking. We interpreted these as (realist) mechanisms and, for the former, were able to find substantive (middle-range) theory in the form of the Millean principle [Reference x1] to explain its interaction with context to influence outcomes. When additional studies were sought to enable programme theory testing, data handling processes .... were repeated." [26]

Explanation

In a realist synthesis, the analysis and synthesis processes occur iteratively and may be sequential or in parallel. At the center of any realist analysis is the application of a realist philosophical 'lens' to data. A realist analysis of data specifically seeks to analyze data using realist concepts. Specifically, realism adheres to a generative explanation for causation - that is, an outcome (O) of interest was generated by relevant mechanism(s) (M) being triggered in context (C). Within or across the included documents, recurrent patterns of outcomes (or demi-regularities) and their associated mechanisms and contexts (CMO configurations) are likely to occur.

During synthesis the goal is to make sense of the analyzed data using theory, at one of two levels. First, theory (or theories) may be sought, developed and/or refined to explain how it is that a program (or part of a program) achieves its outcomes (that is, the mechanism(s) operating within a program) and the contexts in which those mechanisms do and do not fire. This provides a realist program theory. Second, theory (or theories) may be sought, developed and/or refined to explain, at a somewhat more general level, the pattern of contexts, mechanisms and outcomes. A full realist analysis addresses both these levels and attempts to make sense of the relationship between these two levels. Syntheses which address only one level may also be considered realist syntheses assuming that they apply and demonstrate application of a realist philosophy of science. The level(s) of analysis chosen will depend on the review's focus. The theories used may have been developed and/or refined from the data and/or be refinement of existing substantive theory.

The key analytic process in realist review involves iterative testing and refinement of theoretically based explanations using empirical findings in data sources. Reviewers may draw on any appropriate analytic techniques to undertake this testing. Explanation and justification for the choice of techniques should be provided.

Ideally a description should be provided on how all the individuals involved in the review have been involved in the analysis and synthesis processes, and how these evolved as the review took shape.

Results section

The following items should be reported in the results section.

Example

"See Figure ​Figure1:1: Flow diagram illustrating search process and article disposition." [26]

Explanation

A flow diagram provides an accessible summary of the sequence of steps and gives an indication of the volume of data included and excluded at each step.

Example

"Additional File 1 summarises ..., the context, the intervention, the mechanisms triggered and the reported outcomes. Additional File 1 shows that in all the trials, more than one type of intervention was applied to improve CHWs [community health workers] performance. It also shows that the outcomes are reported not in terms of CHW performance, but rather in terms of the consequences of their performance on specific health outcomes." [27]

Explanation

A clear summary of the characteristics of included sources can add to the transparency of the synthesis and some characteristics may help readers judge the coherence and plausibility of inferences. Examples of possibly relevant characteristics of documents that may be worth reporting include, where applicable: full citation, country of origin, study design, summary of key main findings, use made of document in the synthesis and relationship of documents to each other (for example, there may be more than one document reporting on an intervention). While considering specific requirements of any particular publication, reviewers may wish to tabulate key characteristics.

Example

"Using this theoretical concept, we hypothesized that equitable partnerships, with the stakeholders' participation throughout the project, succeed largely through synergy. Through the synthesis process using CMO configuring, we refined the theory by demonstrating that synergy is both an outcome and a context for partnership development - so that when synergy generated positive outcomes (e.g., enhanced trust or improved data collection), those outcomes generated new synergy. Expanding this logic, we demonstrated how partnership synergy created momentum over time, producing resilience in the face of obstacles as well as sustaining health-related goals, extending programs and infrastructure, and creating new and unexpected ideas and activities." [30]

Explanation

The defining feature of a realist synthesis is the nature of the theory(ies) it offers. Such a theory explains why a social program/intervention generates particular outcomes in particular contexts, in terms of one or more mechanisms - that is how the program's infrastructure and resources trigger particular decisions or behaviors in human participants. Program theories are usually 'middle-range' - that is, specific enough to generate propositions that can be tested about aspects of the program but sufficiently abstract to be applicable to other programs. Mechanisms are contingent: they are causal processes that have a tendency to occur in a particular set of conditions, but which do not always occur (because the circumstances have to be right for any particular mechanism to operate, and because many mechanisms can operate concurrently, sometimes cancelling each other out).

The validity of a review which is described as 'realist' and which talks about program theories or mechanisms but which expresses these as simple and linear relationships between variables should be questioned.

The findings of a realist synthesis consist largely of inferences about the links between context, mechanism and outcome and the theory(ies) that seek to account for these links. It is important that where inferences are made these are clearly articulated. Where possible, especially for key findings, it is important to include an explanation to show how these inferences were arrived at.

Transparency of the synthesis process can be demonstrated, for example, by including such things as a detailed worked example, verbatim quotes from primary sources, and (if appropriate) an exploration of disconfirming data (that is, findings which appeared to refute the program theory but which, on closer analysis, could be explained by other contextual influences).

When presenting inferences about context-mechanism-outcome configurations, reviewers should be clear about what they have categorized as context, what as mechanism and what as outcome. In a realist synthesis a mechanism involves the interaction between particular inputs (or resources) and human reasoning, which produces a particular outcome (or not).

More than one piece of data might be needed to support an inference. It is sometimes appropriate to build the argument for an inference as an unfolding narrative in which successive data sources increase the strength of the inference [32]. Provide enough details about each data item to identify its source and enable readers to make judgements about its relevance and rigor.

Discussion section

The following Items should be reported in the discussion section.

Example

"This realist review of 249 primary studies has produced two key findings which are important, if somewhat unsurprising. First, Internet-based courses must engage their target group of learners to use the technology. This is likely to occur only if the technology is perceived as 'useful' (e.g.increases access to learning or saves time) and 'easy to use', though benefits in the former can outweigh challenges in the latter. Second, 'interactivity' is highly valued by learners. Learners wanted to be able to enter into a dialogue with the course tutor, fellow students and/or a virtual tutorial and obtain ongoing feedback on their understanding and performance." [29]

Explanation

In order to place the findings in the context of the wider literature and any specific policy need, it is necessary to summarize briefly what has been found. This section should be succinct and balanced, explaining the relevance of one or more key theories that emerged from the analysis and highlighting the strength of evidence for the main inferences. This should be done with careful attention to the needs of the main users of the synthesis.

Example

"We explicitly chose to do a realist review of the RCTs [randomized controlled trials] to see what they could additionally yield. While the CHWs [community health workers] were an important component of the interventions being tested in the RCTs, none of the RCTs under review explicitly focused on performance of the CHW as an outcome. The RCTs under review offered a fair amount of information about the interventions, only some information about context - allowing us to formulate only generic hypotheses. ...

... Authors seldom described or discussed the mechanisms that explained their study outcomes. We realise that the RCT design, the exacting reporting requirements and word limits of journals, restrict authors from sharing all their operational experiences. In addition RCTs tend to report average effects and not differential effects of interventions, and less so of the context and rarely of the mechanisms triggered by their interactions. This makes the RCTs less useful for answering the questions regarding how interventions work. These generic hypotheses seem to be recurring in the literature, however they have not been explicitly tested across contexts." [27]

Explanation

Realist synthesis may be constrained by time and resources, by the skill mix and collective experience of the research team, by the scope of the review's questions or objectives and/or by anticipated or unanticipated challenges in the data. These should be made explicit so that readers can interpret the findings in the light of them. A common challenge in realist synthesis is that in order to focus the synthesis, some material is omitted at each successive stage. Some aspects of the topic area, therefore, end up being reviewed in detail and rich explanatory insights produced for these. Other aspects are neglected (relatively or absolutely). It is thus inevitable that in generating illumination, the synthesis will also cast shadows. These should be highlighted in the discussion so as to indicate areas where other syntheses might focus.

Strengths and/or limitations associated with any modifications made to the synthesis process should also be reported and justified.

Example

"We were unable to find any comparable attempt at providing an evidence-based-policy framework such as ours. However, we acknowledge that some sections of our framework may be found in sources we have not uncovered and also as tacit knowledge within the heads of seasoned practitioners (e.g. advocates or legislators). We do however hope that our attempts to develop and test it on our one 'case study' will make a primordial tool that will be useful to policy and decisions makers less well versed in the arena of public health legislation." [26]

Explanation

Comparing and contrasting the findings from a synthesis with the existing literature may help readers to put these into context. For example, this item might cover questions such as: How does this synthesis compare to other reviews (for example, were they theory-driven?); What does this synthesis add?; Which body of work in particular does it add to?; Has this synthesis reached the same or different conclusion to previous reviews?; and Has it answered a question previously identified as important in the field?

Example

"Our realist review was based on a housing intervention in the United States, but the results can potentially be applied to urban centers in other nations that implement housing interventions that involve moving families. When a family moves, the experience is likely to be different for each member of the household, and differences in mental health outcomes of moving may occur (Reference x1). All communities, rich or poor, and irrespective of geographic location, should be viewed as complex systems, and as composed of people with social relationships that influence the functioning and health of community members." [28]

Explanation

A clear line of reasoning is needed to link the findings (Results section) with the implications (Discussion and/or Conclusion). If the synthesis is small and preliminary, or if the coherence and plausibility of evidence behind the inferences is weak or moderate, statements about implications for practice and policy should be appropriately guarded.

If recommendations are given, these should take into account the focus of the synthesis and needs of the intended audience and be presented appropriately. The explanations in realist analysis are highly dependent on contextual influences. It follows that recommendations must be contingent (for example, only under certain contexts will a particular mechanism be triggered to generate the desired outcome) rather than statements that X should or should not be done.

Example

"We gratefully acknowledge a financial contribution from the Dutch Development Cooperation (DGIS)." [25]