Before using the tool, please note that you will be prompted to answer a series of questions related to your planned knowledge synthesis. We recommend that you conduct a preliminary search to gain background on the relevant literature.
Is your review question fixed or likely to be emergent?
The review question should be clear and focused.
The review question helps guide the review process and determines what kind of data you are collecting and how you will collect it.
Is your review question fixed or likely to be emergent?
Answer Response 1A: Fixed
A fixed question can have elements of the PICO (Population, Intervention, Comparison, and Outcome) approach that is used in a quantitative knowledge synthesis of interventions (e.g., systematic review and meta-analysis).
A fixed question is defined by the elements that serve to “anchor” a synthesis. For example, you could use these elements as keywords in your literature search to identify relevant studies and aggregate the findings.
Research question: “What are the factors that support (enablers) and inhibit (barriers) the implementation of interventions aimed at improving chronic disease care for Indigenous people within a primary health care setting?”
Population: Indigenous people with chronic disease
Intervention: Interventions aimed at improving chronic disease care
Research question: “What is the patient experience and treatment burden in the context of stroke care?” (N.B. “treatment burden” is a concept describing the self-care practices that patients with chronic disease must perform to enact management strategies and respond to the demands of health care providers and systems).
Population: Adults (≥18 years) diagnosed with at least one cardiovascular accident or stroke at any anatomic location
Intervention: Any care given in a care setting
Outcomes: any patient experience and treatment burden
Is your review question fixed or likely to be emergent?
Answer Response 1B: Emergent
An emergent question generally does not have a set of pre-defined parameters. We can restate an emergent review question as a review objective, which serves as a “compass” that offers a general direction for the conduct of the qualitative knowledge synthesis.
Research question: “To understand factors considered important by patients, caregivers and health care providers in contributing to tuberculosis medication adherence.”
Research question: “To assess the current available evidence, extract findings and highlight key themes that may help to guide midwifery and medical management, training of key healthcare workers and development of support services dealing with bereaved parents going through a stillbirth.”
Who is your primary audience? Select all that apply.
Increasing sophistication in the planning and conduct of knowledge synthesis projects has revealed how important it is to be familiar with requirements and expectations of the intended recipients of the review (i.e., knowledge users) and how review findings are intended to be used.
How are you planning to contribute to existing knowledge by doing the review? Select all that apply.
A review team should consider the assumptions that underpin the knowledge synthesis method and the extent to which these assumptions permit the goals of the review to be achieved.
How are you planning to contribute to existing knowledge by doing the review? Select all that apply.
Answer Response 3A: Synthesis
A synthesis provides a structured presentation of the knowledge in a domain of study that has already been made explicit by authors of studies included in the review.
The authors explicitly identify and structure knowledge of the factors influencing the transfer decision from the views of stakeholders involved in the decision and make the knowledge transparent.
How are you planning to contribute to existing knowledge by doing the review? Select all that apply.
Answer Response 3B: Adoption of a new perspective
By adopting a new perspective in doing a review, reviewers apply new angles, new perspectives or different macro-concepts, which have not been explicated previously.
The review used an iterative approach to the review process by treating the literature as unsettled and uncertain, applying new perspectives to iteratively make sense of findings from included studies in order to reach the points of saturation, and developing a model of the overall ‘motivational atmosphere’ in sport.
Although knowledge on athlete motivation had already been developed before the review, the authors explicitly frame it in a holistic context.
How are you planning to contribute to existing knowledge by doing the review? Select all that apply.
Answer Response 3C: Theory building
Knowledge syntheses, which contribute to theory building, commonly suggest new findings in the domain of interest, which still need to be validated in future research whereby novel research hypotheses and theories are suggested.
The authors identified determinants of independent active free play related to child characteristics, parental restrictions, neighborhood and physical environment, societal changes, and policy issues.
They created an ecological model depicting these factors, and the relationships therein. This model may be viewed as a contribution to theory building.
How are you planning to contribute to existing knowledge by doing the review? Select all that apply.
Answer Response 3D: Theory testing
When a certain number of empirical studies have accumulated, knowledge syntheses can use the study findings to test theories that generalize the study results. This can uncover hidden regularities or irregularities in a domain of study.
The authors suspected, and found, that treatment burden was influenced by a number of health system factors, at micro and macro organization levels. Moreover, patient knowledge deficits added to that burden, resulting in coping strategies that complicated their care.
The existence of varying findings from the included studies – of which the reviewers probably develop a tacit understanding before conducting the review – is externalized in the form of a key concept (i.e., treatment burden) and related theories (e.g., coping strategies) that the authors tested in the review.
How are you planning to contribute to existing knowledge by doing the review? Select all that apply.
Answer Response 3E: Identification of research gaps
Research gaps identify missing data or knowledge from a domain of study. By explicating research gaps, knowledge synthesis converts tacit knowledge into explicit understanding.
The authors summarized study findings by major factors affecting adherence to tuberculosis treatment, such as poverty and gender discrimination, and social context.
However, they identified numerous research gaps as findings of the included studies are limited by the quality and foci of the included studies.
How are you planning to contribute to existing knowledge by doing the review? Select all that apply.
Answer Response 3F: Provision of research agenda
By proposing a research agenda, knowledge syntheses may make explicit the approach to fill the knowledge gaps in a domain of study. The provision of a research agenda is an opportunity to complement research gaps with an analysis at the level of the knowledge of the research in the domain of study.
The review authors examine factors that enable or inhibit the implementation of interventions and conclude that future interventions should consider the findings of this review, as it provides an evidence-base that contributes to the successful design, implementation and sustainability of chronic disease interventions in primary health care settings intended for Indigenous people.
Will the likely included articles contain sufficient detail regarding the role of theory within your planned review?
In the context of this question, theory is defined as developing a conceptual understanding, and can take the form of a model or framework.
Theory can be integrated into a knowledge synthesis of qualitative studies at multiple levels.
Reviewers should be mindful of the philosophical foundations or the integrity of qualitative primary studies (i.e., ontology, epistemology, axiology, methodology).
In planning a review, a researcher may approach the knowledge synthesis of qualitative studies from differing epistemological stances. For example:
“A researcher synthesizing qualitative studies to inductively understand a social phenomenon may adopt a different method from the one synthesizing qualitative studies with the purpose of better understanding the effects of an empirically tested clinical intervention.”
“Alternatively, a researcher planning to synthesize qualitative research primarily as a means of generating theory may use a different approach from the one who intends to apply the results to answering a specific clinical question.”
Within a planned review, reviewers can ignore, acknowledge, generate, explore, or test theory.
Reviewers should undertake preliminary searches to assess the extent of the available qualitative evidence, including carefully reading some full-text reports of potentially relevant studies. This will guide decisions regarding how they will conduct the review.
Studies with rich data on concepts and theories are likely to sustain the generation and testing of theories, whereas studies reporting only the presence/absence of some concepts or phenomena of interest are not.
Will the likely included articles contain sufficient detail regarding the role of theory within your planned review?
Answer Response 4A: Yes
If you expect the included studies will be rich on concepts and theories, respond “Yes” to this question.
Will the likely included articles contain sufficient detail regarding the role of theory within your planned review?
Answer Response 4B: No
If you expect the included studies will report little data on concepts and theories, respond “No” to this question.
Will the likely included articles contain sufficient supporting detail to understand the study context?
As mentioned in Question 3, reviewers should preview the literature to assess the available qualitative evidence.
Relevant studies may only report data on “what works”.
Noyes et al. suggests that when reviewing the qualitative literature, a review team may be interested only in “what works” as opposed to why, for whom and in what circumstances.11 In this case, it would be less important whether the likely included studies report details pertaining to the context of ‘what works’.
Some relevant studies may report data on ‘what works” and provide supporting detail on 'what works for whom, in what contexts, and why’.
Booth et al. suggests that when reviews seek to address healthcare delivery, it may be useful to identify and include studies that explore barriers and facilitators to accessing healthcare, or the impact of specific barriers and facilitators on people, their experiences and behaviors.
Will the likely included articles contain sufficient supporting detail to understand the study context?
Answer Response 5A: Yes
If you expect the included studies will provide supporting detail on the study context, select “Yes” for this question.
Will the likely included articles contain sufficient supporting detail to understand the study context?
Answer Response 5B: No
If you do not expect the included studies to provide supporting detail on the study context, select “No” for this question.
What type of sampling method do you plan to use?
Sampling refers to the process of selecting a subset of studies for inclusion into a review. A sampling method can be comprehensive or purposive.
It has been argued that comprehensive sampling may be too time consuming because the searches often retrieve a large number of studies that are impractical to review and risk producing superficial findings that fail to go beyond the level of description.
As an alternative, researchers have proposed a more purposeful way of sampling papers (i.e., purposive sampling), which can mitigate time and resource constraints.
What type of sampling method do you plan to use?
Answer Response 6A: Comprehensive coverage (of all eligible studies)
Similar to a search strategy for a systematic review of interventions, comprehensive coverage refers to a thorough, objective, and reproducible search of sources in order to identify all eligible studies. This involves searching a variety of databases to exhaust the literature and minimize potential omission in the selection of included studies.
A literature search showed that the literature on volunteers’ motivations is extensive (e.g., >500 potentially relevant abstracts with title screening).
The authors used purposive sampling to reduce the number of included studies.
The reviewers used the hierarchy of needs as a theoretical framework for understanding individual motivations.
They conducted theoretical sampling – the process of selecting additional sample elements based on their potential for confirming or disconfirming the theoretical framework. In the end, they included five studies.
How many studies do you plan to review?
When data from studies are rich in details pertaining to their underlying theories (see Question 3) or thick in the description of the context of the interventions or social phenomena of interest (see Question 4), there are limitations in the number of studies that the review team can process in a coherent manner. This is especially true when the team wishes to synthesize findings of the included studies iteratively to generate, explore or test theory.
As a rule of thumb, the number of studies included in a qualitative evidence synthesis typically ranges from as little as three studies to >40 studies.1 Another suggestion is to include at least 12 studies to make a meaningful knowledge synthesis.
How many studies do you plan to review?
Answer Response 7A: Limited number of studies
Select this response if you intend to iteratively re-interpret findings from studies included in your review or if you anticipate that you will expend effort to make sense of findings from the included studies in order to explore or generate theory.
Maxwell et al. conducted a knowledge synthesis to address the question of "What are the postpatum depression experiences of women belonging to marginalized populations?"
They identified and refined themes from 12 qualitative research articles surveying postpartum depression experiences of marginalised women in North American countries, and themes were synthesized with inputs from all three reviewers.
How many studies do you plan to review?
Answer Response 7B: Large number of studies
Select this response if your review team only seeks to summarize the findings of the included studies as intended by the original authors.
Gibson et al. synthesised international evidence on the factors that enable or inhibit the implementation of interventions aimed at improving chronic disease care for Indigenous people.
Qualitative research findings were grouped into categories on the basis of similarity of meaning. In addition, quantitative research findings were presented in a narrative summary. The authors included 23 studies in their review.
Will your review be limited by any time or resource constraints?
Well-conducted knowledge syntheses such as systematic reviews provide valid evidence to inform decision making.22 However, systematic review of healthcare interventions can be time-consuming (e.g., one year to complete),23 labor-intensive (e.g., requires 1,139 person-hours and five reviewers),24 and expensive (e.g., costs >$100,000).
Various methods exist to expedite the conduct of reviews to inform health policy and systems decisions. The main challenge lies in accelerating review methods while maintaining robustness and transparency. Reviewers can enhance the timeliness of reviews by taking knowledge synthesis shortcuts (e.g., using one reviewer instead of two for study selection, critical appraisal, and data abstraction), using computer automation, and intensifying review steps (e.g., including many reviewers on the team).
Time constraints refer to restrictions on the amount of time a review team has to complete the review. Note that a review team may not have control over timelines. For example, knowledge users (e.g., policy makers) may determine the timelines if they commissioned the review.
Resource constraints refer to restrictions on human resources (e.g., limited number of reviewers, limited skills/expertise) and funding (e.g., money, software, and interlibrary loans).
The time it takes to complete a review is likely to depend on the review question and the size of the relevant body of literature. For the purposes of this tool, we use a cut-off of six months, but this should be considered as a general rule of thumb rather than an absolute value.
Will your review be limited by any time or resource constraints?
Answer Response 8A: Yes
If the timeframe to complete the review is <6 months, select “Yes” as your answer response.
If there are resource constraints in addition to your time constraints, select “Yes”.
Will your review be limited by any time or resource constraints?
Answer Response 8B: No
If neither of the above apply to your review, select “No” as your answer response.
Will your review team include members with experience in knowledge synthesis?
Knowledge synthesis requires the work of a team engaging in a review. A review team often includes review coordinators, librarians, reviewers, content experts and review methodologists.
Studies evaluating steps in the conduct of reviews provide evidence in support of the use of reviewers with experience in titles/abstracts screening, data abstraction and quality appraisal. An experienced reviewer typically has completed or been involved in several knowledge syntheses (e.g., three or more reviews).
Will your review team include members with experience in knowledge synthesis?
Answer Response 9A: Yes
Select this response if your review team consists of members who have experience in knowledge synthesis.
Will your review team include members with experience in knowledge synthesis?
Answer Response 9B: No
Select this response if your review team does not have experience in knowledge synthesis.
Will your review team include members with expertise in qualitative research?
Qualitative research involves any research that uses data that does not indicate ordinal values. It generally involves collecting and/or working with text, images, and/or sounds, and using different kinds of data collection and analysis techniques, as well as a diversity of theoretical and epistemological frameworks (e.g., narrative analysis, thematic analysis).
Methods derived from qualitative research are often also used in knowledge synthesis of qualitative studies (e.g., grounded theory, narrative analysis, thematic analysis). As such, for knowledge synthesis of qualitative studies, expertise in qualitative research is highly desirable.
Will your review team include members with experience in knowledge synthesis?
Answer Response 10A: Yes
Select this response if your review team consists of members who have expertise in qualitative research.
Will your review team include members with expertise in qualitative research?
Answer Response 10B: No
Select this response if your review team does not have expertise in qualitative research.