Skip to main content

Engaging patients in de-implementation interventions to reduce low-value clinical care: a systematic review and meta-analysis



Many decisions regarding health resource utilization flow through the patient-clinician interaction. Thus, it represents a place where de-implementation interventions may have considerable effect on reducing the use of clinical interventions that lack efficacy, have risks that outweigh benefits, or are not cost-effective (i.e., low-value care). The objective of this systematic review with meta-analysis was to determine the effect of de-implementation interventions that engage patients within the patient-clinician interaction on use of low-value care.


MEDLINE, EMBASE, and CINAHL were searched from inception to November 2019. Gray literature was searched using the CADTH tool. Studies were screened independently by two reviewers and were included if they (1) described an intervention that engaged patients in an initiative to reduce low-value care, (2) reported the use of low-value care with and without the intervention, and (3) were randomized clinical trials (RCTs) or quasi-experimental designs. Studies describing interventions solely focused on clinicians or published in a language other than English were excluded. Data was extracted independently in duplicate and pertained to the low-value clinical intervention of interest, components of the strategy for patient engagement, and study outcomes. Quality of included studies was assessed using the Cochrane Risk of Bias tool for RCTs and a modified Downs and Black checklist for quasi-experimental studies. Random effects meta-analysis (reported as risk ratio, RR) was used to examine the effect of de-implementation interventions on the use of low-value care.


From 6736 unique citations, 9 RCTs and 13 quasi-experimental studies were included in the systematic review. Studies mostly originated from the USA (n = 13, 59%), targeted treatments (n = 17, 77%), and took place in primary care (n = 10, 45%). The most common intervention was patient-oriented educational material (n = 18, 82%), followed by tools for shared decision-making (n = 5, 23%). Random effects meta-analysis demonstrated that de-implementation interventions that engage patients within the patient-clinician interaction led to a significant reduction in low-value care in both RCTs (RR 0.74; 95% CI 0.66–0.84) and quasi-experimental studies (RR 0.61; 95% CI 0.43–0.87). There was significant inter-study heterogeneity; however, intervention effects were consistent across subgroups defined by low-value practice and patient-engagement strategy.


De-implementation interventions that engage patients within the patient-clinician interaction through patient-targeted educational materials or shared decision-making tools are effective in decreasing the use of low-value care. Clinicians and policymakers should consider engaging patients within initiatives that seek to reduce low-value care.


Open Science Framework (

Peer Review reports


Clinical interventions that lack efficacy, have risks that outweigh benefits, or are not cost-effective constitute low-value care [1]. In the USA, overuse of low-value practices is estimated to cost upwards of 100 billion dollars annually [2] and is associated with adverse events, poor patient outcomes, and downstream use of healthcare resources [3, 4]. Reducing low-value care is therefore imperative for high-quality, sustainable healthcare. Researchers, governments, and public campaigns have commanded attention about low-value care by classifying hundreds of tests and treatments as low-value [5,6,7,8]. Yet, this increased awareness about low-value care has not translated into a reduction in use [9,10,11]. The use of low-value care may be reduced or stopped through de-implementation, which is defined as a planned process that uses targeted strategies such as education, incentives, or audit and feedback [12]. There is a need to understand how to achieve meaningful reduction in low-value care through comprehensive de-implementation interventions that acknowledge the complexity of this issue and appropriately engage researchers, decision-makers, clinicians, and patients.

Patients are directly involved in and impacted by low-value care and may play a pivotal role in solutions to reduce its use [13, 14]. However, the potential effects of patient engagement in de-implementation initiatives are complex. On the one hand, clinicians often cite patient demand for tests and treatments as a barrier to reducing low-value care [15, 16], while on the other hand, patients may experience mental and/or physical harm from unnecessary tests and treatments [4]. Making this question even more complex is the fact that inclusion of patients within initiatives to reduce low-value care may impute a mistrust within the patient-clinician interaction and create a false sense of the ubiquity of low-value care within the practice of medicine, while this is indeed not the case [2]. So, while it is assumed that patients should be engaged in any initiative that seeks to increase or decrease the use of clinical care, from the perspective of de-implementing low-value care, the risk/benefit ratio of such a patient-engagement strategy is not clearly defined, and a more thorough understanding is warranted before such a blanket recommendation. In a systematic review of interventions to reduce low-value care conducted in 2017, 26 of 108 included studies engaged patients within de-implementation interventions in some capacity (e.g., patient cost sharing, provider report cards) [17]. While the authors summarized the effects of patient engagement as “positive” through a narrative synthesis, the effect of such patient engagement was not quantified through meta-analysis, making it difficult to understand the true magnitude of effect and how it compares to interventions that focus on clinicians. The full extent of the impact of patient engagement in de-implementation initiatives remains unclear. In this study, we conducted a systematic review with meta-analysis to determine the effects of de-implementation interventions that engage patients within the patient-clinician interaction and quantify the impact of this patient involvement on the use of low-value care.


Protocol and guidance

This systematic review and meta-analysis is a follow-up to a scoping review that mapped the literature exploring the public’s role in reducing low-value care [18], the protocol for which was registered with Open Science Framework ( Methodology was guided by the Joanna Briggs Institute Reviewer’s Manual [19], and the reporting was guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist [20].

Search strategy and data sources

We searched MEDLINE, EMBASE, and CINAHL from inception to June 28, 2018, and the gray literature using the Canadian Agency for Drugs and Technologies in Health (CADTH) Grey Literature Search Tool [21]. The search strategy was developed in consultation with a medical librarian and peer reviewed by a second medical librarian using the Peer Review of Electronic Search Strategies (PRESS) Checklist [22]. The search strategy was initially developed in MEDLINE (Ovid) (Table 1) and subsequently translated for EMBASE and CINAHL databases with the help of a medical librarian. Our search terms included keywords and synonyms pertinent to three main concepts: (1) low-value care (e.g., overuse, de-implementation), (2) patients (e.g., consumers, patients), and (3) patient involvement (e.g., patient participation). Searches were limited to the English language as terminology regarding low-value care (e.g., Choosing Wisely, low-value, overuse, etc.) is unique to the English language and may not translate well across languages. Reference lists of included studies were hand-searched to identify additional citations, and suggestions were provided from experts in the field.

Table 1 MEDLINE (Ovid) search strategy

Article eligibility and selection

Detailed inclusion and exclusion criteria are presented in Table 2. We used Elshaug’s definition of low-value care [1], which was operationalized to include clinical interventions (tests or treatments) that lack efficacy, have risks that outweigh benefits, or are not cost-effective. Citations were screened for inclusion in two phases. Prior to screening, the citation screening form was pilot tested using a random sample of 50 citations and refined until agreement was consistent (k > 0.8). In phase 1, two investigators (EES and CD) screened citations by title and abstract to determine potential eligibility. Potentially relevant citations entered into phase 2 screening where two investigators (EES and LWB) screened full-text versions of each citation to determine eligibility for inclusion. The kappa statistic was used to quantify agreement throughout screening [23].

Table 2 Inclusion and exclusion criteria

Data extraction and risk of bias assessment

Data extraction was conducted in duplicate by three investigators (EES, CD, LWB) in DistillerSR (Evidence Partners, Ottawa, Canada). Our data extraction form was pilot tested using six randomly selected citations. Extracted data pertained to study characteristics (e.g., study design, country of origin), characteristics of the low-value intervention (e.g., test or treatment), components of the intervention (e.g., strategy for patient engagement, clinical setting), and outcomes (e.g., proportion of patients receiving low-value care).

Quality assessment was conducted in duplicate by two investigators (EES and LWB) using the Cochrane Risk of Bias Tool [24] for the RCTs and a modified Downs and Black checklist for the quasi-experimental studies [25]. For RCTs, summary assessments did not consider the “Performance Bias” domain, as participants were unable to be blinded due to the behavioral nature of the interventions. The Downs and Black checklist was modified by removing questions pertaining to randomization and control groups as necessary. Question 27 originally had six scoring options based on the percent change a sample was powered to detect, but was modified to “Did the authors conduct a power calculation? 1 = Yes, 0 = No” for simplicity. Due to these modifications, the checklist was scored out of 24, or 25 depending on the design of the study. Percent of the total possible score was calculated for each study to facilitate between-study comparisons. Studies were classified by three categories, which were determined by calculating the median percentage score and assigning scores below and above the median to the “lower quality” and “higher quality” categories, respectively. “Average quality” studies had overall percentage scores equivalent to the median.

Data synthesis and analysis

The primary outcome was the proportion of patients who received a low-value clinical intervention. Individual study estimates of the primary outcome were pooled using the random effects model of DerSimonian and Laird [26] and reported as a risk ratio (RR). Publication bias was assessed using a funnel plot and Egger’s test. Heterogeneity was assessed using the I2 statistic [27] and Cochrane Q test. Explanations for heterogeneity were sought through stratified analyses and meta-regression. Pre-specified subgroups for stratified analysis included study design (RCTs vs. non-randomized), type of low-value care (test vs. treatment), and risk of bias (low vs. high/unclear). Meta-analyses were conducted using the metan package in Stata (version 14, StataCorp, TX, USA), and statistical significance was set as P < 0.05.


Study selection

Database and gray literature searches yielded 6736 unique citations, from which we reviewed 218 in full text and included 22 in the systematic review. The most common reason for exclusion after full-text review was the lack of reported use of the low-value practice with and without the intervention (Fig. 1). Of the 22 included studies, 14 reported the change in use of the low-value practice as the proportion of patients who received the low-value practice with and without the intervention, and were included in the meta-analysis.

Fig. 1
figure 1

Selection of articles included in the review

Study characteristics

Characteristics of included studies are described in Table 3. Most studies originated from the USA (n = 13, 59%), UK (n = 4, 18%), and Canada (n = 2, 9%). Nine studies were RCTs, and 13 studies were quasi-experimental. The most common type of low-value care was a medical treatment (n = 17, 77%). Studies commonly took place in primary care (n = 10, 45%), hospital inpatient wards (n = 6, 27%), or emergency departments (n = 3, 14%). Six studies (27%) focused on pediatric patients and their caregivers, four studies (18%) targeted adult patients, one study (5%) targeted geriatric patients, and 11 studies did not specify a target age group (50%). Studies used one or more of the following strategies for patient engagement: patient-oriented educational materials (n = 18, 82%), shared decision-making (n = 5, 23%), and media campaigns (n = 4, 18%). Three studies (14%) involved patients or the public in the development of intervention materials (e.g., shared decision-making tool, waiting room posters).

Table 3 Characteristics of the included studies (n = 22)

The effect of de-implementation interventions that engage patients on the use of low-value care

Of the 22 included studies, 19 (86%) reported a statistically significant decrease in the use of the targeted low-value practice (Table 3). Low-value treatments were targeted in 17 interventions (77%), of which 12 targeted low-value medications. All but one of these interventions included a patient-targeted education component; six interventions used patient-targeted education exclusively [28, 31, 36, 39, 40, 43], one intervention supplemented patient-targeted education with shared decision-making [29], and two interventions used both patient-targeted education and media campaigns [37, 38]. One intervention engaged patients solely through shared decision-making [30]. All but two [36, 40] of these interventions targeting low-value medication reported decreases in the proportion of patients using the medication, ranging from 15 to 25%. However, in one intervention, medication use was reduced in adults (absolute difference 24%, P < 0.02), but not in children (absolute difference 4%, P = 0.18) [39]. Media campaigns were the sole patient-engagement strategy in two interventions [41, 42], both of which reported a significant reduction in low-value medication use. Further details on the strategies used in these interventions are reported in Table 3.

Targeted low-value procedures included non-indicated cesarean sections [33, 46], elective labor induction [44], unnecessary surgery for knee and hip osteoarthritis [47], and non-beneficial life-sustaining treatments in the intensive care unit (Table 3) [32]. One study aiming to reduce non-indicated cesarean sections designed two interventions, an informational program and a guided decision analysis, of which only the guided decision analysis was effective [33]. Elective labor inductions and non-beneficial life-sustaining treatments were successfully reduced through educational classes and ethics consultations, respectively. Unnecessary surgery for hip and knee osteoarthritis was successfully reduced through the use of patient education and decision aids.

Low-value diagnostic tests were targeted in five interventions which reduced use by 7.5–40.7% [34, 35, 45, 48, 49]; one study reduced the use of computed tomography (CT) scans for mild head injury in children [45], one study reduced voiding cystourethrograms for children with normal renal and bladder ultrasounds [49], two studies reduced cardiac stress testing in adults at low risk for acute coronary syndrome, and one study reduced the use of screening tests commonly ordered in physical examination visits such as complete blood counts (CBCs) and electrocardiograms (EKGs) [34, 35, 48]. Of these five interventions, three studies took place within an emergency department and used shared decision-making. Four of the studies included educational materials to inform patients about risks and care options [34, 35].

Fourteen studies inclusive of 10,234 patients were included in the meta-analysis. Pooling data indicated that de-implementation interventions that engage patients within the patient-clinician interaction decreased the use of low-value care by 31% (RR 0.69; 95% CI 0.60–0.80; I2 84.9%) (Fig. 2). This effect was similar in RCTs (n = 8 studies, 3537 patients) (RR 0.74; 95% CI 0.66–0.84; I2 70.7%) and quasi-experimental studies (n = 6 studies, 6697 patients) (RR 0.61; 95% CI 0.43–0.87; I2 90.9%) (Fig. 2). Egger’s test (p = 0.201) and assessment of the funnel plot (Fig. 3) for asymmetry indicated a lack of publication bias.

Fig. 2
figure 2

Random effects meta-analysis stratified by study design examining the effect of de-implementation interventions that engage patients within the patient-clinician interaction on the use of low-value care

Fig. 3
figure 3

Funnel plot with pseudo 95% CIs

Quality assessment

Randomized clinical trials

Of eight RCTs, five had an overall low risk of bias [29,30,31, 34, 35], three had an overall unclear risk of bias [28, 32, 33], and one had an overall high risk of bias (Additional File 1). All but two studies had a low risk of selection bias, and all studies had a low risk of attrition bias. The reporting bias domain was unclear in three studies.

Quasi-experimental studies

All quasi-experimental studies clearly reported objectives and interventions; however, potentially confounding variables were poorly reported. Nine studies (69%) aimed to recruit participants that were representative of the source population, and 12 (92%) used staff, places, and facilities that were representative of treatment received by most patients. None of the studies provided sufficient information to determine whether participants were representative of the population from which they were recruited. None of the studies blinded participants or outcome assessors. The median quality score was 57 (IQR 52–66). Five studies above the median were classified as “higher” quality, six studies below the median were classified as “lower” quality, and two studies that were equivalent to the median were classified as “average” quality (Additional File 2).

Exploration for sources of heterogeneity

Stratified analyses were conducted to explore heterogeneity. Among the five RCTs with low risk of bias, studies used diverse strategies for public engagement and various low-value practices: four studies used shared decision-making, whereas one used patient-targeted educational materials, and three studies targeted medications, whereas two targeted diagnostic tests. Stratification by the strategy for patient engagement suggested that shared decision-making had a greater effect on reducing the use of low-value care (RR 0.58; 95% CI 0.41–0.82; I2 92.8%) in comparison (meta-regression P = 0.07) to patient-oriented educational materials (RR 0.76; 95% CI 0.65–0.89; I2 77.4%) (Fig. 4). Similar effect sizes were found when the meta-analysis was stratified by studies targeting low-value tests (n = 4) (RR 0.54; 95% CI 0.34–0.86; I2 94.7%) and low-value treatments (n = 10) (RR 0.74; 95% CI 0.64–0.85; I2 78.2%) (Fig. 5). When the meta-analysis was restricted to RCTs with low risk of bias (n = 5), the effect of patient-targeted interventions remained significant (RR 0.69; 95% CI 0.58–0.83) although there was still a high degree of heterogeneity (I2 81.7%; Q-statistic P < 0.001) (Fig. 6). Meta-regression, although limited by the small number of studies, suggested that neither variability due to the targeted low-value care (test vs. treatment) nor strategy for patient engagement (shared decision-making vs. patient-oriented educational materials) contributed to inter-study heterogeneity (Additional File 3).

Fig. 4
figure 4

Random effects meta-analysis of the association between patient-targeted interventions to reduce low-value care by the strategy for patient involvement

Fig. 5
figure 5

Stratified by type of low-value care, random effects meta-analysis examining the effect of de-implementation interventions that engage patients within the patient-clinician interaction on the use of low-value care

Fig. 6
figure 6

Among RCTs stratified by summary risk of bias assessment, random effects meta-analysis examining the effect of de-implementation interventions that engage patients within the patient-clinician interaction on the use of low-value care


We identified nine RCTs and thirteen quasi-experimental studies that prospectively evaluated the effect of de-implementation interventions that engaged patients within the patient-clinician interaction on the use of low-value care. Compared to a prior knowledge synthesis examining this topic [17], we identified additional relevant studies and provide a tangible, quantified estimate of the effect of these interventions. Patient engagement occurred through patient-oriented educational materials, mass media campaigns, and shared decision-making between the patient and clinician. Studies examined a broad array of low-value care including potentially unnecessary tests (e.g., computed tomography scans for mild head injuries), treatments (e.g., antibiotics for upper respiratory tract infections), and surgeries (e.g., cesarean sections). The most common clinical context was adult patients making decisions about medical treatments in primary care. Meta-analysis demonstrated that patient engagement within the patient-clinician interaction reduced the use of low-value care by an average of 31% (range 20 to 40%). Although this effect estimate was limited by inter-study heterogeneity, it is the first to quantify the potential impact of such de-implementation interventions, and despite the heterogeneity, effects remained consistent when examined in subgroups defined by different strategies for patient engagement, types of low-value care, and study design. Researchers, policymakers, and decision-makers should consider the patient-targeted intervention as a strategy for reducing low-value care.

Our results add to and compare favorably with prior research examining a similar question [17]. Colla et al. performed a systematic review examining interventions to reduce low-value care, inclusive of 108 articles, of which 19 reported on interventions that involved a patient education component. Narrative synthesis among the 19 articles concluded that patient-oriented education is an effective strategy for patient engagement within the patient-clinician interaction and can successfully reduce low-value care. However, many of their studies used patient-targeted strategies within multicomponent interventions that included clinician-targeted strategies (e.g., clinical decision support, provider feedback). This makes it challenging to understand which components of the intervention (i.e., patient-targeted or clinician-targeted) were actually effective and whether there is an advantage to focusing on one group or another. Although it would seem logical that the synergistic effects achieved from simultaneously engaging patients and clinicians would be greater than those from strategies engaging the two parties separately, this has yet to be adequately examined. Understanding the effects of patient-targeted interventions is important because clinicians have indicated that important barriers to reducing low-value care include patient care expectations and the risk of patient dissatisfaction if expectations are not met [15, 16]. Moreover, given the costs and resources associated with multicomponent de-implementation initiatives, understanding which strategies have the greatest effect on reducing low-value care is important for advancing the science of de-implementation and informing how to best reduce low-value care. Patient-targeted interventions within the patient-clinician interaction may help to mitigate these challenges. As clinician-targeted strategies did not meet inclusion criteria for our systematic review, this enabled us to isolate studies that only reported on patient-targeted interventions within the patient-clinician interaction and examine their effect on use of low-value care.

Acknowledging the potential benefits of patient engagement in reducing low-value care, patient engagement in clinical decision-making is often viewed as challenging, time-consuming, and potentially costly [50, 51]. Our study demonstrates that the ensuant reductions in low-value care make tackling these challenges worthwhile. However, it is important to consider that the nature and success of patient-targeted de-implementation interventions likely depend on patient and clinician characteristics, clinical context, and the targeted low-value practice. For example, one study found that patient-oriented educational materials reduced unnecessary medication use among adult, but not pediatric, patients [39]. This finding is congruent with and potentially explained by evidence from recent systematic reviews indicating that informational components may be sufficient for interventions that target patients (e.g., adults), but that caregivers (e.g., parents of children) require supplemental information that enables activation (i.e., prompting action) and/or collaboration (i.e., engagement with clinicians or others) [52, 53]. Because these de-implementation interventions occur within the patient-clinician interaction, their success will also depend on the characteristics of participating clinicians and their ability and intention to engage the patient in strategies like shared decision-making. Interventions may benefit from a training component wherein clinicians learn how to effectively engage patients through the selected strategy. In addition, most interventions in our review targeted low-value practices that lacked efficacy in a primary care setting by providing patients with educational materials. This approach to patient engagement was evidently effective for a low-value practice associated with minimal risk for patients with low illness burden. However, for low-value practices provided to patients with greater illness burden cared for in clinical contexts associated with greater risk (e.g., emergency departments, hospital inpatient wards, etc.), de-implementation interventions that seek to engage patients and/or their caregivers will likely require more interaction and collaboration with clinicians. Although few studies in our review examined patient-focused interventions in high acuity contexts, one study did successfully implement an intervention that involved families and clinicians in ethics consultations to discuss non-beneficial life-sustaining treatments in intensive care units [32]. Future research should examine the effect of patient engagement on the use of low-value care in acute care contexts.

Strengths and limitations

A notable strength of this study is the rigorous methodology which included a peer-reviewed search strategy and adherence to published guidelines regarding systematic review and meta-analysis methodology. In addition, our review was narrow in scope and focused solely on de-implementation interventions that engaged patients within the patient-clinician interaction, which is arguably one of the most critical clinical contexts for influencing the use of low-value care. Our meta-analysis quantified the effect of patient engagement within the patient-clinician interaction and confirmed its effectiveness as an approach to reducing low-value care, while highlighting important heterogeneity within published literature. However, this study must be interpreted within the context of its limitations. First, the English language restriction may have omitted relevant articles, yet it is unlikely that this would have altered our main findings. Second, the main outcome of the proportion of patients who received the low-value practice may have overestimated the use of certain low-value practices. For example, if a study’s outcome was the number of patients given a prescription for benzodiazepines and some patients did not end up using the prescription, the true number would have been overestimated. However, this misclassification bias is expected to be non-differential and would suggest that if anything, our results are a conservative estimate of the true population effect. Third, this study focused on low-value clinical interventions, and therefore, we cannot comment on the de-implementation of non-medical forms of care. In addition, the majority of included studies targeted medication prescribing in primary care. This limits the generalizability of our results. Further research is needed to determine if the observed effects persist in other clinical contexts (e.g., emergency department). Fourth, there was significant inter-study heterogeneity that affects interpretation of the pooled estimates. Given the nature of our research question, we anticipated observing inter-study heterogeneity within our pooled estimates. We proceeded with calculating pooled effect estimates in spite of this because (1) effect estimates from individual studies, especially those that were of higher quality, were similar; (2) pooled estimates have greater utility relative to individual study effect estimates, as they may be used to facilitate comparisons with meta-analyses of other types of interventions aiming to reduce low-value care (e.g., provider-targeted interventions); (3) collectively, this small group of studies represents the totality of available evidence for evaluating the average impact of patient-targeted interventions on use of low-value care, so restricting our scope to a specific low-value practice, patient population, or patient-engagement tool would have resulted in a small number of studies precluding any meaningful findings; and (4) similar approaches have been employed in other meta-analyses investigating the utility of interventions that span clinician disciplines and patient populations, such as decision aids [54]. Finally, although this review indicated that de-implementation interventions that engage patients within the patient-clinician interaction is a promising approach for reducing low-value care, we must consider that the implementation and effectiveness of these interventions will be significantly influenced by clinicians, given that they hold the authority to order a low-value test or treatment. This review did not aim to explore how clinicians may respond to or support patient engagement in de-implementation interventions, and therefore, further research is required to determine how to appropriately support and engage clinicians within such interventions.


This systematic review with meta-analysis suggests that de-implementation interventions that engage patients within the patient-clinician interaction through patient-targeted educational materials or shared decision-making are effective in decreasing the use of low-value care, especially for medical treatments prescribed within primary care. Additional research should seek to understand the utility of patient-targeted interventions in the acute care context and how the effectiveness of patient-targeted interventions compares to that of clinician-targeted or multicomponent interventions. However, based on the results of this study, de-implementation interventions that seek to reduce low-value medical treatments provided to patients in a primary care setting should incorporate patient engagement using tailored educational and shared decision-making tools.

Availability of data and materials

All data generated or analyzed during this study are included in this published article and its supplementary information files.



Canadian Agency for Drugs and Technologies in Health


Peer Review of Electronic Search Strategies


Preferred Reporting Items for Systematic Reviews and Meta-analyses


Randomized clinical trials


  1. Elshaug AG, Rosenthal MB, Lavis JN, et al. Levers for addressing medical underuse and overuse: achieving high-value health care. Lancet. 2017;390:191–202.

    Article  PubMed  Google Scholar 

  2. Shrank WH, Rogstad TL, Parekh N. Waste in the US health care system. JAMA. 2019.

  3. Badgery-Parker T, Pearson S-A, Dunn S, et al. Measuring hospital-acquired complications associated with low-value care. JAMA Intern Med. 2019;179:499.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Korenstein D, Keyhani S, Troy A, et al. Development of a conceptual map of negative consequences for patients of overuse of medical tests and treatments. JAMA Intern Med. 2018;178:1401.

    Article  PubMed  Google Scholar 

  5. Bece A, Hamilton C, Hickey BE. Over 150 potentially low-value health care practices: an Australian study. Med J Aust. 2013;198:597–8.

    Article  PubMed  Google Scholar 

  6. Prasad V, Vandross A, Toomey C, et al. A decade of reversal: an analysis of 146 contradicted medical practices. Mayo Clin Proc. 2013;88:790–8.

    Article  PubMed  Google Scholar 

  7. Cassel CK, Guest JA. Choosing wisely. Jama. 2012;307:1801.

    Article  CAS  PubMed  Google Scholar 

  8. Garner S, Littlejohns P. Disinvestment from low value clinical interventions: NICEly done? BMJ. 2011;343:d4519.

    Article  PubMed  Google Scholar 

  9. Canadian Institute for Health Information. Unnecessary care in Canada. Ottawa: CIHI; 2017.

  10. Rosenberg A, Agiro A, Gottlieb M, et al. Early trends among seven recommendations from the choosing wisely campaign. JAMA Intern Med. 2015;175:1913.

    Article  PubMed  Google Scholar 

  11. Niven DJ, Rubenfeld GD, Kramer AA, et al. Effect of published scientific evidence on glycemic control in adult intensive care units. JAMA Intern Med. 2015;175:801–9.

    Article  PubMed  Google Scholar 

  12. Van Bodegom-Vos L, Davidoff F, Marang-Van De Mheen PJ. Implementation and de-implementation: two sides of the same coin? .

  13. Colla CH. Swimming against the current — what might work to reduce low-value care? N Engl J Med. 2014;371:1280–3.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Brownlee S, Berman A. Defining value in health care resource utilization: articulating the role of the patient; 2016.

    Google Scholar 

  15. Buist D. Primary care clinicians’ perspectives on reducing low-value care in an integrated delivery system. Perm J. 2015;20:41–6.

    Article  PubMed  Google Scholar 

  16. Zikmund-Fisher BJ, Kullgren JT, Fagerlin A, et al. Perceived barriers to implementing individual choosing wisely® recommendations in two national surveys of primary care providers. J Gen Intern Med. 2017;32:210–7.

    Article  PubMed  Google Scholar 

  17. Colla CH, Mainor AJ, Hargreaves C, et al. Interventions aimed at reducing use of low-value health services: a systematic review. 2017.

  18. Sypes EE, de Grood C, Clement FM, Parsons Leigh J, Whalen-Browne L, Stelfox HT, Niven DJ. Understanding the public's role in reducing low-value care: a scoping review. Implementation Science. In press. (accepted March 23 2020).

  19. Institute JB. Joanna Briggs Institute reviewers’ manual. Adelaide: SA Joanna Briggs Inst; 2011.

    Google Scholar 

  20. Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement David Moher and colleagues introduce PRISMA, an update of the QUOROM guidelines for reporting systematic reviews and meta-analyses , for the PRISMA Group. doi:

  21. Canadian Agency for Drugs and Technologies in Health. Grey matters: a practical tool for searching health-related grey literature. 2015. Accessed 11 Apr 2018).

  22. McGowan J, Sampson M, Salzwedel DM, et al. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6.

    Article  PubMed  Google Scholar 

  23. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159.

    Article  CAS  Google Scholar 

  24. Higgins JPT, Altman DG, Gotzsche PC, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions; 1998.

    Book  Google Scholar 

  26. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7:177–88.

    Article  CAS  Google Scholar 

  27. Higgins JPT, Thompson SG, Deeks JJ, et al. Measuring inconsistency in meta-analyses. BMJ. 2003;327:557–60.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Macfarlane J, Holmes W, Gard P, et al. Primary care patient information leaflet. Bmj. 2002;324:91–4.

    Article  Google Scholar 

  29. Francis NA, Butler CC, Hood K, et al. Effect of using an interactive booklet about childhood respiratory tract infections in primary care consultations on reconsulting and antibiotic prescribing: a cluster randomised controlled trial. BMJ. 2009;339:374–6.

    Article  Google Scholar 

  30. Légaré F, Labrecque M, Cauchon M, et al. Training family physicians in shared decision-making to reduce the overuse of antibiotics in acute respiratory infections: a cluster randomized trial. CMAJ. 2012;184:726–34.

    Article  Google Scholar 

  31. Tannenbaum C, Martin P, Tamblyn R, et al. Reduction of inappropriate benzodiazepine prescriptions among older adults through direct patient education: the EMPOWER cluster randomized trial. JAMA Intern Med. 2014;174:890–8.

    Article  CAS  PubMed  Google Scholar 

  32. Schneiderman LJ, Gilmer T, Teetzel HD, et al. Effect of ethics consultations on nonbeneficial life-sustaining treatments in the intensive care setting. JAMA. 2003;290:1166.

    Article  CAS  PubMed  Google Scholar 

  33. Montgomery AA, Emmett CL, Fahey T, et al. Two decision aids for mode of delivery among women with previous caesarean section: randomised controlled trial. BMJ. 2007;334:1305.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Hess EP, Knoedler MA, Shah ND, et al. The chest pain choice decision aid: a randomized trial. Circ Cardiovasc Qual Outcomes. 2012;5:251–9.

    Article  PubMed  Google Scholar 

  35. Hess EP, Hollander JE, Schaffer JT, et al. Shared decision making in patients with low risk chest pain: prospective randomized pragmatic trial. BMJ. 2016;355.

  36. Wheeler JG, Fair M, Simpson PM, et al. Impact of a waiting room videotape message on parent attitudes toward pediatric antibiotic use. Pediatrics. 2001;108:591–6.

    Article  CAS  Google Scholar 

  37. Perz JF, Craig AS, Coffey CS, et al. Changes in antibiotic prescribing for children after a community-wide campaign. JAMA. 2002;287:3103–9.

    Article  Google Scholar 

  38. Dollman WB, LeBlanc VT, Stevens L, et al. A community-based intervention to reduce antibiotic use for upper respiratory tract infections in regional South Australia. Med J Aust. 2005;182:617–20.

    Article  Google Scholar 

  39. Gonzales R, Corbett KK, Leeman-Castillo BA, et al. The “minimizing antibiotic resistance in Colorado” project: impact of patient education in improving antibiotic use in private office practices. Health Serv Res. 2005;40:101–16.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Ashe D, Patrick PA, Stempel MM, et al. Educational posters to reduce antibiotic use. J Pediatr Heal Care. 2006;20:192–7.

  41. Gonzales R, Corbett KK, Wong S, et al. Get smart Colorado. Medical Care. 2008;46:597–605.

  42. Hemo B, Shamir-Shtein NH, Silverman BG, et al. Can a nationwide media campaign affect antibiotic use? Am J Manag Care. 2009;15:529–34.

    PubMed  Google Scholar 

  43. Morgan JD, Wright DJ, Chrystyn H. Pharmacoeconomic evaluation of a patient education letter aimed at reducing long-term prescribing of benzodiazepines. Pharm World Sci. 2002;24:231–5.

    Article  PubMed  Google Scholar 

  44. Simpson KR, Newman G, Chirino OR. Patient education to reduce elective labor inductions. MCN, Am J Matern Nurs. 2010;35:188–94.

    Article  Google Scholar 

  45. Engineer RS, Podolsky SR, Fertel BS, et al. A pilot study to reduce computed tomography utilization for pediatric mild head injury in the emergency department using a clinical decision support tool and a structured parent discussion tool. Pediatr Emerg Care. 2018;00:1.

    Article  CAS  Google Scholar 

  46. Navaee M, Abedian Z. Effect of role play education on primiparous women’s fear of natural delivery and their decision on the mode of delivery. Iran J Nurs Midwifery Res. 2019;20:40–6.

    Google Scholar 

  47. Arterburn D, Wellman R, Westbrook E, et al. Introducing decision aids at group health was linked to sharply lower hip and knee surgery rates and costs. Health Aff. 2012;31:2094–104.

    Article  Google Scholar 

  48. Pugel S, Stallworth JL, Pugh LB, et al. Choosing wisely in Georgia: a quality improvement initiative in 25 adult ambulatory medicine offices. Jt Comm J Qual Patient Saf. 2018;44:699–707.

    Article  PubMed  Google Scholar 

  49. Jerardi KE, Elkeeb D, Weiser J, et al. Rapid implementation of evidence-based guidelines for imaging after first urinary tract infection. Pediatrics. 2013;132:749–55.

    Article  Google Scholar 

  50. Kovacs Burns K, Bellows M, Eigenseher C, et al. ‘Practical’ resources to support patient and family engagement in healthcare decisions: a scoping review. BMC Health Serv Res. 2014;14:175.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Domecq JP, Prutsky G, Elraiyah T, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14:89.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Fiest KM, McIntosh CJ, Demiantschuk D, et al. Translating evidence to patient care through caregivers: a systematic review of caregiver-mediated interventions. BMC Med. 2018;16:105.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Gagliardi AR, Légaré F, Brouwers MC, et al. Patient-mediated knowledge translation (PKT) interventions for clinical encounters: a systematic review. Implement Sci. 2015;11:26.

    Article  Google Scholar 

  54. Stacey D, Légaré F, Lewis K, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2017;2017:CD001431.

    Article  PubMed Central  Google Scholar 

Download references


The authors would like to acknowledge Dr. Diane Lorenzetti (University of Calgary) for the assistance in developing the electronic search strategy and Ms. Zahra Premji (University of Calgary) for the peer review of the electronic search strategy.


The funding sources (MSI Foundation, Canadian Frailty Network) were not involved in the study design; collection, analysis, and interpretation of data; and writing of the report; nor in the decision to submit the article for publication. The researchers were independent from funders, and all authors had full access to all of the data in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis.

Author information

Authors and Affiliations



Ms. Sypes and Dr. Niven had full access to all the data in the study and take full responsibility for the integrity of the data and for the accuracy of the data analysis. Concept and design: Sypes, Clement, Parsons Leigh, Niven, and Stelfox Acquisition, analysis, or interpretation of data: Sypes, de Grood, Whalen-Browne, Parsons Leigh, Clement, Niven, and Stelfox Drafting of the manuscript: Sypes, Niven, and Stelfox Critical revision of the manuscript for important intellectual content: Sypes, de Grood, Whalen-Browne, Parsons Leigh, Clement, Niven, and Stelfox Statistical analysis: Sypes and Niven Administrative, technical, or material support: Niven and Stelfox Supervision: Niven and Stelfox The corresponding author attests that all listed authors meet the authorship criteria and that no others meeting the criteria have been omitted. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Daniel J. Niven.

Ethics declarations

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Risk of bias and quality assessment of included randomized clinical trials (n = 9).

Additional file 2.

Quality assessment for non-randomized interventions using the Downs & Black tool (n = 13).

Additional file 3.

Results from meta-regression analysis.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sypes, E.E., de Grood, C., Whalen-Browne, L. et al. Engaging patients in de-implementation interventions to reduce low-value clinical care: a systematic review and meta-analysis. BMC Med 18, 116 (2020).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: