Nature of evidence | Study design | Study execution | Consistency | Directness of evidence |
---|---|---|---|---|
A | Pairwise meta-analysis of comparative randomized controlled trials (RCTs) (for interventions) | No important flaws | Consistent | Direct or strong indirect |
RCTs (for interventions) | ||||
Non-randomized studies (for diagnosis and prognosis) | ||||
B | Meta-analysis of RCTs or RCTs (for interventions) | Important flaw < OR > Inconsistent < OR > Weak indirect | ||
Non-randomized studies (for diagnosis or prognosis) | Important flaw < OR > Inconsistent < OR > Weak indirect | |||
Non-randomized controlled studies (for interventions) | No important flaws consistent direct < OR > Strong indirect | |||
C | Non-randomized controlled studies (for interventions) | Important flaw < OR > Inconsistent < OR > Weak indirect | ||
Meta-analyses or RCTs with a combination of important flaws AND inconsistency AND/OR indirect evidence | ||||
D | Other evidence (not expert opinion) | |||
E | Expert opinion | |||
Exceptions that can alter the quality of grading | ||||
Sparse data (few events); use of data not in its initial randomization or apparent publication bias can lower the quality; a very strong association can raise the quality | ||||
Coding notes | ||||
Important flaws occur when the highest standards of research that could be achieved by a study are not applied | ||||
Consistency occurs at two levels – design: consistent methods, patients, outcomes; and statistical: a test of homogeneity of a summary estimate when the level of design consistency is acceptable and meta-analysis appropriate | ||||
Directness – direct evidence: relevant patient benefits and harms are measured in studies; strong indirect: the surrogate endpoint is strongly related to desirable endpoints, or direct evidence is available for a sufficiently related patient group; weak indirect: the relationship between the study outcomes and patient benefits or harms is insufficient | ||||
Summary of quality of evidence | ||||
A. High quality of evidence – future evidence is unlikely to change confidence in the estimate of effect | ||||
B. Moderate quality of evidence – future evidence is likely to have an impact on the confidence of the estimate of effect and may change that estimate | ||||
C. Poor quality evidence – future evidence is very likely to have an impact on the confidence of the estimate of effect and is likely to change that estimate | ||||
D. and E. Very poor quality evidence – Any estimate of effect is very uncertain |