Imagine that you are now ready to begin rating evidence. You have a systematic review of evidence whether completed by another organization or as part of your guideline development. You know It meets criteria for a good systematic review because you assessed the quality using one of the available tools (see yesterday's blog for links).
Now you are ready to GRADE this body of evidence. Remember, rating evidence is more than just quality assessment.
For RCT derived evidence, you'll remember rating starts at HIGH confidence and is rated down for deficiencies in one or more of the five domains.
The domains are:
- Risk of bias
- Publication bias
What does GRADE mean by Risk of Bias? You'll note all the domains are worded in the negative, since they identify deficiencies for which confidence should be downgraded. So rather than describing the domain as an assessment of the validity of the studies, which would be in the positive, the wording focuses on the negative. What are the limitations, or the risks to validity, in the studies that comprise the evidence? What is the overall risk of bias?
The Cochrane Risk of Bias tool for RCTs covers the main concerns and is commonly used by a variety of organizations to assess risk of bias. See Cochrane methods for a nice description of major biases in studies.
Below is a screenshot from the instructions for GRADing risk of bias from Guyatt et al J Clin Epidemiol. 2011 Apr;64(4):407-15. This is the list of study limitations or risk of bias GRADE recommends you should consider when evaluating individual randomized trials.
So these are the limitations you will evaluate in the individual studies. Now remember that you are considering not just the risk of bias of each individual study, but across the body of evidence comprised of those studies. GRADE suggests you do not downgrade if most of the evidence is from studies at low risk of bias. If most of the evidence is from studies with limitations, you will consider whether to downgrade one or two levels for this domain.
Tomorrow I'll cover inconsistency. As always, you can read ahead or review more detail in the GRADE HANDBOOK.
TheEvidenceDoc November 15, 2017