In yesterday's introduction to this series, I started with an overview of the first step in the process for rating a body of evidence. And I promised more detail on how to use the most commonly used evidence rating system, GRADE. 

GRADE starts with outcomes. Your outcomes. The outcomes specified by your panel when developing your clinical questions at the very beginning of the guideline process.

The confidence you should place in the body of evidence for your clinical question will be evaluated separately for each of the outcomes you have selected.

Does this surprise you that the validity, consistency, and strength of the evidence should vary for different outcomes? Remember, the evidence for different outcomes may come from different sources, from different studies or from subpopulations within studies. And remember that rating a body of evidence is different than just rating the quality of the individual studies that comprise the evidence (though that is one of the components).

The GRADE method includes 5 categories for rating down your confidence in the quality of evidence derived from Randomized Controlled Trial studies (RCTs). There are other considerations for observational study designs. For now, we will start with the RCT study design to keep the explanations as simple as possible. GRADE labels these categories "domains". The 5 domains are:

  1. Risk of bias = limitations in the design and conduct of the studies that impact validity
  2. Inconsistency = lack of reproducibility of the effect across multiple studies
  3. Indirectness = in any of the PICO elements; if tested in population that differs from the one of interest, if difference in the intervention itself, or use of surrogate outcomes
  4. Imprecision = when confidence intervals around the effect estimate include both benefit and harm and impact the clinical decision threshold
  5. Publication bias = difficult to assess, but GRADE provides some indicators for suspicion that positive studies have been selectively published for the topic

For all of the domains, GRADE provides instructions for how to downgrade confidence in the evidence by one or two levels of confidence. This is where your methodologist who is experienced in the system can help with the details.

I'll eventually get to some of the major concepts for downgrading in each of the domains, but for now consider today's main point, that 5 domains contribute to the evidence GRADE for RCTs and that study quality is only one of those domains. What does this mean for the overall rating? What is the overall rating? How does GRADE summarize the overall rating?

Ah, you'll have to wait until tomorrow...unless you want to study ahead on your own using the official GRADE handbook

TheEvidenceDoc November 8, 2017