Congratulations. You've made it through all five domains for downgrading the evidence from RCTs using the GRADE approach. Now it's time to learn about rating up the quality.

First some details. We are still talking about rating evidence for interventions. There are some differences in rating evidence for questions about prognosis, for example, and we'll cover those later.

So you are still looking at the evidence for your PICO question for an intervention. But now you are evaluating evidence from observational study designs.

You may remember that RCTs are the preferred design for a fair test of an intervention. But for whatever reason, you have observational studies providing evidence.

Observational study designs have limitations for evaluating clinical effectiveness. In observational studies such as cohort and case-control studies, the population of interest generally self-selects the intervention, making it necessary to match or adjust for confounding factors that could impact the results. Because of this, GRADE starts the evidence rating level for observational studies at Low rather than the High rating where RCT studies begin. 

Yet it is still possible for well designed and conducted observational study designs to contribute high quality evidence. There are 3 domains for rating up the quality of evidence.

Today we'll start with the most common reason for rating up the quality of evidence. It is a large effect.

To begin, only observational studies that have no important threats to validity are eligible for upgrading. So the studies must be well-designed and well conducted and have minimized the known risks to bias for observational study designs. This means that even very good observational studies will start at Low, but they will be eligible for upgrading.

The most obvious example of observational studies that could be upgraded for for large effect would be studies of the relationship between smoking and lung cancer. Even the early studies found large effects on lung cancer incidence from smoking. Recent CDC data show that in the 1960s, smokers were 12 times more likely to get lung cancer than non-smokers and in 2010 that had increased to 25 times more likely. These aren't just large, they are huge effects. GRADE recommends upgrading evidence by one level if the exposure doubles the effect (increases by 2 times) and upgrading by 2 levels if there is an increase of 5 times. This means the studies on smoking and lung cancer would be upgraded from a rating of Low to a rating of High.

Note that GRADE issues a caution when the outcome is subjectively measured, such as assessments of pain. Remember that observational study designs are not blinded and can be impacted by patient perception or belief of benefit.

Studies that find large effects even when subject to the limitations of observational design provide greater confidence that the effects are real. Hence GRADE allows for upgrading of that evidence by one or two levels depending on the strength.

As always, for more information you can consult the GRADE Handbook

TheEvidenceDoc November 28, 2017