It's time for the last domain for upgrading.
I've covered the most straightforward and most common reasons for upgrading evidence from well-done observational studies:
- large effect
The third and last domain is a little trickier to explain to those without methodology background. It has to do with confounding, which itself may need explaining. Confounding occurs when something is associated with both the intervention and the outcome of interest. For example, a study I published in the early 90s sought to evaluate the impact of a specific herbicide exposure on a specific eye disease. An earlier, unpublished study looked at all eye diseases in the cohort and found an association with long term exposure to the herbicide and development of cataracts. That study did not control for age. Increasing age was a confounder, since cataracts are more common as we age and long term exposure is also associated with increasing age.
Confounding is a known problem for all studies and especially for observational designs. Randomization is used in trials in an attempt to prevent confounding (which may or may not be successful, but that's another topic for another time). In observational designs, matching is used to prevent confounding. If we control for enough of the known confounders for the outcome, like age which is a common one, and sex, race, SES, or educational level if relevant, we are also hoping to match for confounders we don't know about. Confounding can also be adjusted for in the analysis through stratification or multivariate analysis.
Now that you have a bit of an understanding of confounding, here are the rules for upgrading.
Did you find an effect, even though all the known confounders should have reduced or eliminated an effect? If so, consider upgrading.
A common example is the study of an intervention where the sickest patients got the intervention and yet they improved more than the comparison group. The confounding factor of sicker patients should have reduced the impact of the intervention and yet it did not. So you would upgrade confidence in the evidence.
Of course, consider the converse, that is, did you fail to find an effect even though the known confounding factors should have increased the effect? Again you may decide to upgrade confidence in the evidence.
So there are 3 reasons to upgrade:
- Large effect
- All known confounding should be working against the direction of the observed effect
That's it for upgrading in this very brief overview.
You know where to find more detail - GRADE Handbook
TheEvidenceDoc December 1, 2017