It's been 13 years since Shaneyfelt and colleagues compared 279 clinical practice guidelines to accepted standards for guideline development and found that the guidelines only met 43% of those standards.
Unfortunately, a new comparison by Kung and colleagues got the same result. Only 44% of standards were met by the guidelines. This time the researchers analyzed 114 guidelines which they'd randomly picked from the National Guideline Clearinghouse. You need to know that the Clearinghouse requires that a set of minimum criteria must be met before inclusion (see here). One of the minimum criteria requires documentation that a systematic search for evidence was conducted and a review of that evidence incorporated. You would expect, then, that this would create a selection bias for higher quality guidelines which would improve guideline adherence to the standards. You also need to know that for their evaluation, Kung and colleagues only held the guidelines to the simplest criteria, for example giving credit for evidence supporting individual recommendations if some but not all recommendations had supporting evidence and using only 18 of the 25 standards recently codified by the Institute of Medicine (IOM). They note that if guidelines were held to the more stringent criteria, the results would have been much worse.
The findings are discouraging. The standards collected and released by the IOM are not new. That they needed to be systematically presented and described is now even more obvious. Until developers of guidelines adhere to these standards and journals require their adherence for publication, users will be unable to trust the recommendations from clinical practice guidelines. TheEvidenceDoc
Post Script - If you can get full text access, you should read Shaneyfelt's invited commentary that accompanies Kung and colleagues' new work. He makes several recommendations, some controversial, about how the US could improve not only the quality but also the consistency and relevance of clinical practice guideline recommendations.