Composite Endpoints - Canny or cunning use of #healthoutcomes data?

You are on a guideline panel and you're at the PICO (Population - Intervention - Comparison - Outcome) stage of development. It's time to choose important outcomes.

A reminder - important outcomes are those important to the patients who are affected by the disease or condition. So for studies of diabetes, patient important outcomes are things like premature mortality or heart attack but not blood sugar levels. Lowering blood sugar levels is an intermediate step in the process to better health for diabetics, so would be considered a surrogate or intermediate outcome. This only indirectly measures what we are interested in, so the evidence wouldn't be rated as strong as the evidence for outcomes of direct importance.

So how do composite endpoints fit into this? What are composite endpoints? Composite endpoints (CEP) or composite outcomes are combined endpoints used in some clinical trials, particularly common in cardiology trials.

 According to a systematic review by Ferreira-Gonzalez et al, the most common reasons cited for using CEP are the smaller study size requirement and to evaluate the net effect of an intervention. Avoiding adjustments for multiple comparisons was also cited as a rationale for use. Disadvantages to using included misinterpretation when the components differed in patient importance or in size and direction of the effect.

A systematic review by Cordoba et al of 114 RCTs published in 2008 that used CEP found that changes in the definition of the composite outcome during the trials were common. Selection of components was often not pre-specified and definitions were inconsistently described throughout the study reports. Those trials also failed to report treatment effect for the individual components in a third of the publications. The less important components often had higher event rates and larger effects associated with treatment. Cordoba and colleagues recommended that "composite endpoints should generally be avoided, as their use leads to much confusion and bias. If composites are used, trialists should follow published guidance."

Fortunately, there is published guidance to direct decisions on how to create composite endpoints.  We can use this guidance to help us in determining whether or not composite endpoints may be valid and utilized in our guideline development.

Freemantle and colleagues use examples to demonstrate the problems with composite outcomes, including the presumption that the benefit described may be attributed to all the components when in fact, it is derived from only one component. The opposite also occurs; measures of a positive treatment effect for a critical outcome can be diluted by an outcome with no effect. And they provide data showing that CEP including clinician driven outcomes - where physicians order the intervention - were twice as likely to be associated with statistically significant results for the composite outcomes. Examples would include things like revascularization, hospitalization, and initiation of new therapy.

Montori and colleagues have produced an educational paper using examples to summarize three major considerations for evaluating the validity of composite endpoints. They are:

  1. Ensure that the component endpoints are of similar importance to patients. Most patients would not equate serious endpoints like death or heart attack with need for change in therapy.
  2. Ensure that the more and less important endpoints occur with similar frequency. If the more important events are uncommon (as is often the case for mortality) the composite measure is likely to be driven by the more common though less important events.
  3. Ensure that the component endpoints are likely to have similar risk reduction. Individual components should be similarly affected by the intervention.

There's another challenge when systematically collecting and summarizing the evidence on a given topic. Since CEP definitions frequently change, even within studies, it is very difficult to find standard definitions used across studies. This limits your ability to collect and combine the data from multiple studies for your guideline.

The easy answer for many guideline panels will be to simply exclude CEP from your outcome selections. But if you decide to consider their importance for your topic, you now have some guidance for evaluating that CEP.

And if you want to ponder that proposed benefit of using CEP to evaluate net effect by accounting for competing risks, I suggest you read this systematic review by Manja and colleagues

And though this very brief summary is directed at guideline developers, it wouldn't hurt trialists to learn a bit more about CEP.

TheEvidenceDoc August 7, 2017

 

Azithromycin and arrythmia - another reason to Choose Wisely

Today the FDA released a Safety Announcement to warn the public and health professionals that azithromycin can cause heart abnormalities leading to sudden death.  Azithromycin is a macrolide antibiotic, a type of antibiotic already known to be associated with abnormal heart rhythm and an increased risk of sudden death. But azithromycin was widely believed to have minimal toxicity to the heart, because prior studies had not detected the potentially lethal effect. It was that belief that lead it to become widely prescribed, often in elderly patients with heart disease who may be more susceptible to the drug's cardiac effects.

In 2009 JAMA published a study of antibiotic use for acute respiratory tract infections (colds) with Big Data from a national database. The study had encouraging findings of an overall decrease in the use of antibiotics for colds - most caused by viruses which are not treatable with antibiotics. But it also found that even though most antibiotic use decreased, azithromycin use increased substantially, six times between 1995 and 2006. This data shouldn't surprise us. The Z-pack has become so common that patients ask for it by name.

The FDA summary of the data that prompted the warning comes from a NEJM study published in May of 2012 and a manufacturer study that found azithromycin is associated with QT interval prolongation - an irregularity of the heart rhythm that can lead to sudden death.

The NEJM study was a very well designed and conducted large study in a very large population with relatively complete medical record.  The NEJM study is a good example of the use of Big Data to examine a patient safety issue. The study authors used a statewide database to evaluate the risk after published case reports suggested it. But it was an observational study, a design often disparaged outside the research community. However, well done observational studies are often our best chance at finding harm associated with therapy because, thankfully, harms are relatively rare. Randomized Controlled Trials (RCTs) are often too small or too short in follow-up time to detect increases in harmful side effects.  Large observational studies that are carefully planned and carried out to reduce the risks associated with non-experimental design can find these increases.

So we have data that the Z-pac is not as safe as once believed. Data to consider when balancing the benefit and risk to an individual patient. Even when side effects are rare, as they are in this instance, we need to evaluate the seriousness of the side effect, in this case sudden cardiac death.  We must balance that risk against the potential benefit from using the drug and assess whether or not the patient needs this specific antibiotic or even any antibiotic. 

We have data that while overall antibiotic prescribing for colds is down, prescribing for azithromycin is up, way up. It kills more bacteria and it's easier for patients to complete a full course of therapy.  Patients ask for it by name.

And we are concerned about antibiotic resistant bacteria. 

Do we need more evidence to Choose Wisely when considering a prescription for antibiotic therapy?  The FDA is not saying we should stop using azithromycin. The FDA is warning that our prior belief that it did not share the cardiac toxicity of others in its class was wrong. Dead wrong.

It's time to consider this drug choice more carefully and choose wisely when evaluating antibiotic therapy. Does this patient need azithromycin? Would amoxicillin be effective? Does this patient need an antibiotic at all?

TheEvidenceDoc

Postscript -Dr. Ireland commented on this issue last year, shortly after the release of the study and the FDA statement.

Highlight of Guidelines Conference = David Eddy’s “Swan Song”

The Guidelines International Network of North American (G_I_N NA) conference Evidence-Based Guidelines Affecting Policy, Practice and Stakeholders was held this week at the New York Academy of Medicine (NYAM).  David Eddy began his keynote stating this was most certainly his last public lecture, as he is moving into retirement.  I certainly hope not.

Though he doesn’t know it, David Eddy has been a virtual mentor for my own career as a clinical epidemiologist who practices evidence based medicine.  His experience in medical training was identical to mine with the discovery that medical decision- making was not at all rational.   So after his clinical training in the 1970s, David Eddy got a PhD in engineering mathematics at Stanford and began his long history of bringing systematic evaluation of evidence to medical care. 

He summarized some of this in his lecture, which he renamed Evidence Based Guidelines, Looking Backward and Forward.  You can learn quite a bit about his experiences, as well as his important contributions to the history of EBM on his website http://www.davidmeddy.com/  Rather than summarize the look back, I encourage you to read it in his own words.  You might also want to read his perspective on the origins of EBM here http://virtualmentor.ama-assn.org/2011/01/mhst1-1101.html

Looking forward, David Eddy made what he anticipated would be controversial recommendations. Among them:

· We need to move beyond Markov models (Dr. Eddy is one of the developers of the method)

· We need to stop using Cost/QALY as a measure since we can’t trust the measurement of either the numerator or the denominator

· We should replace population based guidelines with single, integrated individualized guidelines that:

o   Span all conditions

o   Include all important information

o   Calculate the risk of all important outcomes

o   Calculate the change in risk with all potential treatments

   Then give the information to the physician and patient and let them decide

· Replace current, process-based performance measures with single, integrated outcomes based quality measure

You can read about individualized guidelines in the Annals of Internal Medicine publication of Dr. Eddy’s paper: http://annals.org/article.aspx?articleid=746954 and about the global outcomes metric in the Health Affairs publication of Dr. Eddy’s paper: http://content.healthaffairs.org/content/31/11/2441.full

As always, David Eddy is way ahead of the rest of medicine. We only now have medical specialty societies making recommendations for cancer screening intervals that agree with the evidence based recommendations made by Dr. Eddy more than 30 years ago.  You can learn more about his current innovative work at Archimedes developing models to help us evaluate the risks and benefits from all possible therapies for patients with multiple disease here http://archimedesmodel.com/

I sincerely hope that what I heard was not David Eddy’s Swan Song  (read the Wikipedia definition he provided for a Swan Song here  http://en.wikipedia.org/wiki/Swan_song ) I’d like to think that many others will get to hear his intelligence, wit and his passion for the use of evidence to guide clinical care and its improvement. 

TheEvidenceDoc 2012