How to design a trial to ensure you don't get the result you don't want

Earlier this month I wrote a post that focused on a trial involving the cholesterol-reducing drug ezetimibe. Ezetimibe works in a different way to statins (the most commonly prescribed cholesterol-reducing medication). While statins block cholesterol’s production in the liver, ezetimibe block its absorption from the gut.

When you add ezetimibe to a statin (there’s a medication called Vytorin that includes both), you can surely get a stunning reduction in cholesterol numbers. The problem is, as I’ve been at pains to point out before, the effect drugs (or anything else) have on cholesterol is irrelevant – it’s the impact they have on health, that counts. Trust me when I tell you if can lodge this fact in your head, you’ll be way ahead of many doctors and researchers in your thinking.

Ezetimibe beautifully demonstrates why it’s important to focus on its impact on health rather than cholesterol, as not one single study to date has shown it to improve clinical outcomes. In fact, as the study I wrote about earlier this month demonstrates, there is evidence that ezetimibe might actually worsen outcomes. Yes, that’s right, we have a system that allows we doctors to dispense drugs that have not been proven to be beneficial, and where the balance of evidence suggests they have the potential to cause harm. This is what happens when those we entrust to keep us safe buy into the ‘cholesterol is king’ mentality that the drug companies push incessantly. My advice is not to buy into it.

There simply isn’t any truly good news about ezetimibe, and on a deep level its manufacturers (Schering-Plough Corporation) must be disappointed and embarrassed by it. But that won’t stop the research. In such a situation all you can do is conduct more and more studies and hope, in the end, something sticks.

In the bad old days, negative studies with disappointing results for companies and their shareholders could be ‘buried’, which made things a lot easier. Now, there is much more pressure to publish negative results as trials are generally registered prior to getting underway. So, researchers and the drug companies who pay them are having to get increasingly creative to ensure they get the result they want (or avoid getting the result they don’t want).

By way of example, I’d like to discuss briefly another ezetimibe study that was published recently [1]. The study subjects had varying degrees of kidney failure. About half of them were treated with simvastatin (a statin) and ezetimibe for an average of almost 5 years, while the rest got a placebo (inactive medication).

The results showed that those taking the drugs had a 17 per cent reduced risk of the ‘primary end-point’. The primary endpoint, though, was made up of a rag-bag of things including heart attack, fatal heart attack, ischaemic stroke (stroke caused by vessel blockage) and stenting (the placing of a tube in an artery to keep it open). The more things you group together like this, by the way, the more likely you are to get a ‘statistically significant’ result.

Also, the primary endpoint excluded death hemorrhagic stroke (stroke due to vessel bleeding), which is salient seeing as lower cholesterol is associated with an increased risk of this.

Focusing on individual outcomes for a moment we see that the treatment did not lead to a significant reduction in non-fatal heart attacks, fatal heart attacks nor, crucially, overall risk of death. In other words, the treatment did not save lives.

But never mind these results, for a moment. Let’s look at the broad design of this study. Here, the active treatment was statin and ezetimibe. The comparison group took a placebo. How do we know from this study if ezetimibe contributed in any way to the (unimpressive) benefits seen? We don’t. To know if ezetimibe added something, the study would need to have included a group taking a statin on its own.

Now, remember we have previous evidence that ezeitmibe not only does not help when added to a statin, the outcomes can actually be worse. Of course, one way to prevent such embarrassments and dips in share price would be to design your trial in a way that stands no chance of detecting such things. I think the researchers responsible for this latest trial need take note.

References:

1. Baigent C, et al. The effects of lowering LDL cholesterol with simvastatin plus ezetimibe in
patients with chronic kidney disease (Study of Heart and Renal Protection): a randomised placebo-controlled trial. Lancet. 2011 Jun 8. [Epub ahead of print]