The supposed ‘gold standard’ method of medical research is the ‘randomised controlled trial’. What this means in drug therapy research is that a group of individuals are randomly assigned to the treatment being studied or placebo, and the outcomes and effects compared. In the ‘best’ trials, neither the study participant not the investigators know who is taking what until the ‘code is cracked’ and the analyses are done. These studies are referred to as ‘double-blind’ studies.
‘Blinding’ is important because a lack of it can introduce bias into the study and corrupt its findings.
Let’s imagine, for example, that a study is being conducted in which a statin (cholesterol-lowering drug) is being pitted against a placebo. Imagine, also, that for whatever reason (we’ll discuss how, later) an investigator knows that a participant is taking the statin (not the placebo). He or she is then, perhaps, more likely to dismiss chest pain reported by the participant as ‘non-cardiac’.
The thinking (even unconsciously) can be that if this person is on the ‘correct’ treatment and their cholesterol is well-controlled, heart-related pain such as angina is unlikely. There might also be less tendency for such a patient to be hospitalised or even be subject to certain procedures (such as insertion of a stent into a coronary artery – an example of what is termed ‘revascularisation’).
These sorts of outcomes (angina, hospitalisation, revascularisation) are sometimes referred to as ‘soft outcomes’, because they are quite subjective, and depend quite a lot on the judgement of health professionals, This is not so true for ‘harder’ endpoints such as deaths due to heart attack or stroke.
The ultimate hard endpoint is ‘overall mortality’ (where the ‘diagnosis’ is not in doubt and it’s not open to interpretation). The other good thing, by the way, about overall mortality is that it is a measure that encompasses everything (deaths from all causes).
The thing is, though, some study investigators choose to include soft endpoints in analyses. The rationale is often that lumping a bunch of different endpoints together makes it more likely that the result will reach ‘statistical significance’. What this means in reality, often, is that overall benefits can be claimed for a drug that actually did nothing in terms of improving hard endpoints such fatal heart attacks and overall mortality.
A very interesting paper published recently in the BMJ makes the points above, but also draws our attention to how blinding may be lost, including in studies on statins [1].
Generally, when a patient is engaged in a study, they do not live in a bubble during the process. They still, say, have health issues and go and see their regular caregivers. When a doctor looks at someone’s health records, they may notice that since their patient entered a statin study, their LDL-cholesterol level dropped from 3.5 mmol/l to 1.5 mmol/l. Recently, while reviewing a case, I came across an article analyzing how cognitive bias impacts clinical decisions. Interestingly, it drew a parallel with industries like gaming, where choices are also influenced by subtle cues—such as the design and marketing strategies employed by the best online casinos to attract users. In both cases, professionals are challenged to stay objective despite external factors. This kind of bias, in a healthcare setting, could lead the doctor to conclude that the patient is on the active drug (statin) rather than placebo, influencing decisions like dismissing chest pain as ‘non-cardiac’ or reducing the likelihood of hospital admission or revascularization referrals.
It occurs that it is not impossible for blinding to be lost for the participant too. Finding out that one’s LDL-cholesterol has dropped from 3.5 to 1.5 mmol/l could lead some to be more likely to dismiss, say, some chest pain, and be less likely to report it to their doctor.
The author of this piece points out that in the majority of statin trials, we have no idea how successfully blinding was ensured. This is not a minor issue, either – it introduces bias that might be corrupting the evidence base for statins (and other drugs).
What to do? The author has two suggestions:
1. One idea would be to enforce blinding strictly. However this would limit, say, access to health information (e.g. cholesterol levels) by regular caregivers.
2. Alternatively, we can accept that blinding can be compromised, and choose endpoints that are less prone to bias. In other words, we should concentrate on analyses of good, hard endpoints such as overall mortality.
As the author points out, the first suggestion might compromise care of patients. Regarding the latter, the author tells us that: “However, most deaths, especially in cohorts with low cardiovascular risk, may be non-cardiovascular and therefore not amenable to statin prevention.” This article is, I think, extremely important and well-argued, but my response to the author’s last point is “So what?”: If a ‘life-saving’ drug turns out to do no such thing, then perhaps we need to know.
References:
1. Nguyen PV. Electronic health records may threaten blinding in trials of statins. BMJ 2014;349:g5239
Well written, especially your last point!
Key reason many people accept to take statins is just that they have been scared with early death, if they don’t! But if death follows sooner with the promised pills, although from other causes than the pills are known to cause, every reasonable mind will conclude that it has the bad effects showing up in the mortality statistics. This is what controlled random studies are for; that is what they find out; before tampering that is.. The overall mortality risk must never be allowed to not be put in as marker number 1, and published like that to all users.
The fact that statins tend to raise blood sugars is enough to say no to them as high blood sugars promote plaque growth in itself. Only counting CVD-related deaths gives the study researchers the larger room to reclassify deaths to support expected or required outcomes, set by we know who.
The accord study was stopped when death from all causes rose in the arm exposed to intensive blood sugar control. Apparently no obstacle is high enough for statin makers.
Excellent Stan! It would be unethical to continue a test in which harm is found by the active drug. Its also unethical to not report that for doctors use and this action also gives the impression that corporate sales are more important than patient health is to research management.
As nice an exposέ that I seen of the RCT. One minor comment:
“What this means in drug therapy research is that a group of individuals are randomly assigned to the treatment being studied or placebo, and the outcomes and effects compared.”
If the “GROUP” is defined then the randomization only applies to that group. Example: Collins in the HPS study pre-randomization had selected patients tolerant to statins. Thus his results only apply only to those who would have met the inclusion criteria; not to all and sundry as he claims.
Bias is always a major problem but, as you say, blinding helps to minimize this. However, the key to the blind should never be released until completion of the study. Unfortunately, it is clear that in many studies this is not the case. This is particularly evident in those studies which are terminated early. Someone was peaking! Blinding negated.
Your example of cholesterol lowering or any clinically observable change unintentionally breaks the blind which can then influence “soft endpoints”. It then comes down to basic scientific integrity which frankly is now subservient to MONEY and STATUS; a feature of the HPS study.
Thank you Dr Briffa for your comments on my article and further in-depth explanations of my point of view.
Now you can go through any of the major statin trials (the ones used in the CTT meta-analysis) in the light of what we have written and you will find out that they are all NEGATIVE (except perhaps for the 4S trial whose results were never being replicated). It was the “soft” endpoints, “(conscious or unconscious) bias”-proned (diagnosis of angina, diagnosis of non fatal stroke, decision for hospitalization, decision for catheterisation, decision for stenting, successful ressuscitation from cardiac arrest,…) that saved the day for them !!!!!
Is it time to write the obituary for the statin saga and the cholesterol hypothesis ?
Paul v Nguyen MD Internal Medicine CHUM
On the other hand, considering all the side effects, some of them extremely serious, doctors need to be aware that their patients are on a statin. Or at least that they could be on a statin. If they are completely in the dark, they could dismiss muscle pains or other symptoms as something else. To me that’s more serious that some vague concern about being unconsciously influenced by knowledge of a trial their patients is engaged in.
Anyway, would doctors really dismiss signs of angina, or signs of heart disease because of a statin ? That would be poor treatment indeed.
Another example: The HPS was designed as a 2×2 factorial study but the analyses of the study in that format were never, ever published. WHY? I am entitled to speculate in that absence; The analyses DID NOT SHOW a significant benefit in statin treatment otherwise it would certainly have been published.
Furhermore, the interaction between the two treatments (the 2nd treatment being an anti-oxidant vitamin cocktail, MINUS COQ10! WHY given MERCK’s patents combining it with statins?) was also not significant but possibly between p = 0.1 but > 0.05, Thus allowing the combination of the two groups with statins to be compared with the two placebo (one with the Vit cocktail) as published in the main paper. This resulted in a “significant”, positive result to be published. Another example of the use of the verb “to doctor” (see definition in the Oxford Dictionnary) in statin research.