Evidence is Gambling, and we are all Addicts
Evidence. It’s such a powerful word.
Who wants theory, speculation, falsehood when you can have evidence? There are even those who want the power to insist on certain types of evidence. Evidence is real, tangible and useful – or that’s what we would like to think.
Are we addicted to evidence to the detriment of ourselves and our patients?
It seems that in the world of Evidence Based Medicine, very few people take the time to examine what evidence really is, what it can be used for and how useful it is.
Sackett’s original dream for the practice of EBM was to give practitioners the tools to be able to search for and use published evidence in order to make evidence-based decisions.
In 2007 there were 25361 RCTs published, out of 679,858 medical peer reviewed studies. How many of them did you read? Did the practice of medicine change at all as a result of those studies? What was the return on investment? How can it be that something that consumes so many resources (time and money of researchers and consumers) provides so little actual benefit?
Like you, I want to stay up to date. I probably read the abstracts of 1000 published studies a year, chances are you do too. They are interesting and informative but are they useful? Not at all. Are they a good return on investment? Not a chance.
There are two types of knowledge. Useful and useless. There is nothing inherently wrong with useless knowledge, it’s often quite interesting. Where useless knowledge becomes dangerous or counterproductive, is when we don’t recognise the knowledge as useless and try to let it influence future decisions.
Randomised Controlled Trials (RCTs) have a very specific usefulness, being helpful to a patient is not one of them. The usefulness of an RCT is to remove chance, bias and confounding. Sadly, chance, bias and confounding are NOT causes of disease and are therefore of no concern to the treatment of disease.
For knowledge to be useful to an individual, it must have predictive value. Without predictive value, all you are doing is gambling.
If a piece of knowledge has no predictive value, you may as well be sitting at a blackjack table. The blackjack player has access to extensive knowledge about the possible outcomes and the odds of each outcome but all of that knowledge cannot tell him what the next card or the next hand will be. Which might be OK if your decision is whether to have the strawberries or the ice cream, but what if that next hand was the life of your brother, mother or sister? What if the choice you made really did matter? Are you still happy gambling?
You may be happy gambling with the health effects of eating a piece of carrot, but are you as happy gambling with open heart surgery? In the world of clinical research, both of these interventions are subject to the same levels of proof, even though this is clearly absurd.
An RCT might have a thousand patients. Twenty-five percent improve on the treatment, eighteen percent improve on the placebo, the result reaches statistical significance, is considered “proven” and may be recommended to everyone with that condition, even though the majority in each group had no benefit. In the treatment group, some patients improved, some got worse, most stayed the same. In the placebo group some improved, some got worse, most stayed the same. When the next patient walks in it is impossible for the doctor to know whether that patient will benefit most from taking the treatment or the placebo based on the “evidence” of the RCT. He or she is gambling.
Let’s be very clear here. When you sit down at the blackjack table, you can know the odds. When you do a perfect RCT, you know the odds. You might know that the positive result from the study will be wrong one time in 20. You might know that you have to treat 50 people for one to benefit. You might know the percentage that will have an adverse reaction but just like you can know the odds in blackjack, you cannot tell what the next hand will be. It’s the same with medical care based on RCTs.
If we want predictive value, we have to study that which is real and consistent – which is why science works extremely well in the physical world with the hard sciences. Science works best with real things that have little variability and where the factors that influence variability are known. In other words, science works best when we can be reasonably certain that the reality we observed yesterday, will still be the same reality we observe tomorrow. It is very easy to experiment using controls when the subject is constant and the important variables are known. This is what gives us predictive value. We can’t get predictive value when the situation we are trying to study has unlimited variables and each variable has an unknown importance.
Studying a human being is like studying a famous battle. A battle is a complex interaction with thousands of different and rapidly changing variables. No matter how often you studied that battle, you could never predict the outcome of the same battle on any other day, in any other location or in any other situation. The outcome could be changed by any number of unforeseeable circumstances, from the way the General slept the night before, to whether a stray bullet knocked out specific gunner, to whether the politician who ordered the war had needed to impress the father that neglected him as a boy.
In medicine we would like to believe that by randomising patients to two different groups we can average the unknown variables and therefore measure the influence of the one variable we would like to measure, which we can do, as long as we realise that the result we get can only be applicable to the group, not the individual. The mistake of thinking that the average result can and should be applied to any individual is known as the ecological fallacy.
An RCT is generally constructed to investigate the effects of a treatment on a condition. We measure the condition before and after the imposition of a treatment or a non-treatment and compare the outcomes. Unfortunately, a condition is not a thing. It does not exist independent of it’s host and it is not constant from day to day or from host to host. It is not like a brick or a chair. Things that are real can be studied because the important variables that go into their formation are known and their nature remains consistent. We might like to presume that low back pain is a condition that can be studied but what if it isn’t? What if low back pain is just a symptom that might be caused by 100 different factors or a unique combination of any or all of them? How would an RCT help us then? Changing just one variable at a time would never allow us to reach statistical significance, which is why RCTs generally produce such insignificant results.
We are good at treating conditions where the cause is known. Bruising, broken bones, sprains and strains caused by trauma are easy to treat because the cause is known and in many cases the external force that caused it has passed.
We can’t reliably cure a condition like diabetes or sciatica because we don’t know what causes them. We don’t know what the external or internal forces created them or whether those forces are still present or not. We don’t know whether the cause is singular or multifactorial.
Perhaps the fact that diabetes and sciatica are not real is why modern clinical science is unable to find their cause and the causes of the 10,000 other named conditions that have no known ethology.
The reality is that all conditions are merely a normal physiologic reaction to one or more external forces (toxicities) and the host’s ability to defend against those forces (weakness and deficiency).
Reducing or removing the external forces and strengthening host deficiencies or weaknesses are the only treatments we know that are truly reliable and effective. Everything else produces only temporary, spasmodic or occasional benefit.
It’s time we stopped looking for “treatments” for “conditions” and concentrated on looking for ways to avoid that which is destructive and introduce only that which is strengthening and constructive FOR THE INDIVIDUAL. Only then can we get truly patient-cantered healthcare.