But aside from the integrity of the research we practitioners rely on, there’s also a question regarding the quality of research. Meaning, even if there were no concerns for commercial conflicts of interest resulting in omission or skewing of data, the actual merit of the research we rely on is also in question.

In 2016, Fleming et al published a paper in the Journal of Clinical Epidemiology titled "High quality of the evidence for medical and other health-related interventions was uncommon in Cochrane systematic reviews.”[6] The authors looked at 1,394 systematic reviews (SRs) published between January 1, 2013 and June 30, 2014 to assess the quality of those reviews.

Briefly, in case you weren’t aware, in the hierarchy of scientific evidence, systematic reviews are supposed to be the top level. The authors of these types of studies reportedly follow a systematic method of looking at piles of data pertaining to a scientific question from the “highest quality” studies available and then conduct a thorough review and present a summary of the results that help clinicians draw conclusions. Cochrane reviews specifically are regarded as some of the highest quality type of reviews. They use the Grading Assessment, Development, and Evaluation (GRADE) framework to assess the quality of reviews.

Now, when you think about it, you can see the value in having something like Cochrane reviews as they help us busy medical practitioners comb through years of data in one single paper that has already synthesized the information for us. The problem is that as their paper’s title suggests, Fleming et al found that high quality systematic reviews were uncommon.

Specifically, the authors summarized their findings by stating that out of the 1,394 SRs evaluated in their study, only 43.6% (608) actually used the GRADE framework to assess the quality of evidence and of those 608, only 13.5% (82) reported a high quality of evidence, 30.8% (187) reported a moderate quality of evidence and a whopping 31.7% (193) had low level and 24% (146) had very low level of evidence.[6]

Again, to make it clear, these researchers looked at 1,394 of some of the most respected and trusted high level scientific reviews published in the span of 1.5 years. They found that less than half of them actually used a framework to assess the quality of the reviewed studies. Of those that did include the framework, almost one-third where of low quality, almost one quarter were very low quality and less than one third even met the mark for moderate quality. But that means for more than half of the reviewed studies, we have no idea what the quality of evidence they reviewed was because the original researchers didn’t report it. I’m willing to bet they probably weren’t very high in quality.

Dr. John P. A. Ioannidis is a physician and researcher who was one of the publishers of that study. He went on to write an article in the European Journal of Clinical Investigation titled “How to survive the medical misinformation mess.” In his article he breaks down how pervasive the problem of medical misinformation is. In his introduction he outright states:

Currently, there are nearly approximately 17 million articles in PubMed tagged with ‘human(s)’, with >700 000 articles identified as ‘clinical trials’, and >1·8 million as ‘reviews’ (approximately 160 000 as ‘systematic reviews’)... Popular media also abound with medical stories and advice for patients.

Unfortunately, much of this information is unreliable or of uncertain reliability. Most clinical trials results may be misleading or not useful for patients. Most guidelines (which many clinicians rely on to guide treatment decisions) do not fully acknowledge the poor quality of the data on which they are based”[7](emphasis mine)

If you’ve read up to this point, it’s likely you’re blown away by the realization that what we have been taught about medical science and the so-called evidence for what is practiced in the medical field is quite inaccurate. But more than just mind blowing, this reality has important implications for you as a patient.

Firstly, before we get carried away thinking that this implies that all medical practitioners are corrupt, please understand that that is far from the case. We are all human and subject to error. Many of us aren’t even aware that the information we rely on is actually unreliable because, like you, we were raised to believe it was. The point of what you’ve learned here is not to give you the sense that your medical provider is part of the Pharma and Medical device conspiracy, but rather to illuminate that your medical provider may very well be an unwitting participant in this broken system, and you don’t have to take everything that is said as the gospel truth.

The real point of this post is to impress upon you the need that you have as a patient to take seriously the responsibility to inform yourself about your own health. I’m not suggesting that every patient need to get formal training or education in medicine. What’s the point in having medical practitioners then, right? But what I mean is that you should take seriously the responsibility to ask your providers questions. Ask them what evidence they’re relying on. Ask them what their own assessment of the evidence is and why. And if and when you run into a practitioner who seems defensive, explain that you’re just trying to take your health seriously and want to ensure that whatever recommendations being made are based on your providers sound understanding of the strengths and weaknesses of the evidence he or she is evaluating.

I also want to affirm those of you who have walked away from clinical encounters feeling as if you’re not sure what your provider recommended was the best course of action. You’re not crazy. It is a reasonable possibility. And while I do not encourage you to shop around for practitioners that always tell you what you want to hear (that’s not helpful either), I do want to encourage you to make it a point to find one who is willing to educate you and willing to take the time to help you make an informed decision for your health. That includes knowing the limitations of the research and the possible risks that come with making decisions in light of those limitations.