Hierarchy of Evidence – can you trust what you know?

Thank you to the legend that is Abi White for authoring this very important post.

What should you trust in a world of too much information?

“OMG! Wow! Thanks for sharing. I always had a hunch that was true. Now I know for sure…”

Or maybe you don’t. Maybe you just read a headline on Facebook and because it was a published article you believed what they said. If someone wrote it, it must be true, right? So should you or shouldn’t you believe it? And, how the heck do you know the difference?

Here’s the thing: Few things in life are the same, including scientific studies. And just because someone performs a study doesn’t immediately give it a limitless merit. Neither, for that matter, does having a study listed on PubMed – which is simply a free search engine. And just like Google, not everything you find on their search engine will be marvellous. Some studies will be downright shoddy.

“Relying on scientific papers can be highly recommended if you go about it the right way. But this can be hard for the many of us that didn’t spend years at university – or years studying science.”

Introducing the hierarchy of evidence

Although not to be set in stone, broadly speaking, the ‘hierarchy of evidence’ is a way of people ranking evidence consistently from the weakest to the most strong. Of course, there are exceptions but this is a pretty good guide for most of us. So let’s dive in.

⇒  Opinion pieces: Strength – extremely weak

Despite what the name suggests, opinion pieces are not “what Karen said on Facebook” – Facebook comments are not even on the scale. Opinion pieces are articles written, usually by scientists, in academic journals. They don’t hold a lot of value and are simply articles where the author is arguing for a particular scientific position.

⇒  Case reports: Strength – very weak

Described by some as “glorified anecdotes”  case reports ‘report’ on a single event. There are no control groups. Case report could be reporting on things like a person’s reaction to a medication, an individual’s physical changes after an exercise program or observations on some new attribute or behaviour. Case reports are NOT consensus.

⇒  Animal studies: Strength – weak

This is where animals are used to test chemicals or medications to see if they are safe enough to move on to human trials. Animal studies are not taken that seriously because human bodies – and their reactions – are quite different from animals’. Animal studies are used as a starting point and it’s recommended that you don’t base your arguments on such trials.

⇒  In vitro studies: Strength – weak

In vitro means ‘in glass’ – they are studies performed in a test tube or dish. Why are they labelled ‘weak’ in strength? Well, just because bleach can kill cancer cells in a petri dish, this doesn’t mean they can kill cancer in the human body. Disappointing! Our bodies are highly complicated machines, with thousands of chemical reactions happening by the minute so you can’t compare the two. In vitro studies are simply a good starting point for people to follow a small part of the puzzle, allowing them to move onto the next step.

⇒  Cross-sectional studies: Strength – weak to moderate

Also called prevalence studies, they study the prevalence of a particular trait of a particular population at a particular time, as well as looking at any associations between that trait and any other variables. Cross-sectional studies are observational only, and usually involve asking a group of people some questions or assessing medical records. They are looking at how often something occurs and seeing if they can correlate anything from what they see.

⇒  Case-control studies: Strength – moderate

Case-control studies work backwards from an observation. They look at the outcome and try and work out what caused it.It typically involves two groups, one would have the outcome they are focusing on, (the control group) and then another group with a different outcome, but similar confounding factors. (Like one group of athletes having tendonitis, and the other group not, but for example, both groups exercise for the same amount of time each week and don’t have any other injuries). Although useful, it’s still difficult to work out the difference between cause and effect.

⇒  Cohort studies: Strength – moderate to strong

Cohort studies are observational studies of groups of people.They can be forward-looking (prospective) or backward-looking (retrospective). They usually involve studying larger groups of people than case-control studies.

⇒  Randomised control trial: Strength – strong

Considered the gold standard of scientific research, randomised controlled trials (RCTs) are what you’re looking for to help back your claim. RCTs hold the most powerful experimental design and give you the most clear results. They aim to minimise any types of bias to find out the effectiveness of, for example, a new treatment. This is done by randomly allocating subjects to two or more groups, treating them differently, then comparing their response. For example, one control group may be given a medication while another group may be given a placebo.

⇒  Systematic reviews & meta-analyses: Strength –  very strong

Don’t have time to wade through academic studies all day (let alone understand them)? You don’t have to. A systematic review can help you here. Sitting on the top of the evidence pyramid, these are not studies, but reviews and analysis of previous studies. Systematic reviews carefully wade through articles on a particular subject, then conveniently condense the results of several trials into one single paper, explaining everything we know about that topic.

A meta-analysis goes one step further again and combines data sets from multiple papers and run statistical analyses across all of them.

https://thelogicofscience.com/2016/01/12/the-hierarchy-of-evidence-is-the-studys-design-robust/

Bonus section: groups with no scientific evidence

Parents groups, mummy instinct, gut feelings, your ‘really intuitive’ yoga friend, anecdotes from your mum, Facebook friends, YouTube videos; and information from dubious websites such as Mercola.com, Infowars, Dr Axe, Natural News, etc.