Science Isn’t Perfect | Part 1
In the years that we were still scrolling through our Facebook timelines and Instagram was a distant addiction, we all probably liked one or two posts similar to this:
“New Scientific Study Shows Dark Chocolate Makes You Lose Weight”
Perhaps we tagged some of our friends in this post, declaring that we found our new favorite diet and then scrolled on to find the seventh cute cat video of the day. Ah the good old days.
That day we might have easily moved our attention away to something much more imperative and entertaining, but many of these social media headlines do remain stuck in the back of our heads and shape the way we think and live our lives.
Now, this article is not going to be about the continuous health fads people seem to immediately embrace as the new dietary truth. This is however still a serious issue. No, today I want to tell you about John Bohannon, a science journalist with a PhD, who was able to fool the general public and illustrate one of the most significant shortcomings in the scientific and journalistic world. How he did that?
By conducting a real scientific study which concluded from honest results that chocolate makes you lose weight.
The tastiest conclusion I’ve ever heard. How wonderful would it be if we could obtain our ideal body by just eating chocolate every day!?
Well, that attraction was what John Bohannon and his team were counting on to spread their false statement across the globe. With their elaborate hoax, they exposed how badly nutrition science is reported on and dispersed in the media.
So you might have seen it coming, I do have to disappoint you (and myself). Chocolate does NOT make you lose weight.
Yes, the study and the results were real. 16 test subjects followed a low-carb diet, of which half ate 40 grams of dark chocolate daily. In the end, this group lost their weight faster than the control group that was only on the low-carb diet.
However, the conclusion that chocolate helps you lose weight is still not valid due to several reasons.
First of all, the study design itself was complete nonsense. 8 subjects per test group is not enough for a trustworthy study as individual deviations have a large effect on the average results of the entire group. Plus, the research included a large number of potential factors, forcing the likeliness that some random factor would appear to have statistical significance.
“You have to know how to read a scientific paper — and actually bother to do it. For far too long, the people who cover this beat have treated it like gossip, echoing whatever they find in press releases. Hopefully our little experiment will make reporters and readers alike more skeptical.” – Bohannon
Then there is the journal that published the research. Because of increased pressure on scientists to publish their study, so-called pay-for-play journals have risen in popularity. In order to do research, scientists need money. Unfortunately, the chance of getting that often depends on how many times you were able to publish your research. These type of journals however very quickly accept any type of (including bad) research without anyone reviewing the actual paper. Thus giving the opportunity for mistakes and wrong conclusions to deceive people due to a seemingly professional seal of approval.
Finally, all of the reporters who covered the publication accepted it with blind faith. But good scientific reporting always requires the opinion of an outside expert. In this case, it would have stopped this falsehood to enter the minds of millions.
If you want to dive deeper into his study, its problems, and its purpose, Bohannon beautifully explained everything himself right here.
Whether this study was really ethical of Bohannon remains a question…
You might think this is just a one-time thing, not to mention deliberately done. Something like this would never happen normally. But that’s where you are mistaken. There are many flaws in the world of scientific reporting and not only in the field of dietary science. This is what I want to continue to address in my Science Isn’t Perfect Series.
In the end, what should we learn from this? Well, we always need to remain critical, even if the title of an article mentions a scientific article. Scientific publications are not always correct (think of the paper that linked vaccines to autism which has been debunked countless times), scientific journals are not always trustworthy, and the media loves sensation a bit too much.
Especially with social media and social influencers these days, it is imperative to remain critical. By the way, did you know that one shot of tequila a day cleans up your skin completely?! It’s amazing and DEFINITELY true. You should trust me, I’m a scientist.
Just one more thing! I highly advise you to read this article of Vox, which debunks the entire health glorification of dark chocolate. Hint: A lot of positive chocolate studies are directly paid for by the chocolate industry.
Photo credits in order of appearance: Charisse Kenion on Unsplash, Jessica To’oto’o on Unsplash