How to identify good science

Wine drinkers and chocolate lovers love it when a new study comes out claiming that their favourite indulgence is good for us. This type of media gives us the feeling that we don’t have to feel guilty about eating something we love.

This is a natural reaction—nobody likes to feel guilty, and we want to be able to justify our actions. However, it seems like we are getting a little too good at extrapolating, confirming our own biases, and failing to see the big picture. For instance, the cocoa bean has health benefits but how many of us chocolate lovers are eating 100% cocoa powder? Hot chocolate, milk chocolate, chocolate candies and chocolate chip cookies are not the chocolate with health benefits. As for wine, the scientific community is still debating whether it is healthy.  Good science seems to tell us that abstaining is healthier than consuming one glass of wine a day and according to the Mayo Clinic “it is not advisable to start drinking alcohol for health benefits.”

We are quick to latch onto a study that tells us what we want to hear, and it is our tendency to make that leap that I want to call to your attention. Being aware of your bias is critical in being able to evaluate good science. I love sugar and I do not like alcohol, so I want to believe that alcohol is bad for us. I think chocolate cake is a beautiful thing to do to cocoa. I am fully aware of my bias so when I am reading studies on alcohol and chocolate. Instead of looking at what the study says, I first ask, is it a good study?

A new language

Here, I want to give you a new language which allows you to read articles, journals, and studies on fitness, health, and nutrition with a critical mindset. In general when it comes to distinguishing which studies are good from which studies are questionable, look for the following:

• Randomized: A Randomized Control Trial (RCT) is a study in which the participants are assigned by chance to separate groups; neither the researchers nor the participants can choose which group.  (PubMed Health Glossary-Source: NIH – National Cancer Institute).

• Peer Reviewed: A Scholarly peer review (also known as refereeing) is the process of subjecting an author’s scholarly work, research, or ideas to the scrutiny of others who are experts in the same field, before a paper describing this work is published in a journal or as a book. (Wikipedia).

• Double Blind: A double-blind study is one in which neither the participants nor the experimenters know who is receiving a particular treatment. This procedure is utilized to prevent bias in research results. Double-blind studies are particularly useful for preventing bias due to demand characteristics or the placebo effect. (Verywell.com).

It is helpful to become familiar with these terms as well:

• Anecdotal Evidence: Evidence from anecdotes is evidence collected in a casual or informal manner and relying heavily or entirely on personal testimony. (Wikipedia). Dictionary.com defines anecdotal evidence as non-scientific observations or studies, which do not provide proof but may assist in research efforts.

• Cherry Picking: Cherry picking involves suppressing evidence because a company or person selects which studies it is going to reference to support their claim, or to confirm a particular position while ignoring a significant portion of related cases or data that may contradict that position. (Wikipedia).

• Extrapolate: This means to infer or extend the application of (a method or conclusion) to an unknown situation by assuming that existing trends will continue or similar methods will be applicable. (Oxforddictionaries.com).

In addition, look for studies that are current and that have been reproduced several times, by different researchers.

How I read fitness media

Before I read an article I do a quick search on the author to find out what is their background and who is paying them. For example, a friend of mine follows the Paleo diet and I am skeptical of this diet, as it eliminates legumes, which is a very healthy food group and a staple in my diet. So, I did a quick Google search for “paleo diet randomized studies.” I found some interesting articles and this one was particularly good—it examined several studies on the Paleo diet. 

Adding the terms I provided above to your vocabulary gives you a new language allowing you to be more critical and informed. Once you know the words to use in your searches, it becomes a lot easier to weed out the “bad science” and become aware of the “media’s habit of oversimplifying and overstating study results.” (American Journal of Clinical Nutrition, “Paleo diet still lacks evidence”).

For an example of this, read my article in which I review two popular books on nutrition.