How can you tell if the claim for the latest potion, pill or procedure is based on good science or junk science? There are so many scientific claims made—especially around health and fitness—that I really want to help my readers understand the scientific method, in hopes of learning the difference between legitimate and illegitimate claims. This is a different style of post and longer than usual. Bear with me and (hopefully) I’ll give you some useful information.
I am both a person of faith and a person of science. My education includes a Bachelor of Science degree in exercise science, a Master of Science in counseling and a Master of Arts in theology. I consider faith and empirical knowledge different but complementary ways of thinking about reality. It’s very important for me to know when to apply each. On Sunday morning at St. John’s Episcopal Church, when I recite the Nicene Creed, I’m very clear that I’m making a faith statement. When I tell a client that her cholesterol level puts her at greater risk for heart and vascular disease, this is a statement based on empirical evidence. I don’t expect empirical evidence for matters of faith and I don’t believe in that which is demonstrated empirically.
It troubles me to hear otherwise intelligent, well-educated people accepting claims for which there is little—if any—good science. Let me unpack the phrase, “good science.” Good science requires that research is conducted according to the standards of practice for each discipline. Physicists, psychologists and physiologists, for example, each have their own professional organizations that set standards for research. Reputable scientific organizations hold meetings and publish journals to report the results of studies conducted in their discipline. A scientific paper will contain the following: 1) the purpose of the study, 2) a review of previously published papers on the same topic, 3) a detailed account of how the study was conducted, 4) a statistical analysis of the data and,5) a conclusion. Prior to presentation at a meeting or publication in a journal, researchers submit their papers for peer review. In the peer review process, a panel of similarly qualified researchers read the paper, anonymously, and scrutinize it for methods, statistical analysis and conclusions. The panel may approve it for presentation or publication or send it back to the author for changes. Good science is also replicated, meaning that other investigators conducted the same or asimilar study and reached the same or similar conclusions. When testing the effectiveness of, say, a new drug, a double-blind method is the standard of practice. Double-blind means neither the investigator nor the participant knows who’s getting the actual drug and who’s getting the placebo. If a researcher has a financial relationship with, for example, the drug’s manufacturer, this must be disclosed. Replication by investigators not having any financial relationship is usually done to validate the original study. In a nutshell, good science is conducted according to established protocols, is peer reviewed, presented at a reputable meeting and/or published in a reputable journal; financial relationships are disclosed and it is later replicated by others with no financial interest in the outcome. When this process has worked long enough, consensus is established.
In contrast to good science is what is often calledjunk science. One example of junk science is a manufacturer testing its own products on a small number of people and calling it research. A “doctor” (doctor of what, it doesn’t seem to matter) writes a book or puts up a website and makes any claims he or she wants and it passes for authoritative. A prominent TV doctor has managed to convince many people that coconut oil lowers cholesterol. This directly contradicts the positions of the American Heart Association, The Academy of Nutrition and Dietetics, the National Lipid Association and the Center for Science in the Public Interest. Scientific consensus is that while coconut oil may, in the short term, raise HDL (or good cholesterol), it doesn’t result in decreased risk of heart disease. In the long term, coconut oil raises LDL (or bad cholesterol), which increases the risk of heart disease. When you hear or see claims for a product or service, ask yourself if these claims have been tested by good science, as described above.
In my field of health fitness, there are all kinds claims made for “scientifically-proven” products or practices. (Good science never claims to prove anything. Rather, scientists say things like, “All available evidence supports X” or “There is a lack of evidence for Y.”) Fortunately, the American College of Sports Medicine produces its Guidelines for Exercise Testing and Prescription. This book is the closest thing my profession has to a standard of practice. Every five or six years a committee of some50 authorities in medicine, public health, exercise physiology, kinesiology and other disciplines conduct a thorough review of all related scientific literature. Based on this review, they reach consensus on what represents the best practices for health fitness professionals. It is on these practices that I was tested on my certification exam. I deliver my services to my clients based on the Guidelines.
Good science isn’t easy, simple, quick or cheap. It’s complicated, laborious, and expensive. It sometimes it takes decades to reach consensus (think female hormone replacement). Good science is all about establishing the safest and most effective products and practices. In contrast, junk science is easy, simple, quick and. It’s all about selling something, be it books, potions, pills or procedures. The next time someone tries to sell you the latest potion, pill or procedure, tell ‘em, “SHOW ME THE SCIENCE!”
David Wheeler, MA, MS is Wellness & Health Recovery Coordinator at Premier Health & Fitness Center. He is an American College of Sports Medicine Certified Exercise Physiologist. David provides fitness training and health coaching for those contending with health challenges and for healthy adults who want to stay that way. He can be reached at email@example.com or 850-431-4835.