Research studies: “inconclusive” doesn’t mean “unscientific”
Friday, August 28, 2009 14:43Megan Steintrager, who covers the health beat over on the Epicurious Blog recently wrote about two new studies dealing with dietary fat–one seemingly "pro" and the other "con." (See her post: "This Week's Big Fat News")
The first study, which I discussed in this post, observed that people whose diets were higher in saturated fat gained less weight around the waist. Fat is good! The second study Megan mentioned was an equally intriguing mouse study which found that high-fat meals seemed to impair memory and athletic performance. Fat is bad!
Megan asked her readers what they made of studies that seemed to point in two different directions. How do you make decisions about your diet when the researchers can't seem to agree on the most basic issues?
Frustration and disdain
Not surprisingly, many of Megan's readers expressed frustration and even disdain for the whole field of nutrition research:
"The studies happen because someone's got to justify their job/funding, etc., or they are being paid to do it by Big Business."
"I've heard enough waffling between positions to ignore all the noise."
"You frequently bring attention to these (highly unscientific) studies"
"Love these "studies'. Lots of words like 'appears to' or 'suggests', or
'could'. Who commissioned the studies and what did they have to gain
from the "results"?""I feel like the news about nutrition studies is so often contradictory
(fat is bad! fat is good!) that more and more I just ignore it."
I hear a lot of the same sentiments from commenters on this blog as well, and I can understand why. Here you are, trying to pay attention, trying to make good choices and these dratted scientists keep yanking the rug out from under you. While I think this "research fatigue" is understandable, I think some of the cynicism is misplaced.
The problem isn't with the studies
Just because a study is inconclusive does not necessarily mean that it was unscientific. Good studies aren't designed to answer questions such as "Is fat good or bad?" They're designed to gather data about one tiny aspect of a question: "Does changing the amount of fat in the food change a rat's ability to run a maze?" Or, they're simply mining existing data for possible links: "Can we observe any connection between diet patterns and waist size?"
Although it may seem trivial or pointless, it is worthwhile to ask questions like this. The scientific process moves forward one tiny fact, answer, or observation at a time. Usually, a study raises more questions than it answers. So we keep gathering information, testing hypotheses, and looking for patterns. Zeroing in to see if we can pin down a detail, then stepping back to look at the big picture and see if anything has changed.
In other words, I think the scientific process is, for the most part, proceeding exactly the way it needs to. The problem is not with the studies. The problem is that these incremental, partial, and inconclusive findings are funneled–incessantly and indiscriminately–to the public as the Final Answer.
Following the money trail
I also want to say a quick word about funding. Research is expensive and money is tight. Many times, commercial interests feel that they would benefit if research on their products moved faster. Therefore, a lot of research is funded by corporations and trade associations. But corporate money does not necessarily invalidate the results.
Many sponsored studies go through rigorous vetting by Institutional Review Boards (IRBs), which critique and approve study designs before the research begins, and peer-review boards, which examine and validate the results before they are published. It's not a perfect system but it's an attempt to insert a firewall between the money and the science.
That's not to say that every study is equally well-designed or that it's not important to know who funded a study. Caveat emptor applies to consumers of research just as much as it applies to consumers of anything else. But those details are made fairly transparent.
Finally, just because a study was funded by a government agency or university does not guarantee that the researchers had no bias or prejudice. An ideological bias (such as a personal conviction that saturated fat causes heart disease) can be just as powerful as a financial tie (such as your lab being funded by a dairy farm). In fact, in a way, ideology can be more treacherous because it's often less visible.
What's the point of following research?
When it comes to diet, simple common sense and moderation go a long way. If you were to revamp your diet in response to every new study, you'd surely go nuts. But I still find it interesting and worthwhile to follow and discuss the research as it comes out.
Believe it or not, the scientific community is learning valuable things about diet and nutrition–things that can make a positive difference in your health. We just probably won't have it all figured out by Friday.