Even when a publication only offers statistics (as it usually happens in medical fields) the same requirement stands: the statistics should be reproducible. If it only can be applied for given, local, non-reproducible, conditions, the conclusions are not real science.
John T said...
I was ok with the
opening post, but I'm not so sure I'm on the same page with where this thread is going with the comment and apparent agreement, above. I will note that it is important
to observe the type/quality of publication that information is published in—is it People magazine, or the New England Journal of Medicine. This is incredibly fundamental, but that wasn't a point that's been brought up here (yet). For the rest of this post, I'm only talking about
peer-reviewed medical journals, which I believe was the intended reference to medical field publications in the comment above.
First point, I have never
seen a medical publication "only offer statistics
." Never. If you have any example whatsoever, please post the link here in reply so that we can all be simultaneously stunned.
Second, and tied to the first point, let's review what it means to be published in a peer-reviewed publication: a) Author must submit the article to the journal editor who forwards the article to experts in the same field for review. b) An impartial review panel evaluates the quality, accuracy and validity of the research methodology and procedures. If it's "only statistics," it's not going to get published.
Third point, when there are
issues (like small sample size/low statistical power, as a very common example) with a study (and not all—or any—studies are perfect in the ways we'd like them to be), it's not like the issues are invisible. Too many people, however, fall victim to reading only the headlines. I have no sympathy for those. You have to read and make an attempt to understand the content if you are going to comment or forward a study, in my humble opinion.
Fourth point, is merely a comment on replication; our current system represents an expert initial judgement of whether enough information has been given to replicate the study...but actual replication attempts might then be reserved—from a practical standpoint—for particularly important or controversial or unexpected results.
Is our current system perfect? No. Is it pretty good? Despite the existence of "publication bias," I think it is pretty good and moreover I believe that there is no way to make it bullet-proof.
One aspect of the apparent "contradictions" noted in some of the earlier posts of this thread are, I believe, just the way that the progress of science unfolds in front of us in real time. We are smarter today than yesterday. I'm not so quick to suddenly reverse course on the multiple older opinions of Vitamin D, for example. In my view, we simply may not fully understand, yet, the complete set of variables (perhaps not yet controlled) which influence the results in our very complex bodies. We know there is a correlation
between low Vit D and lots of bad things, but we don't even understand (yet) which came first...the chicken (low Vit D) or the egg (bad things). The best we can really say is that there is conflicting data which hasn't yet identified all the key variables.
Are we having a crisis of confidence on medical studies? Is the sky falling? I don't think so.
How good are studies in peer-reviewed medical journals? Pretty good.
Post Edited (NKinney) : 4/18/2018 12:10:22 PM (GMT-6)