6 Comments

This is just some more junk science by an advocacy group.

That study, published by the National Bureau of Economic Research (NBER) in March, suggests that publication bias—the tendency to report positive findings and ignore negative ones—explains the “consistent” results highlighted by the IOM committee.

Publication bias a is polite name for scientific fraud. In any large nationwide sample of data one can find locations where the values are above or below the average. In fact, if the data is truly random then by definition of randomness we will find such locations. That is why in the absence of any other evidence cancer clusters are meaningless. In graduate school we used to joke about throwing away data that did not confirm our hypothesis because it was obvious that the data was in error. Unfortunately now we have so-called researchers that publish statistical anomalies as if the data was statistically significant. Anyone who publishes such data should be barred from doing research because they are nothing but charlatans.

Publication bias isn’t necessarily fraud (although it certainly could be, if motivated by the intent to hide data that might cut off ones gravy train – read AGW, the anti-smoking bunch, all drinking is evil, drug warriors, chemo-phobes, many environmental groups, especially the EPA, etc).

It’s simply “not interesting” to all those high power journals report a negative or non-finding. Who gets the big corner office in Research U by publishing non-results (such and such has no discernible effect on malady of the day)?

That said, knowing what doesn’t work, or doesn’t cause something is important to those of us in the scientific or engineering field. It allows us to avoid dead end’s in searching for solutions (as in – great to know that Sally and Joe tried that before and it didn’t do squat, lets focus elsewhere) .

Publication bias isn’t necessarily fraud

I guess they could also be incompetent researchers.

That said, knowing what doesn’t work, or doesn’t cause something is important to those of us in the scientific or engineering field.

Yes, in physics, negative results are just as important as positive ones. It would have been very unfortunate if there had been “publication bias” in the Michelson-Morley experiment!

The Framingham heart study and the World Health Organization MONICA study found no statistical association between smoking and heart disease. These were two very big studies with large sample sizes. The anti-smoking zealots never mention these studies.

RN – Perhaps we’re coming at the definition of publication bias from different perspective. I’m coming at it from (partially at least) the perspective of the journals. It would appear you’re coming at it exclusively from the perspective of the researchers.

Lets say for the purposes of discussion:
You’re the editor of Big Prestigious Scientific Journal (BPSJ).

You have only room for so many articles an issue.

So, are you going to fill BPSJ with articles and studies showing compound Y didn’t affect the environment / cancer rates / concern of the day, etc?

Or are you going to fill BPSJ with articles and studies showing compound Y in parts per billion concentrations kills cute furry critters faster than shooting them in the head and will cause the entire atmosphere to blow off into space and that we need more money to study the problem of compound Y?

Which of the above do you think will see the light of a cover article on BPSJ?

One form of publication bias comes in that editors of BPSJ would select the latter type of article for publication over the former type of article. Both could be on compound Y, both could be equally correct on describing the actual physical results of compound Y, but the latter would more likely to get published while the former would be passed over (and we all know why this is the case). This has nothing to do with the competence or lack thereof of the researchers.

It’s this form of publication bias that then shows up in later “meta” studies, as the published studies that are the raw data for the meta study aren’t representative of all information on the subject.

And yes, it would have been a shame if MM wasn’t published. How far we as a society have fallen into this post scientific age. Just goes to show what is wrong in the world where journal editors gate keep out unfavored views, when peer review replaces replication, where some claim their methods, computer codes, etc are proprietary, where it’s about “publish or perish”, where computer models instead of experiment and measurement of the physical world pass for evidence, etc.

My statistics classes back in the 1960’s used to set alpha = 5% as the test of significance of the null hypothesis. That implied that roughly 5 % of significant findings could result from chance alone. But people had to publish or perish. This publication bias was well explained in the comments above. The comments on this site are often of high quality, and I thank Mr. Olson for providing us with this wonderful site.

My sense of the matter is that our medical and safety institutions have been corrupted by Naderites. One is astounded that Dr. Sidney Wolfe is in our government despite his horrific behavior with respect to breast implants.

I still remember the joy resulting from the declaration that “it works” for the Salk vaccine. Yet greedy lawyers and ignorant citizens actually put children at risk of the doom of iron lungs.

We have put all kinds of efforts in “no child left behind” for our schools, yet Jon Stewart, a really smart guy, can not challenge his assumptions about the first responders. Air over Manhatten is completed turned over at least once an hour. Claims of toxic dust are
ridiculous. Any ‘scientific” findings would fit better with suggestion and self-serving PR from stupid doctors.