Publication Bias, 1916 Style

Whilst browsing Wikipedia, I came across a poignant early example of publication bias, the failure to make public scientific results that don't support a given hypothesis.

The Judenzählung was a census of the German military carried out in 1916, at the height of the First World War. It was designed to measure the number of Jewish soldiers in the army.

The background to this was the feeling, very powerful in Germany at that time, that the Jewish German minority were "unpatriotic" or "traitorous", and were dodging military service or avoiding front line combat.

The survey was completed, but the results were not published, apparently because they revealed that, contrary to popular belief, Jews were at least as likely as non-Jews to be serving in the army, and were overrepresented on the front lines as well. (Some of the data did emerge after the war, however, when they were criticized for being inaccurate by Jewish groups. 12,000 German Jews died in battle during the War.)

This is an exceptional example of publication bias, but in essence it's no different to what happens when academic or corporate researchers decide not to reveal data which they're not happy with, for whatever reason. The best-known culprits are pharmaceutical companies who often decline to publish data showing that their drugs don't work, but it's a problem that affects most of science, and Big Pharma are certainly not the only ones doing it.

One solution is to have scientific journals or websites dedicated to publishing "negative" results, and there are several, but this still relies on people choosing to reveal their data. It seems to me that, ultimately, the best way to combat publication bias is to require the pre-registration of studies, so that everyone knows in advance what research is being done, and "missing" results can be noticed.

[BPSDB]