Humanities are behind the sciences in the promise and danger of post-publication review

So there's a small scandal because the University of Chicago Press has dished on open-access in the latest Chicago Manual of Style. Let me state clearly that the editors of the Chicago Manual of Style have it dead wrong in taking a position about open-access within a style guide. This is as smart and productive as trying to stop a freight train by laying copies of the Publication Manual of the American Psychological Association on the tracks. The reason why is not only the philosophy of many humanities scholars in favor of open-access publication but institutional support (which the Manual's editors dislike), the federal government's support of open-access in science grants, the widespread use of arXiv, and the success of PLoS journals.

Kathleen Fitzpatrick's theorizing about the professional role of reviewing and the possibilities of post-publication review is fairly theoretical, but we've seen the ugly side of post-publication review with the evolving controversy over Science's publication of Felisa Wolfe-Simon's article on bacteria that may or may not use arsenic for central life processes. [Addendum: see David Kroll's observations, which I forgot to link to when writing this.] To be fair, a good part of the hostile reaction appears to have been motivated by NASA's press conference advertising the paper in advance as about "extraterrestrial" life. But apart from possible motives, what we can see is the seamier side of peer reviewing open for everyone to see, sort of like the reversing ("eversion" I think is the correct term) of a starfish stomach in its external digestion process. 

There are fairly serious blog responses to the paper, but even Rosie Redfield indulges in vitriol, spending a very long blog entry in substantive critique before describing the paper as "flim-flam" and the authors as unprofessional: "I don't know whether the authors are just bad scientists or whether they're unscrupulously pushing NASA's 'There's life in outer space!' agenda." So much for show, don't tell in science writing. The reality is such personal attacks exist within closed review environments, but there's something advocates of post-publication reviewing need to acknowledge in addition to all the other issues involved with reviewing–what if it's the experts in the field who cannot separate the analysis from the attacks? 

If you enjoyed this post, please consider subscribing to the RSS feed to have future articles delivered to your feed reader.
m4s0n501

2 responses to “Humanities are behind the sciences in the promise and danger of post-publication review”

  1. Bob Calder

    The choice to have a press release prior to publication is almost always a bad one. I’m not positive, but I think it would be up to the NASA public relations folks. You know, the ones that stated that nobody was allowed to utter “Big Bang” or say “Origin of the universe” before the public?

    At any rate, there is a disconnect at NASA that doesn’t exist at other research institutions because of the nature of their multiple roles.

  2. CCPhysicist

    Producing an embargoed press release is usually a good idea. It gives journalists extra time to do background research on news that might be difficult to process in a few minutes when writing a story for that day’s paper.

    What the Chicago folks assert makes no sense to me. The point of a style manual is to improve the clear communication of new results, and the point of open access is to actually communicate them. I’ll just observe that the adoption of TeX by the folks who created the cleverly (but unfortunately named) xxx.lanl.gov server in 1991 (what became arXiv) probably led to the adoption of a specific template within LaTeX for the Phys Rev journals — which replaced most of their old style manual.

    What I don’t understand is the big deal about post-publication review. Most of the controversy about peer review is that the real review of science takes place after publication. One or two reviewers (sometimes three) can hardly identify every possible flaw in a particular paper. You can’t re-take their data or run a one-month calculation just to check their claims. Not possible. The biggest role of a reviewer is to be sure the claims are clear enough to attack them later. That seems to be the case with that article. (The problem in this one instance is that publication in Science has come to carry an extra imprimatur of correctness.) People get justifiably upset when a reviewer holds up a paper that is correctly showing that the reviewer’s work is wrong.

    Every paper is, to some extent, a review of the work it cites. Some tear that work to shreds, either in a regular paper or something like the formal “Comment” mechanism that the Phys Rev journals have.