Who understands peer reviewing?

In the last few months, I’ve sent out a sprinkling of manuscripts for Education Policy Analysis Archives to folks outside the normal spheres of academe—not only those in think tanks who generally understand reviewing processes because they often submit to refereed journals, but also those in the surrounding “wonkosphere” (for want of a better word), whom I think would understand refereeing. In two cases, from individuals I generally respect, I’ve received e-mails that indicate they don’t get the concept. “I’m not sure why you want me to review this. Don’t publish it!” ran the spirit of one e-mail. “You wouldn’t be doing your journal a favor, because of the following errors:…” So I thanked my correspondent for doing exactly what I asked by providing me a reality-check. Much to her credit, she replied to indicate she got it. I think.
Here are the rules of refereed journals:

  1. Not everything submitted is published.
  2. Nothing published is perfect.
  3. The editor(s) dispose(s) of some portion of manuscripts before anyone else sees them, for a variety of reasons.
  4. Referees are sometimes cranky and sometimes misread pieces of a manuscript.
  5. Editors take referee reports very seriously and still ignore a certain portion of them.
  6. There is no law requiring a certain number of reports, or a certain quality of reporting, before an editor makes a decision.
  7. Journals differ in what precisely is blind to either referees or authors.
  8. Thick skins are required when reading reviews. Einstein apparently got huffy when faced with peer review for the first time in the 1930s. (Thanks to Ralph Luker for the pointer.) (See also the item above about variations in refereeing practices.)
  9. There is no natural law about refereeing, but it sharpens published works, and Plato would approve of it. (See Book 7 of The Republic—no, not the cave allegory, but the part later where the Socrates character is discussing the proper training of philosopher-kings.)
  10. There are suspicions in academe that journals that charge page costs also accept a higher proportion of manuscripts.
  11. There is a suspicion in this editor that fields with higher needs to demonstrate status sometimes have obsessive-compulsive disorders over refereeing processes. The main route for getting new ideas into circulation in physics is not refereed. Then again, physics is a Science Biggie that doesn’t have to prove itself to anybody.
  12. I edit a journal that is refereed, and I don’t anticipate ending the refereeing for a simple reason: I don’t know everything or even nearly everything I need to know to make publication decisions.

The test on this material is next Friday.