For the past year, I’ve been an associate editor for two journals, and for one of them, the run of manuscripts has been about 50% greater than one of the co-editors said was likely. So I’ve had to up my game in terms of the main job of associate editors for both journals, organizing the peer review process once a manuscript has been approved by the editorial leadership for peer review (i.e., not a so-called desk rejection, on first read). This blog entry describes my individual process of finding peer reviewers, in an era where the first impression of many journal editors is that manuscript submissions have increased at the same time as review invitations have been declined, especially for so-called ad hoc reviewers, outside the editorial board of a journal.
In some ways, this task is parallel to the obligation I had as an administrator when asked to solicit external reviewers for colleagues.1 The obligation on me is to create a list, where the incentive is to create a long list of potential reviewers, while I am not generally an expert in the manuscript’s focus.
In general, with each manuscript my goal is a list of 15-20 potential reviewers, including a few members of the journal’s editorial board (with the target of at least one board member reviewing the manuscript), potential reviewers who have substantive knowledge of the research area, reviewers who have enough methodological expertise to cover issues that I or the editorial leadership might not spot, and for everyone, reviewers without structured conflicts of interest (reviewers from the same institution, or collaborators of one of the authors, as well as individuals the authors recommend not be in the reviewer pool–assuming the authors don’t try a peremptory striking of dozens).
To that end, what follows are elements of my regular workflow. I don’t always need each step, but I know I have several paths to a good reviewer list.2
- Download the manuscript and any supplementary files into my “active MS” folder for the journal.
- Check the submission letter for recommendations about reviewers.
- Start annotating the manuscript with the names and institutional affiliation of authors. (I typically scribble all over the PDFs once I transfer a copy to my iPad. Maybe I’ll invest in a ReMarkable tablet for this.)
- Read the manuscript to identify content, methods, and major internal citations, as well as potential issues. This is not a detailed reading in the way I would as a reviewer; I want to identify issues that are obvious to me on first impression that I want to check later against reviewer comments.
- For the subject of a manuscript, brainstorm adjacent areas where relevant expertise might lie — especially, what fields outside education have some connection to this research. (A useful question here for myself: in other fields, who might read and potentially cite this piece if it’s published?)
- Search for relevant content and methods experts, both among those I know, and using other tools such as a journal’s reviewer database.
- Engage in a second-order citation search, using major works of relevant content and methods experts — for a piece about cultural capital, for example, I might look for who cites Annette Lareau’s Unequal Childhoods, and within that set, whose work is most closely related to the content or theoretical underpinnings of the manuscript. For this type of search, Google Scholar’s “cited by” list is invaluable, as it allows me to search within the “cited by” collection.
- Engage in a second-order citation search, starting with the major studies cited by the manuscript that are old enough to have a large cited-by set.
These steps generally allow me to create a sufficiently large list of potential reviewers, which include members of the journal’s editorial board, as well a range of ad hoc reviewers with sufficient knowledge in the topic and/or in the methods.
One final note: The standard invitation email from most journals invites recipients to identify other potential reviewers. If you are asked to review for a journal where you are not on the editorial board, and do not have time, do not ever second-guess your decision to decline to review. But you can still help with the review process by recommending peers. I assure you, I am unlikely to know the area anywhere as well as you, and every one of your recommendations is gold to me!