In April, I wrote about finding peer reviewers in a journal context. This blog post is about a task that seems similar but isn’t: finding external reviewers of scholarship for promotion and tenure purposes. In most American colleges and universities with any research expectations for tenure, part of the process includes asking tenured faculty to write an external review letter assessing the scholarship of the candidate for promotion and/or tenure. Ideally, letter-writers are scholars who can provide an independent judgment (i.e., with no conflicts of interest), and who have both sufficient expertise in the field and understanding of tenure expectations in the U.S. to be able to explain their assessment for review committees and administrators at the candidate’s college or university.
Those scholars are always nominated by someone — usually, candidates for tenure and promotion nominate at least half and sometimes all of the potential reviewers. And the normative (and sometimes written) rule is that the candidate for tenure or promotion cannot ask potential reviewers if they will write a letter, because it is the institution that invites reviews. This post is about how to create that list. I have conducted many workshops on tenure and promotion over the years and advised dozens of colleagues, and most of the informal advice bandied about omits critical information: what are the real qualifications for being a reviewer? As I wrote in 2018,
I have told dozens of colleagues over the years that at the tenure stage, the most critical qualification of an external review letter is the ability to put your scholarship in a broader context: what does it contribute to the field? It does not matter if you and they are on the same side of a disciplinary divide if they cannot write a competent review to save their lives. As an historian, I cheated when putting together my nominee list: I read a sample of book reviews from all my potential nominees. After deleting the peers who had written incompetent or capricious reviews, I was left with wonderful scholars whose judgments of my own work was unknown for the most part, but where I could say they had demonstrated competence and fairness. Most education faculty do not have that type of public writing sample for potential external reviewers, and they have to seek out potential external review nominees reviewing in the wild — as conference discussants, as lit review writers, as solo authors writing lit review sections, and the like. It’s an invisible part of the tenure-track life.
From https://shermandorn.com/wordpress/?p=8624
If you are a tenure-track faculty member where the book is the coin of the realm, and where most tenured faculty in your field regularly write book reviews, you can follow the same approach I did: vet your potential candidate list by the public and accessible evidence of their ability to put scholarship in a broader context. But you still need to assess that skill (or maybe habit) even if you do not have book reviews as evidence.
This is why I call this process a search for competent reviewers in the wild, because you need to hunt down the evidence of how someone assesses scholarly contributions. Sometimes, you can spot the evidence in written, public form: solo-authored discursive review articles, or solo-authored articles that are long enough to have substantive lit review sections. Sometimes, conference discussants have enough time, and the expectation, to reveal how they analyze the contributions of new scholarship. But sometimes, the best evidence is not public: it consists of the confidential reviewing that journal editors and grant program officers request and read. Sometimes that evidence leaks into public view, with reviewing awards: many journals issue them! But not always, and there are far more competent potential reviewers than there are recipients of reviewing awards.
Sadly, this is where one’s circle of networks and mentors matters most: if you know current or recent associate editors at journals in your field (and I mean a journal that overlaps the most with your scholarly community), they are probably the best source of information about who reviews fairly and clearly (and promptly). If your mentors know current or recent associate editors, that is another route to this information. If you ask, be very clear about what you are seeking: information about fair, clear, and prompt reviewers. Do not ask for “good reviewers for me” but “reviewers who are consistent about stating the contribution of a manuscript and are fair and clear.” You want to prompt people to think about reviewing competence and temperament, not disciplinary politics.
I say “sadly” because access to this information is a reflection of existing professional networks, and these connections are not distributed equitably. This is one additional reason to build a network of mentors.
If you are seeking tenure, and administrators (or occasionally peer review committees) nominate potential external reviewers, you can shape their search by clarifying your scholarly community. This is best done through narrative statements on your annual reviews — this is how your peers learn about you! — but you can also be explicit, especially with department chairs, school directors, and associate deans. Some years ago, one of my (now-tenured) colleagues spoke with me about their scholarly community, and also about a scholarly community that looks like it was the same, but was different in some very important ways. I immediately took that and started asking other candidates for tenure and promotion to provide me with this type of information, along with one other piece of information:
- A substantive explanation of the relevant scholarly community (or communities, more rarely)
- Examples of journals that members of the scholarly community read and publish in
- False cognates, to borrow a term from languages: Fields that might look similar to an academic from another discipline, but wasn’t, what the distinction was, and examples of journals that were thus inappropriate representatives of the candidate’s community of interest
- True cognates in other fields (this was my addition): If I were looking for reviewers who were faculty in other types of colleges, who might have similar concerns and be reasonable if unexpected sources of reviewers? Some of these connections are obvious: sociologists of education work in sociology departments as well as colleges of education, and people who study measurement often work in psychology departments. But some are truly unexpected: some of the interesting work in qualitative research methodology comes out of nursing faculty.
One final item: talk explicitly about potential conflicts of interest, even if you are not asked to when providing nominees. If you can say “I have collaborated with none of these scholars, none of them have my advisor, and none of them received their doctorates from the same place as me,” that’s great. If there is a potential conflict of interest, be clear, such as the following: “{Name} and I both graduated from Purdue, 12 years apart, and our committees did not share any common members.” And for goodness sake, do not nominate your own advisor, your advisor’s regular collaborators, your collaborators, your relatives, or (if a relative is in the same field as you) any committee member of a relative. And if your chair, director, or associate director has input on potential reviewers, give them a list of all your committee members, remind them from where and when you received your doctorate, and alert them to any well-known scholars where there is a clear conflict of interest because of collaboration ties (with you or your committee members).
