How to write an effective peer review report (for humanities & social sciences)

· by · in

Peer reviewing is a common and essential part of everyday scholarly labor. But like many day-to-day academic activities, graduate programs rarely teach students how to be an effective peer reviewer. It’s mostly something you learn by inference as you read reviews of your own work.

I have seen a lot of peer review reports—I edited an academic journal for three years, and currently am an acquisitions editor at a scholarly press—and I can attest that at least among humanities scholars it’s not common knowledge what a good peer review report should contain. Some reports are short and focus only on a few things; some reports track every copy editing error; some reports—the most thoughtful reports—read like a whole conference paper discussing the strengths and weaknesses of the manuscript. Some people under-do it, some people overdo it. 

I thought I would make everyone’s job easier—editors, reviewers, and authors—by explaining what a peer report needs to do and what the most effective reports do and do not contain.

Who is the audience?

The most important source of confusion when it comes to writing peer reviews has to do with the report’s audience. Editors and editorial boards are the primary audience for peer review reports; authors are only the secondary audience. Editors and editorial boards take reports in advisement when deciding whether to accept a manuscript and in advising which revisions they will require.

Having read hundreds of peer review reports, it appears to be the case that most reviewers mistakenly treat authors as the primary audience. And in so doing they make another mistake: they approach peer reviewing like a form of grading, where the point is to account for what is correct and incorrect. But because the primary audience for peer reviews is editors and editorial boards, peer review reports need to be more like tenure & promotion letters or recommendation letters because they are explaining and situating work for non-experts. As such, their primary objective is to first situate the work in the wider field and point out the contribution the work makes to the existing scholarly literature, and then comment on the relative quality of the scholarship.

So, while editors definitely do want suggestions for revisions, those should come after a discussion of the project’s main argument and how that is situated with respect to current scholarly literature.

What should a peer review report cover/contain?

Editors have some knowledge of the manuscript’s field, but they generally are not experts in the literature specific to a project’s subfield(s). For this reason, the first thing a peer review report needs to do is to identify what the manuscript’s main argument is, what its methods are, and how these are situated with respect to current scholarly literature. The report should also clearly identify and explain the project’s primary and secondary scholarly contributions to that literature. What is this project doing, how is it related to what’s going on in the field right now, and what are the key contributions this project makes (or, with some revisions, could make)?

Another important thing peer reviewers need to do is assess the quality of the scholarship: is the research and writing adequately rigorous and ethical? In assessing this, it is crucial that you evaluate the project in its own terms and don’t try to make it the project you would have done. 

That said, authors also need to be sure to manage reviewer expectations, especially if a project uses any nontraditional or experimental methods. 

Also, if an editor has asked you to review a project, it’s likely they want the perspective of someone in your field; however, if it’s an interdisciplinary project please be sensitive to this. In such cases, it’s often helpful to flag things that audiences from your discipline might not understand or might misinterpret. Again, the point isn’t to turn an interdisciplinary project into a disciplinary one, but to help authors adequately contextualize their project for a disciplinarily heterogenous audience.

In cases where it is clear the manuscript is a revised dissertation and still needs some work to get it from student level to professional level, please approach your task as a form of mentorship. Peer reviews are one of the primary ways this mentorship happens. Be charitable and help guide the author to and through the revisions they need to make to help their project realize its full potential.

Here are some things reviewers should comment on:

  • Main argument: re-state and explain in non-expert terms.
  • Scholarly contribution: explain if and how this manuscript makes a contribution to scholarship in XYZ field(s).
  • Situation wrt the literature in XYZ field(s): With whom is this project in conversation? Does it leave anyone important out? Is it only in conversation with cis/white/men?
  • Structure and organization: is this clear and effective?
  • Scope: What does the project cover? Does it do what it claims/aims to do? Does the focus need to be narrowed or broadened?
  • Methods: What methods does it use, and does it use them successfully? 
  • Relevance: To whom is this relevant, and how?
  • Quality of scholarship: research, citations, etc.
  • Quality of analysis: What is the author doing with their archival material/data/etc? Is the analysis coherent? Original? Adequately rooted in the data?
  • Quality of argumentation
  • Potential objections the author hasn’t addressed
  • Audiences who would be interested in the project
  • If single-anonymous, any ethical violations the author is known to have committed (e.g., Title IX violations such as sexually harassing and/or abusing students). Journals and presses have reputations to protect and we don’t want to publish work by known predators or trans-exclusionary ‘feminists,’ etc.
  • Whether you would accept, request revisions & resubmit, or reject the submission.

Here are some things reviewers should not comment on:

  • Copy editing matters. We pay someone else to do that.
  • Voice or tone: only point out uncharitable and/or sexist/racist/etc. cases. If a project has a more casual tone, editors already recognize this and are fine with a less stodgily academic project.
  • If single-anonymous (as most book proposal/manuscript peer reviews are), the author’s employment status. 

Other helpful things

If a press or journal uses a template, please use the template. If they don’t have a template, it’s helpful to use descriptive headings and break the review down into sections–that helps editors and boards navigate what would otherwise be a long block of text more quickly.

Remember to not put your name in the document or file name.

I know everyone is overwhelmed and being forced to do too much with too few resources, but please try to complete reviews in a timely manner. Your colleagues’ careers directly depend upon it.