If you’ve been paying attention, then you probably know academics do a lot of unpaid labour.
From hours of lesson prep (sometimes included in the contract, sometimes not), to months and months on research articles (that get locked behind paywalls which fill the journals’ coffers but never seem to find their way into the authors’ bank accounts), to pastoral care of students (if you’re lucky, the university might give you some training in this, so at least you’re not talking out of your ass when a burned-out student crawls into your office at 9:30 on a Tuesday, begging for help). Some universities accommodate for unpaid labour, while others avoid the question by telling the scholar to ‘work smarter, not harder’*.
Regardless of whether they are paid or not, these duties are considered the bedrock of the discipline – if you fail to do them, there could be ramifications for your career.
One of the duties academics occasionally have is peer-reviewing each other’s work. This can be in a blind peer-review for a research article, or in a published review of a colleague’s book. We’re very rarely given any training in how to review. It’s something that you’re supposed to pick up, via osmosis, after a few years of reading reviews from others. This is ok in theory… but if the others never learned either? Then it’s the blind leading the blind.
This is not even to mention the reviewers who – to put it delicately – woke up on the wrong side of the bed. I can’t tell you how many times I’ve received a review (as an author, and as an editor in charge of a publication) that may have had good feedback in it, but was so tonally inappropriate that any benefit was lost because the author was left reeling from the strength of the reviewer’s ire.
I’m not the only one who sees how cruel academics are allowed to be to each other. The anonymity of blind review is a safe space to be as vitriolic, spiteful, and cruel as people like. I’ve never met an academic who didn’t have some well-remembered quote in their back pocket, ready to pull out after a few drinks for the rest of us to commiserate over.
Sometimes, the tone of these reviews borders of hateful; other times, petty and childish.
There’s even a twitter account dedicated to the more memorable ‘insights’ that reviewers have given to their colleagues.
Here are just a few choice ones:
Truly, a wonderful help to the authors, and a clear and practical way to learn the craft of giving feedback to your peers.
These are things that actual human beings have put on paper, knowing that they would be read by someone else. I’d say that the internet trolls have invaded academia, but a brief discussion with any colleagues who worked pre-Web2.0 will tell you that this has always been the case. Empathy is the first thing to go when you don’t see the person you’re speaking to.
As discussed above, this labour is often unpaid. Maybe the reviewers are in rush, and don’t have time to spare making sure that a potentially sensitive author doesn’t get their feelings hurt?
Surprisingly, I don’t think it takes more time to be kind. In fact, I think it would take much less time than the time it obviously took some of these reviewers to come up with their witty put-downs.
The fact is, learning how to do anything by osmosis is difficult. Learning from bad actors is near-impossible. All it does it create a cycle of poorly-executed academic feedback masquerading as ‘no-nonsense’ or ‘cruel to be kind’ advice.
I’m not saying we’re supposed to be gentle to avoid hurting the delicate, baby feelings of the snowflake generation. I’m saying that many reviewers clearly don’t know how to give feedback in a way that actually helps. Or, perhaps, in a way that won’t immediately put the receiver’s hackles up and give them every reason in the world to believe that the feedback was given in bad faith, and therefore not worth listening to.
So… here’s a quick and dirty version for how to review an academic work properly. This can be useful for reviewers of scholarly articles, or for university students critiquing each other’s work in the context of a workshop/feedback session.
Step 1: Read the whole damn article.
Don’t be like that reviewer who refused to read the article because they didn’t think the author would be able to write in English. Reading the whole article will show you the progression of their argument, and help you make a better decision about whether their conclusions are justified. This is not the time to be correcting grammar or inserting your own comments. Just read and try to understand the work.
As you’re reading, feel free to take notes on questions you have and whether or not those questions are answered by the time you’ve finished reading. Make sure that you do this in a separate document to the one you received (and plan to send back with comments) so that you won’t have to spend extra time later going back over your feedback to make sure it wasn’t ‘too harsh’ – instead, you’ll be able to copy and paste the relevant comments into the document and leave any more personal or vitriolic statements to yourself.
Step 2: Check the bibliography – but not for your own name!
There’s nothing more cringe than a reviewer recommending their own article when giving feedback. Especially when that article is not open-access. I once had a reviewer order me to read an edition of a journal they guest-edited. The entire issue was behind a paywall and my institution didn’t have access, so their feedback was useless to me. Unfortunately, the state of our discipline means that there is not equal access to new research, and sometimes an author’s bibliography will reflect that. Don’t get classist in your feedback.
Instead, you should check the publication dates and journal names that the author did use. Is the bibliography out of date? There should be a majority of papers published in the last 5-10 years. Has the author relied too heavily on one source? Are the journals reputable, or did they use online sources that weren’t peer-reviewed?
This check usually only takes a few minutes and can give a good indication of the strength of the research that went into the article. If the author needs guidance here, then you can direct them to some open access journals, or journals that most institutions will have a subscription to. You’ll know the ones, if you’ve been paying attention to your discipline.
Step 3: Check the strength of their structure
The basic structure of an article is: introduction (with argument), literature review, methods, findings/discussion, and conclusion. If the author has done their job correctly, then there should be a logical progression to all of them. A lot of articles have good ideas but confuse the reviewer because of the structure of those ideas – help your author by explaining that their work needs to be more clear.
The introduction should lay out what the article is about and what the author plans to do. It should do this in the first few pages, if not the first few paragraphs. If it doesn’t, then that’s where your feedback should start. If the conclusion is doing its job, then it should have a very basic summary of the findings and results, followed by a call-back to the purpose of the article as stated in the introduction. A good way to quickly check whether the structure is working is to read the introduction, and then read the conclusion without reading anything else. Do the two logically follow one another? If they don’t, then that is a point that the author will need to improve, because they clearly lost their way somewhere in the middle. Go back to the notes you made when you did your first read-through – you might be able to pinpoint the moment that the author’s focus shifted, and then direct them to it. Otherwise, just tell them that the beginning of the article and the conclusions do not align, and they need to restructure to ensure clarity.
If the author made logical leaps in their writing that you couldn’t follow, go back to your notes to find where those leaps were made (ie: any points that you didn’t immediately understand or needed further elaboration) and make those notes in the author’s copy so that they can see where they lost you. Ask them to make the transition between ideas clearer so that it is easier to follow on the first read.
Step 4: Check the strength of their argument
Let’s be honest. You likely knew after the first read-through whether the argument was convincing or not. You either thought ‘Yeah, makes sense’, or ‘WTF?’ If it was the latter, then it is now your job to figure out why the article’s argument doesn’t make sense. It’s no use to anyone to simply tell the author that it doesn’t. After all, it made sense to them. So they’re not in a position to understand why it doesn’t make sense to others. They won’t be able to fix the problem if they don’t realise that there is one.
According to Wendy Laura Belcher’s brilliant Writing Your Journal Article in Twelve Weeks, an argument always has at least two parts: a claim and evidence for that claim. One of the easiest ways to tell whether a statement is an argument is if you can coherently respond ‘I agree’ or ‘I disagree’.
If the article you’re reviewing does not state outright what its thesis or argument is, then there is a problem. Either the author has failed to express this key idea but it was present in the way they expressed themselves or the structure of the article (possible, often likely), or the article doesn’t even have an argument (which can sometimes be the case, especially if it is a field study or a humanities article focused on ‘exploring’ a new topic). The feedback you give should be something along the lines of: ‘The argument needs to be explicitly stated early in the paper to ensure the reader’s understanding going forward’. If there is an argument that was just not stated plainly, then this is an easy fix for the author – if not, then the author will have a lot more work to do, but you don’t need to do it for them.
If, however, the author has stated their argument and you simply do not agree, ask yourself why. Is it that they have not convinced you, or is it that their argument goes against what you already believed (or perhaps argued elsewhere) about the topic? Make sure that you have taken your own ego out of the equation. Has the author given enough evidence to logically lead to their conclusion? If the answer is, objectively, ‘yes’, then your own feelings about the topic are irrelevant. If they have not given enough evidence to convince an objective third party, then you should limit your feedback to what they need more of (quantitative data, quotes from the text, etc). Limiting your feedback in this way will shorten the amount of time you spend on the review, while also giving the author clear directions about how to proceed.
Step 5: Check their grammar and spelling
Don’t spend too long on this – the final copy-editing work ought to be done by the journal. Run it through a grammar checker to pick up the most obvious problems and send the article back with those, along with a note that says something along the lines of: ‘There are some minor errors in expression. I have marked some on the paper, but I encourage the author to do a more thorough edit to find others I might have missed.’
Step 6: Write the review
You should send back two documents: one review, and one copy of the MS with in-line comments. The in-line comments should point out specific problems, either with sentence structure/grammar, or key statements that either need to be cut or elaborated on. Try to be as clinical as possible here to avoid letting your own emotions get the better of you. It might have been a frustrating process to read and review this work, but no one benefits from those feelings.
You don’t need to point out every problem in the MS. Just the most glaring ones. In the review document, make some notes about what the author needs to improve in general terms, and state that the MS includes some in-line comments but the author needs to take that feedback into account when revising the whole document.
Key things to include in your review doc:
- What is the main thesis/argument of the paper? What is the author’s purpose and what is the general content of the article?
- Where does it fit in the wider discipline? What is its scope and limitations (the author should have identified these for you. If not, then use your own knowledge and leave a note that the author ought to include this in the finished paper)
- The article’s strengths
- The significant areas for improvement
- Any key references or sources that you think the author might find useful
- Your recommendation to the publication (don’t try to be clever. Just state whether you think it should be accepted, accepted with minor changes, accepted with major changes, or rejected)
remember: Review the article you have, not the article you wanted
If there is an argument that you think might be more relevant, perhaps that’s something that your own research could cover? There’s no reason to make this article twist itself into an argument or a methodology that the author clearly did not find as compelling as the one they chose. Instead of creating an image of the perfect article in your head, and then critiquing the one you have as it fails to live up, try to focus instead on what the author has given you. This will make things less frustrating for the both of you.
And there you have it. One quick and dirty academic review. Feel free to adjust this process as you see fit – as the title suggests, this is a quick, relatively painless approach that I have found to work for me in the past. If you have more time (bravo!) then you should of course go deeper and really give the author the guidance they need.
Just please, oh please, I beg you: be kind. The author could be new to the field, or they could be a seasoned professor with dozens of publications to their name. Either way. Be kind.
PS – I love writing and I love eating! If you want to help with the latter (and ONLY if you want) you can maybe buy me a coffee? 🙂
* I’ve worked for both
Jessica Seymour 
One thought on “Writing an academic peer-review (quick and dirty version)”