Peer review · Lesson 2 of 3

How to review a paper

Separate major from minor, number your points, name the location, and say what would fix it.

9 min read · Free — no account needed to read

A useful review answers a specific set of questions and gives the authors something to act on.

Is the question clear and worth asking? State what you understood it to be — if you got it wrong, that is itself a finding about the writing.

Does the method suit the question? Not whether it is the method you would have chosen. A cross-sectional survey cannot support a causal claim; that is a real problem. Preferring interviews to a survey is a preference.

Is the execution sound? Sampling, measures, analysis, and whether the reporting is complete enough to be checked.

Do the conclusions follow from the results? Compare the results section with the abstract and the discussion. This is where most real problems are.

Is relevant work engaged with? Including work that disagrees.

Are limitations honest?

Then how to write it. Separate major issues — things that must change for the conclusions to stand — from minor ones. Number your points so the authors can respond to each. Be specific: "the conclusion on p.12 is causal but the design is cross-sectional" is actionable; "the methodology is weak" is not. Say what would fix it where you can. And distinguish what the work *is* from what you would have done.

The tone rule that costs nothing: write about the work, never the author, and remember that somebody spent a long time on this. A harsh review and a rigorous one are not the same thing, and the rigorous one is more useful.

See it happen

You are reviewing a paper whose conclusions go beyond its cross-sectional design.

The read that arrives on its own

Write: "The methodology is weak and the conclusions are overstated. Reject."

The same situation, with this idea

You write something the authors can act on: "Major, point 3 — the abstract and the discussion on p.14 state that X causes Y, but the design is cross-sectional and cannot establish direction. This is addressable in the text: restating the finding as an association, and adding the alternative explanation that Y may drive X, would make the conclusions match the evidence." Same judgement, and now it improves the paper rather than only scoring it.

After this, you can

write a review comment that names the location, the problem and a fix.

The takeaway

Separate major from minor, number your points, name the location, and say what would fix it.

Easy to take the wrong way

That being demanding means being severe. The most demanding reviews are usually the most specific and the most usable.

Try this

Take a published paper and write three numbered review comments on it, each naming a location, a problem and a possible fix.