Reproducibility and open science · Lesson 2 of 3

Pre-registration and registered reports

Pre-registration separates predicted from discovered. Both are publishable; only one carries the weight the statistics assume.

8 min read · Free — no account needed to read

Pre-registration means recording your hypotheses, design, sample size and analysis plan in a time-stamped public registry before you collect or look at data.

What it fixes is precise: it separates what you predicted from what you found. Confirmatory analyses — pre-specified — carry the weight the statistics assume. Exploratory analyses — decided after seeing the data — are valuable and carry different weight. Both belong in a paper; the distinction is what pre-registration preserves and what p-hacking and HARKing destroy.

What it does not do: it does not prevent exploration, it does not lock you into a broken plan, and it does not make a study good. Deviating is entirely allowed — you report the deviation and why.

Registered reports go further: the introduction and method are peer-reviewed *before* data collection, and acceptance is based on the question and the design rather than the result. This removes publication bias at source, because the paper is accepted whether the finding is positive or null.

For qualitative and exploratory work, pre-registration fits awkwardly and there are adapted forms — registering the question, the sampling strategy and the analytic approach, while accepting that emergent design is the method. The principle transfers even where the template does not.

The honest incentive: it protects you. When a result is surprising, "this was pre-registered" is the difference between a finding people believe and one they assume you fished for.

See it happen

Your pre-registered analysis shows nothing, but a subgroup shows a strong effect.

The read that arrives on its own

Report the subgroup finding — it is the interesting result and it is in the data.

The same situation, with this idea

You report both, labelled. The pre-registered analysis found nothing; the subgroup effect is exploratory, and the number of possible subgroups means one of them showing something is expected by chance. Written that way it is an honest lead for a future study. Written as the headline it is a false finding with a real number attached — and the pre-registration is what made the distinction visible.

After this, you can

label exploratory findings as exploratory rather than promoting them.

The takeaway

Pre-registration separates predicted from discovered. Both are publishable; only one carries the weight the statistics assume.

Easy to take the wrong way

That it forbids changing your mind. Deviation is normal — reporting it is the requirement.

Try this

Write your analysis plan for a current project before you look at the data, and date it somewhere you cannot silently edit.