Module 09 · 3 lessons

Statistics for researchers

Enough statistical reasoning to read honestly, plan sensibly, and not be fooled by a number.

Free — no account needed to read

Enough statistical reasoning to read honestly, plan sensibly, and not be fooled by a number.

Why it matters

Most misuse of statistics is not advanced — it is a handful of confusions about significance, effect size and sampling that are entirely learnable without becoming a statistician.

What it teaches

• Population, sample, parameter, statistic
• Distributions, and why the average can mislead
• Sampling methods and where bias enters
• What a p-value is and what it is not
• Effect size, confidence intervals and precision
• What statistics cannot tell you

When you reach for it

• A paper reports a significant result and you want to know if it matters
• You are planning data collection and need to think about sample
• Somebody quotes an average at you

The common misreading

That statistics is about computation. The computation is done by software; the reasoning about what the numbers can support is the part that is yours.

What this module will not tell you

This module will not teach you to run analyses. That belongs in R, Python, JASP, jamovi, SPSS or Stata, which are mature and mostly free — this teaches you what you are asking them for and how to read what they return.

Think first

Average income in a small town is reported as far higher than the region. One resident is a retired billionaire. What should have been reported?

Write down what you think it means when a study says a result is "statistically significant". Keep what you wrote — the second lesson will be more useful if you can compare.

The lessons

• Population, sample, and the shape of data — A mean alone hides the shape. Ask for the median, the spread, and ideally the picture. (8 min)
• Sampling, and where bias gets in — Selection decides what a result can support. Size only decides how precise the wrong or right answer is. (8 min)
• Significance, effect size, and what statistics cannot say — Significant means "probably not nothing". Effect size tells you whether to care. They are different questions. (10 min)

Practise it: Read the numbers in one real paper

Take a quantitative paper relevant to you and work through what its numbers can support.

1. Write the population it claims to be about, and the sample it actually had.
2. Note how the sample was selected, and the response or participation rate.
3. Find the main result: its effect size, and its confidence interval if given.
4. Ask whether an effect of that size would change anything in practice.
5. Write what the numbers cannot tell you, however well they were calculated.

What a considered answer tends to contain (not a mark scheme): A considered answer separates "probably real" from "large enough to matter", and names at least one thing the analysis cannot settle.

Use it on real work: Plan the numbers side of your own work

For a project of your own, think through the quantitative side before you collect anything.

1. Name the population you want to speak about.
2. Say how you will select cases, and who that selection will systematically miss.
3. Say what size of difference would actually matter to you — before you see any data.
4. Write what you will report alongside any average.
5. Record all of it in the project brief so your later analysis is held to it.

Reflect

What had you been reading a p-value as saying, and what does it actually say?