Module 14 · 3 lessons

Causal inference

Free — no account needed to read

What would have happened otherwise — and how anybody could possibly know.

Why it matters

Almost every decision that matters is causal, most available data is not experimental, and the gap between those two facts is where the strongest and the worst research lives.

What it teaches

• Counterfactuals and the fundamental problem
• Randomisation, and what it actually buys
• Identification strategies when you cannot randomise
• Difference-in-differences, instruments, regression discontinuity
• Controlling for the wrong thing
• What no amount of data can settle

When you reach for it

• You need to know whether something worked, not whether it correlates
• You have observational data and a causal question
• Somebody adds control variables until the result changes

The common misreading

That causal inference means statistics. The statistics are the easy half; the argument for why your comparison is a fair one is the work.

What this module will not tell you

This module will not make observational data settle a causal question on its own. It shows which designs make a credible argument and what each one assumes — the assumptions remain assumptions.

Think first

Hospitals with more staff have higher death rates. Should the health service cut staffing?

Write down how you would find out whether a training course improved the people who took it. Then write down who you are comparing them with, and why that comparison is fair.

The lessons

• The counterfactual, and the problem at the centre — Every causal claim needs a substitute for what would otherwise have happened. Name it and defend it. (8 min)
• Randomisation, and what to do when you cannot — When you cannot randomise, look for something in the world that assigned the treatment for reasons unrelated to the outcome. (10 min)
• Controlling for the wrong thing — Control for common causes. Not for mediators, not for colliders, and never chosen after seeing the result. (8 min)

Practise it: Build an identification strategy

Take a causal question you genuinely have and construct the argument for a fair comparison.

1. State the causal claim and write the counterfactual in one sentence.
2. Name what will stand in for it — which group, which period, which model.
3. Write why that substitute is comparable in every relevant way except the cause.
4. Name the assumption this rests on, and how you could support it with evidence.
5. List your controls and label each as common cause, mediator or collider.
6. Write the alternative explanation your design does NOT rule out.

What a considered answer tends to contain (not a mark scheme): A considered answer states an assumption plainly and names something it cannot rule out. A strategy with no stated assumption has usually hidden one.

Use it on real work: Put the strategy in the project

Record the causal argument where your write-up will find it.

1. Write the identification strategy into your project brief.
2. Write the assumption it rests on beside it.
3. Find one piece of evidence for that assumption — a pre-treatment comparison, a balance check, a placebo test.
4. Record what you found, including if it was unfavourable.

Reflect

What had you been controlling for out of habit, and what role does it actually play?