Causal inference · Lesson 3 of 3

Controlling for the wrong thing

Control for common causes. Not for mediators, not for colliders, and never chosen after seeing the result.

8 min read · Free — no account needed to read

"We controlled for it" sounds like rigour, and adding variables can make an estimate worse rather than better. Three cases are worth recognising.

A mediator is on the causal path. If education raises income *through* occupation, controlling for occupation removes part of the effect you were trying to measure. The estimate drops, and it looks like the effect was weaker than thought.

A collider is caused by both your treatment and your outcome. Controlling for one creates an association that does not exist in the world. This is why studies of hospitalised patients find strange relationships between conditions: being hospitalised is caused by both, and conditioning on it links them.

A proxy for the outcome deflates the effect for a trivial reason — controlling for last month's sales when the outcome is this month's.

The habit worth building: before adding a control, ask what role it plays. Is it a common cause of both the treatment and the outcome? Control for it. Is it on the path between them? Do not — or report both, and say which question each answers. Is it caused by both? Definitely not.

And the honest tell: if the result changes each time a variable is added and the write-up reports the version that worked, that is module 10's problem wearing a technical costume.

See it happen

A study finds remote work has no effect on productivity once it controls for hours worked and meeting time.

The read that arrives on its own

A careful analysis that adjusts for the obvious differences — the effect disappears, so there is none.

The same situation, with this idea

You ask what role the controls play. If remote work changes productivity partly *by* changing hours and meeting load, those are mediators on the causal path, and controlling for them removes the very mechanism. The finding is now "remote work has no effect on productivity, holding constant the things through which it has its effect", which is a different and much narrower claim than the one in the abstract.

After this, you can

ask whether a control is a common cause or a step on the path.

The takeaway

Control for common causes. Not for mediators, not for colliders, and never chosen after seeing the result.

Easy to take the wrong way

That more controls mean a more careful analysis. Each one is a claim about structure, and the wrong one introduces bias rather than removing it.

Try this

List your control variables and label each: common cause, mediator, collider, or unsure. Investigate the unsure ones.