Judging quality in each family · Lesson 3 of 3

How many is enough, in each family

Ask "enough to detect what?" or "enough until what stops appearing?" — never "enough" on its own.

8 min read · Free — no account needed to read

In quantitative work the answer comes from what you are trying to detect. The relevant ideas are effect size — how big a difference matters — and power — the chance of detecting it if it is there. Small effects need large samples; large effects can be seen in small ones. A study that finds nothing may have been unable to find anything, and that is a different statement from "there is no effect".

The failure to watch for is the underpowered study: too small to detect the effect reliably, so it produces scattered results. Counter-intuitively its published findings tend to be *overstated*, because only the larger, noisier estimates cleared the threshold to get published.

In qualitative work, numbers work differently because the goal is not estimation. Saturation is the usual principle: you continue until new cases stop producing new categories. It is reached with a handful for a narrow, homogeneous question and needs far more for a broad or varied one.

What matters more than the count is who. Purposive sampling — choosing cases precisely because they can speak to the question — is the norm, and a well-chosen 12 will beat a convenient 60. The reporting standard is that the researcher says who they chose, why, and who they could not reach.

So both families ask "enough for what?" and the honest answer in either is tied to the question rather than to a convention.

See it happen

You are planning to interview people about why they left a service, and you do not know how many to plan for.

The read that arrives on its own

More is better and safer, so aim for as many as you can manage — perhaps 50.

The same situation, with this idea

You ask what would make you stop. If all leavers left for broadly similar reasons, new categories stop appearing quickly and 12 to 15 may cover it. If they leave for very different reasons — price, a bad experience, a competitor, a life change — you need enough within each type, which means planning by variety rather than by total. You plan 15, review what you have learned, and decide from the material rather than from a number chosen in advance.

After this, you can

plan a sample around what would make you stop rather than around a round number.

The takeaway

Ask "enough to detect what?" or "enough until what stops appearing?" — never "enough" on its own.

Easy to take the wrong way

That saturation is a licence for small samples. It is a claim that has to be demonstrated, and a study asserting it after six interviews on a broad topic has not met it.

Try this

Write the stopping rule for your own data collection before you start it, and say what would make you extend it.