Module 04 · 3 lessons

Evidence and sources

Free — no account needed to read

Knowing what kind of source you are holding tells you what it can and cannot support.

Why it matters

Most people search the same way for everything and then treat every result as equally weighty, which means their conclusions are decided by whatever happened to rank well.

What it teaches

• Primary, secondary and tertiary sources
• Grey literature, preprints, datasets and reports
• Search techniques that change what you find
• Citation chaining, forwards and backwards
• Evaluating a source on what it did, not how it looks

When you reach for it

• Your searches return either three results or three hundred thousand
• You need to know whether a source is good enough to build on
• Everyone keeps citing the same study and you want to read it yourself

The common misreading

That source quality is a property of the publisher. A prestigious journal publishes weak studies and a government report can be excellent — quality is a property of what was done.

What this module will not tell you

This module will not give you a ranking of sources to apply mechanically. Any such list breaks the moment you meet a well-conducted piece of grey literature or a badly conducted trial.

Think first

Four sources all state the same statistic. Three of them cite the fourth. The fourth cites "industry estimates" with no link. How much evidence do you have?

Where did the last thing you confidently told somebody come from? Trace it back as far as you can — and notice where the trail goes cold.

The lessons

• Primary, secondary, tertiary — and grey — Follow every important number back to the place it was first produced. The trail often ends somewhere you would not have cited. (8 min)
• Searching so that what you find is not an accident — Your findings are downstream of your search terms. Design them, write them down, and use more than one database. (9 min)
• Evaluating a source on what it did — Evaluate on method, sample, comparison and what came after — not on the logo at the top. (8 min)

Practise it: Build a search and chase one citation

Design a real search for your own question, run it, and then follow one paper in both directions.

1. List your key concepts and at least two synonyms for each, including an older term if your field has one.
2. Write a Boolean string and record it.
3. Run it in at least two different databases and note the counts.
4. Pick the single most relevant result and read its reference list for anything you had missed.
5. Find what has cited it since, and note whether anyone has argued with it.
6. Write two lines: what your search covered, and what it certainly missed.

What a considered answer tends to contain (not a mark scheme): A considered answer names its terms and databases, and can say honestly what the search did not reach — language, date range, paywalled work, a field that uses different vocabulary.

Use it on real work: Run a real search into a real project library

Use the search and import tools in a research project of your own.

1. Open or create a project and go to its Library.
2. Run your Boolean search and save what is genuinely relevant.
3. For the three most important results, record where you found them and why you kept them.
4. Trace one claim you intend to rely on back to its primary source.
5. Write your search terms into the project brief so future you knows what was covered.

Reflect

What did your original search miss, and what does that tell you about how you usually look for things?