Key Idea: Measurement gives a construct a precise, observable definition. Its quality is judged by consistency (reliability) and by what the scores represent (validity, or credibility in qualitative research).
Measuring
- Operationalise: say exactly what is recorded
- Numbers or words; asked or watched
- Direct or indirect; at the time or later
Reliability
- The same result when nothing has changed
- Over time, between raters, among items
- Helped by clear categories and a standard procedure
Validity
- Construct: the right idea
- Internal: a safe conclusion about cause
- External: holds beyond the study
Doubt
- Type I error: a false alarm
- Type II error: a miss
- Reliable is not the same as valid
The words to know
| Word | What it means |
|---|---|
| Construct | An idea, such as stress, that cannot be seen directly |
| Operationalisation | Defining a variable by the exact way it will be measured |
| Self-report | Asking people about themselves |
| Reliability | The consistency of a measure under the same conditions |
| Inter-rater reliability | Different observers rating the same thing closely agree |
| Validity | Whether a measure or study captures what it claims to |
| Construct validity | Whether the measure captures the actual idea |
| Internal validity | Whether an effect is really due to what the researcher changed |
| Ecological validity | Whether findings apply to real-life settings |
| Type I error | Finding an effect that is not real |
What exactly was recorded? Would it come out the same again? Is it the right thing, and how far does it reach?
Exam-style question
Your class practical was an interview. Your class interviewed twelve students about how they choose their subjects. The interviews were recorded and written out. Two of you sorted every answer into themes separately, and agreed on most of them. You then showed the list of themes to four of the students you had interviewed. Explain the concept of measurement in relation to the interview in your class practical.
Model answer plan
See the mark-by-mark plan — for / against / judgement, with marking guidance — in study mode.
Important: Reliable means consistent. Valid means measuring the right thing. A measure can be the first without being the second.
Tap each card to check yourself.
What is a construct? An idea that cannot be observed directly and is inferred from what can be.
Self-reported or empirical: a count of behaviour made by an observer? Empirical.
How is inter-rater reliability established? Two raters score the same behaviour independently, and their scores are compared.
A rival construct explains the score. Which validity is in doubt? Construct validity.
What is the qualitative counterpart of internal validity? Credibility.
What happens to type I and type II errors if the significance level is made stricter? Type I errors fall and type II errors rise.
Exam tips
- State the operational definition before judging it.
- For reliability, say who or what would have to agree.
- For validity, name the kind and what threatens it.
- Ask what the measure misses, and what else it could pick up.