The big idea: A model learns whatever is in the data, including its faults.
Data cleaning is finding and fixing what is missing, duplicated, inconsistent or plainly wrong — and it routinely takes more time than building the model.
What goes wrong in real data
- Missing values — a blank field, or a 0 that means "not recorded"
- Duplicates — the same record entered twice, which quietly doubles its influence
- Inconsistent formats — "UK", "U.K." and "United Kingdom" as three different countries
- Outliers — a height of 3 metres, or a genuine extreme that must be kept
- Wrong types — a number stored as text, so it sorts as "10" before "9"
Free preview
This is the free notes preview
You're reading the free notes. Aimnova Pro unlocks the full study experience — and you can try it with your first topic free to keep:
- FlashcardsLock in vocabulary and key terms with spaced repetition.
- Practice questionsAnswer exam-style questions and get instant AI marking.
- Mock exams & past-paper vaultSit full mocks and see exactly how examiners award marks.
- Personalised study planA daily plan built around your exam date and weak areas.
Start Studying Free Full access to Aimnova Pro · cancel anytime
| Approach | Does | Risk |
|---|---|---|
| Delete the row | Removes the record entirely | Loses everything else it held; biases the data if absence is not random |
| Delete the column | Drops the attribute | Fine if mostly empty; otherwise throws away a signal |
| Impute the mean | Fills with the average | Shrinks the variance and invents data that was never observed |
| Flag it | Adds a "was missing" column | Keeps the fact of absence, which is often informative |
Absence is rarely random: If income is missing more often for low earners, deleting those rows makes the dataset richer than reality — and the model learns a world that does not exist.
Ask why a value is missing before deciding what to do with it.
A zero is not a blank: A recorded blood pressure of 0 is not a patient with no blood pressure — it is a missing reading encoded as a number.
A model treating it as real learns from an impossible value, and no error is ever raised.
See how examiners mark answers
Access past paper questions with model answers. Learn exactly what earns marks and what doesn't.
Try Exam Vault FreeYour first topic is free to keep • No credit card required
Outliers
- An error — a typo, a sensor fault → remove or correct
- A genuine extreme → keep; it may be the thing you care about
- In fraud detection the outliers are the target
- Deciding which needs domain knowledge, not a rule
Duplicates
- Double the weight of one record on what the model learns
- Often arrive from merging two sources
- Exact duplicates are easy; near-duplicates are the problem
- "J. Smith" and "John Smith" at one address
Consistency
- One format per field, decided and applied
- Dates, units, capitalisation, country names
- kg against lb in one column is a silent disaster
- The Mars Climate Orbiter was lost to exactly this
Garbage in, garbage out: The phrase is examinable, but say what it means: a model has no way to know a value is wrong. It fits the pattern it is given, including the faults, and reports high confidence either way.
How this is tested — you must name a specific data fault and say what it would do to the model. It comes up two ways:
Paper 1 Section A
- Describe the significance of data cleaning, 3-4 marks
- Identify problems in a described dataset
- State one way to handle missing values
Paper 1 Section B — case study
- Explain the consequence of a fault for a deployed model
- Justify a choice between deleting and imputing
The classic trap: Answering "the data would be inaccurate". Name the specific fault and trace it to a specific effect on the model — zeros dragging an average down, duplicates doubling a record's weight. Vague harm earns vague credit.
IB-style questionDescribe[5 marks]
A hospital dataset records blood pressure, with missing readings stored as 0 and some patients entered twice. Describe the significance of cleaning this data before training a model.
Model answer plan
See the mark-by-mark plan — for / against / judgement, with marking guidance — in study mode.