The big idea: Model selection is choosing which model to deploy. The most accurate is frequently the wrong answer.
More than accuracy: Explainability, speed, cost, data required and the consequence of each kind of error all count. In some domains they outrank accuracy outright.
What a comparison must weigh
- Performance — and the right measure, not just accuracy
- Explainability — can a decision be justified to the person it was made about?
- Speed at prediction time, which is often what decides deployability
- Data needed to train it at all
- Cost — hardware, energy, and who has to maintain it
- Which error is worse, and whether the model can be tuned towards that
Free preview
This is the free notes preview
You're reading the free notes. Aimnova Pro unlocks the full study experience — and you can try it with your first topic free to keep:
- FlashcardsLock in vocabulary and key terms with spaced repetition.
- Practice questionsAnswer exam-style questions and get instant AI marking.
- Mock exams & past-paper vaultSit full mocks and see exactly how examiners award marks.
- Personalised study planA daily plan built around your exam date and weak areas.
Same data, same split
Every candidate sees the identical training and test sets. Comparing a model tested on easy data with one tested on hard data measures nothing.
Cross-validation
Compare over k folds, not one split, so a lucky division cannot decide the winner.
A baseline first
Always measure the simplest thing — a majority-class guesser, or linear regression. If the complicated model barely beats it, the complexity is not earning its place.
Test set once
Choose on validation. Open the test set once, at the end, to report. Choosing on it makes the reported figure optimistic.
A small accuracy gap is often noise: 94.1% against 93.8% on one test split is not evidence that the first model is better. Re-split the data and the order may reverse.
Where the gap is within the variation between folds, choose on the other criteria — they are the ones that actually differ.
Study smarter, not longer
Most students waste 40% of study time on topics they already know. Our AI tracks your progress and optimizes every minute.
| Situation | Choose | Because |
|---|---|---|
| A decision must be justified | Interpretable model | "Component 3 was high" is not an explanation |
| Prediction must be instant on a phone | Small model | A large one cannot run there at all |
| Only a few hundred records | Simple model | A complex one will fit noise |
| A regulator may audit it | Interpretable model | You must be able to show the reasoning |
| Accuracy is all that matters | The most accurate | Nothing else is constrained |
How the data changes which model does best
| Data characteristic | Tends to suit |
|---|---|
| Few records | Simple models (linear regression, small decision tree): complex ones overfit |
| Millions of images or sounds | Neural networks such as CNNs |
| Straight-line relationship | Linear regression |
| Curved or rule-like relationships | Decision trees, neural networks |
| No labels at all | Unsupervised methods such as clustering |
| Many features | Feature selection first, or models that cope with many inputs |
The strongest answers name the trade: "Choose the decision tree — it is 2% less accurate, but the bank must explain refusals to applicants, and a tree's path is the explanation."
That sentence scores far higher than "choose the most accurate model", because it prices what is being given up.
How this is tested — you must compare candidates on more than accuracy and justify the choice from the scenario's constraints. It comes up two ways:
Paper 1 Section A
- Explain the importance of model selection, 3-4 marks
- State one criterion besides accuracy
- Identify why a baseline matters
Paper 1 Section B — case study
- Choose between models for a scenario and justify it
- Explain why the most accurate is not always chosen
The classic trap: Choosing the highest number. The question almost always contains a constraint — a regulator, a phone, a clinician, a small dataset — and that constraint is what the marks are for.
A bank compares a decision tree (91% accurate) with a neural network (94%) for loan decisions. Evaluate which it should deploy.
Model answer plan
See the answer broken into clear points, with guidance on how each point earns marks.