Machine learning approaches

A **continuous** value, by fitting $y = mx + c$ through the data.

By minimising the **sum of squared residuals** — least squares. Squaring stops errors cancelling and punishes large ones heavily.

The change in the predicted value **per unit** of the input — for example price per square metre. It is the interpretable part of the model.

The line continues forever but reality may not. The model has no notion of the range it was trained on and reports **no uncertainty**.

"**How much?**" is regression — a number on a scale. "**Which one?**" is classification — a category.

**Explainability**, speed at prediction, data needed, cost, and **which error is worse** — any of which can outrank accuracy.

If the complicated model barely beats a majority-class guesser or a linear fit, the **complexity is not earning its place**.

Usually **no** — within the variation between cross-validation folds it is noise. Decide on the criteria that genuinely differ.

When a decision must be **justified or audited** — lending, medicine, recruitment. An unexplainable model is then not deployable at any accuracy.

On the **same** training and test data, ideally the same **cross-validation folds**, with the test set opened **once** at the end.

Which **category** something belongs to, learned from labelled examples — binary (two) or multi-class.

**Precision**: of those flagged, how many really were? **Recall**: of the real cases, how many were caught?

A model answering "no" every time scores 99.9% on a 1-in-1,000 condition and finds **nothing**. Ask what a majority guesser would score.

Catches **more real cases** at the cost of **more false alarms** — recall rises, precision falls. It is a human judgement about which error is worse.

Opposite. Spam: a **false positive** (a real email lost) is worse. Screening: a **false negative** (a missed tumour) is worse.

Ask whether it was **chosen before training started**. If the model worked it out from the data, it is a parameter.

How big a step each update takes. **Too high** overshoots the minimum and never settles; **too low** never arrives in the time available.

**Training** learns parameters, **validation** chooses hyperparameters, **test** is opened once to report. Tuning on the test set leaks it.

Splitting into k folds, training on k−1 and testing on the held-out one, rotating — so every record is tested exactly once.

**Grid** tries every combination and is exponentially expensive. **Random** samples within ranges and usually finds something good sooner, because few hyperparameters matter much.

Groups similar records with **no labels** — the groups are the output, which is what makes it unsupervised.

Choose k, place centres at random, **assign** each point to the nearest, **move** each centre to the mean of its members, repeat until stable.

The starting centres are **random** and it finds a **local** optimum, so where it settles depends on where it began.

k-means uses **distance**, so a feature measured in thousands dominates one measured in units — income would decide the clusters entirely.

What the clusters **mean**. It returns "cluster 0, 1, 2"; naming them is a human interpretation that nothing validates.

Items that **occur together**, written {A} → {B}. Unsupervised — nothing is being predicted.

**Support** = both ÷ all baskets. **Confidence** = both ÷ baskets with A. **Lift** = confidence ÷ how often B occurs anyway.

**>1** occur together more than chance (real) · **=1** independent · **<1** less than chance. It is what separates a finding from a popular item.

It rewards whatever is **popular**. 80% confidence is worthless if the consequent appears in 90% of baskets anyway — lift would be 0.89.

**No.** Support is identical, but **confidence differs** — everyone buying caviar buys bread; almost nobody buying bread buys caviar.

Supervised learning is told **the correct answer**. Reinforcement learning is told only **how well things went** — a reward, often long afterwards.

**Agent**, **environment**, **state**, **action**, **reward** — and the **policy**, which maps states to actions and is what is actually learned.

Whether to **exploit** the best action known so far or **explore** another in case it is better. Only exploiting never improves; only exploring never benefits.

Working out **which** of many earlier actions earned a reward that arrived much later — a game won after 200 moves.

The agent maximising **exactly what was measured** rather than what was meant. The system works perfectly; the objective was wrong.

**Population → fitness → selection → crossover → mutation → repeat.**

When the search space is **too large to enumerate**, no formula gives the answer, candidates can be **scored**, and "good enough" is acceptable.

**Mutation.** Crossover only recombines values already in the population; mutation can produce one present in neither parent.

It becomes a **random search** — good solutions are destroyed as fast as they are found.

**No** — a good one. There is no optimality guarantee, and two runs can give different answers.

Each input **times a weight**, summed, **plus a bias**, passed through an **activation function**.

**Input** (one unit per feature), **hidden** (where the work happens), **output** (one unit per class).

Without it, stacked layers collapse into a **single weighted sum** — a hundred layers would have the power of one.

Working **backwards** from the error to find each weight's contribution, so every weight can be nudged in the direction that reduces it.

It changes the **weights and biases**. The number of layers, units and the activation function are **hyperparameters**, fixed beforehand.

Slides a small **filter** of weights across the input, multiplying and summing at each position to produce a **feature map** of where that pattern occurs.

The **same** filter is used at every position, so a feature is recognised **wherever it appears** — and one detector costs a handful of weights rather than a set per location.

Keeping the strongest value in each block: **fewer numbers** to carry forward, and tolerance to small shifts in position.

**Edges**, then shapes built from edges, then objects built from shapes — a hierarchy that is learned rather than designed.

It **destroys the geometry** — the network no longer knows which pixels were adjacent — and a 200×200 colour image gives 120,000 inputs, needing millions of weights.