A First Taste of Algorithms · Learning as an Algorithm

The whole loop, honestly

You now hold every part of the modern machine, so here is what “training an AI” is, in one paragraph, with nothing hidden. A model is a formula with knobs — not one knob w but billions, the same mathematics with the derivative grown into a gradient (a slope per knob; the deep water below). The loss is a number measuring wrongness averaged over examples — for a language model, roughly: across mountains of text, how badly did you predict each next word? Training is the loop you ran by hand on the bowl: compute the loss’s gradient, step every knob a little downhill, repeat — not three steps but trillions, on hardware built for exactly this loop and nothing else. That is the entire ritual. The gradient of a billion-knob formula sounds unaffordable; the algorithm that makes it cheap is a beautiful machine built from this book’s own parts — the chain rule riding a graph — and Chapter 9 does not describe it. Chapter 9 builds it, and then builds the model, and then trains it, on your machine, until it speaks.

The real question

One question stands above all of it, and nearest neighbor — remember the island — asked it first. A learner is fitted to examples; agreeing with those examples is therefore no evidence at all. Nearest neighbor agrees with its own examples perfectly, always, by construction — ask it about a memorized point and it finds that very point at distance zero and copies the label. A parrot scores 100% on the questions it was trained on. The only honest exam is one the student hasn’t seen.

So the field’s entire experimental discipline is one move: split the examples. Fit on one part (the training set); measure on the part the learner never saw (the test set); report only the second number. The gap between the two numbers is the gap between memorizing and learning — and in Lab 8 you will measure that gap on nearest neighbor itself, watch a single poisoned example widen it, and hold in your hands the distinction this entire subject turns on. When the test score is good, something genuinely transferred from the examples to the world. Why transfer happens as well as it does for enormous models — models with vastly more knobs than examples, which by every classical right should be parrots — is, as of this book’s writing, an open research problem. People are working on it. The author of this book is one of them. The honest sentence is: we can build it, we can train it, we can measure that it generalizes, and we cannot yet fully say why. You are not reading a finished subject. You are reading one with the paint wet, and the last brushes are within reach of people who, like you, started by counting steps.

Deeper water: many knobs, and the shape of high places

With knobs w₁, …, w_d, the loss is a surface over d-dimensional space, and “slope” becomes the gradient: the vector of partial derivatives (∂L/∂w₁, …, ∂L/∂w_d) — each entry answering “if I turn only this knob, how does the loss tilt?” The gradient points in the steepest uphill direction (a lovely fact, provable with Book 2’s tools plus one inequality), so the update w ← w − η·∇L steps each knob against its own slope, all knobs at once: one line, any dimension. Everything in this chapter survives the generalization untouched — bowls (now called convex functions) still come with guarantees, valleys still trap.

But dimension does something strange to the zoology of flat places. In one dimension a flat point is a floor or a hilltop. In a million dimensions it is almost always neither: a saddle — downhill some directions, uphill others — and a walker pausing there can usually find an exit, where a true 1-D hilltop offers none. Some of the empirical mercy of deep learning (greedy works far better than the two-valleys picture predicts) is suspected to live here, in the geometry of very high places; the suspicion is not a theorem, the geometry of real loss landscapes is an active research frontier, and the tools being thrown at it include some you would recognize — eigenvalues, phase transitions, the spectra of large matrices. The mountains are real; the cartography has barely begun.

Figure 8.4 — Three temperaments of the stride: the crawl, the brisk arrival, and the explosion. One dial, every fate.

Deeper water: the stride, exactly

On the bowl, the Diamond’s law |x_{k+1}| = |1 − 2η|·|x_k| classifies every stride with one number, |1 − 2η|, and it is worth tabulating the taxonomy because every behavior in it occurs daily in real training: η tiny → factor near 1, convergence at a crawl (real name: “undertrained”); η = 0.5 → factor 0, the floor in one step (possible only because this bowl is perfectly known — never in real life); η ∈ (0.5, 1) → negative factor, oscillating convergence, the loss curve ringing downward (you will see this ringing in Chapter 9’s training runs and now you know its name); η ≥ 1 → factor at or beyond 1, ringing forever or exploding (real name, seen in every practitioner’s logs: “loss went to NaN”). One honest limitation, which the Diamond’s part (e) makes you locate precisely: the exactness of this law leans on the bowl’s slope being exactly linear. General smooth bowls keep the qualitative story — small strides converge, oversized strides explode — but the clean factor and the magic η = 0.5 are this bowl’s private property. The idea generalizes; the arithmetic does not. Knowing which part travels is most of mathematical maturity.

Where this leaves us

The two questions held. “Correct” softened to approximately correct on average over unseen examples — measured by the train/test discipline, trusted by Chapter 7’s concentration. “Cost” never softened: nearest neighbor pays at answer time, descent pays per step times steps, and we counted. The new machinery is small and yours: examples instead of definitions; loss — wrongness as one number; descent — greedy on a landscape, with greedy’s old guarantee on bowls and greedy’s old disease among valleys, now incurable by the notebook and managed by dice. And above it all, the field’s one honest exam: never grade a learner on questions it has memorized.

What remains is to stop gesturing at “a formula with billions of knobs” and build one — the knobs, the chain rule that prices them all at once, the loop, the model, the training run, the first words it speaks. Everything needed is already in this book, scattered through eight chapters like parts on a workbench. Chapter 9 is assembly.

All 32 chapters of A First Taste of Algorithms · Foundations · All courses