Calculus I · When You Can’t Solve It, Chase It

Newton’s method: replace the curve with its tangent, and repeat

We want a root of $f$ — a place $x_\star$ where $f(x_\star) = 0$. Suppose we have a guess $x_n$ that is in the neighborhood but not exact. From Chapter 6 we know that near $x_n$, the function is almost equal to its tangent line:

$$f(x) \approx f(x_n) + f'(x_n)\,(x - x_n).$$

The curve is hard. The tangent line is easy — it is just a straight line, and a straight line crosses zero in exactly one obvious place. So here is the whole idea: instead of asking where the hard curve hits zero, ask where its easy tangent line hits zero, and use that as our next guess. Set the right-hand side to $0$ and call the solution $x_{n+1}$:

$$0 = f(x_n) + f'(x_n)\,(x_{n+1} - x_n).$$

Solve for $x_{n+1}$ — just rearranging:

$$x_{n+1} - x_n = -\frac{f(x_n)}{f'(x_n)} \qquad\Longrightarrow\qquad \boxed{\;x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}\;}$$

That is Newton’s method. You did not memorize it; you built it. Geometrically: stand on the curve above your guess, slide down the tangent line until it hits the $x$-axis, and step over to that spot. The new spot is almost always closer to the true root. Then do it again. (Figure 9.1: the curve, the tangent at $x_0$ sliding to the axis at $x_1$, the tangent at $x_1$ sliding to $x_2$, the guesses marching in toward the root.)

Watching it work: computing $\sqrt{2}$ by hand

In Book 1, the very first irrational number we met was $\sqrt{2}$ — a number with no exact fraction and no terminating decimal. We could prove it existed but we could not write it down. Now we can chase it.

Finding $\sqrt{2}$ means finding the root of $f(x) = x^2 - 2$ (since $x^2 - 2 = 0$ means $x = \sqrt 2$). Its derivative is $f'(x) = 2x$. Drop these into Newton’s formula:

$$x_{n+1} = x_n - \frac{x_n^2 - 2}{2x_n} = \frac{x_n}{2} + \frac{1}{x_n} = \frac{1}{2}\!\left(x_n + \frac{2}{x_n}\right).$$

Start with a rough guess, $x_0 = 1.5$, and turn the crank:

The true value is $\sqrt 2 = 1.414\,213\,562\,37\ldots$. Look at the last column: the number of correct digits roughly doubles every step — two, then five, then eleven. After three steps from a sloppy start we have eleven correct digits. This blistering speed is called quadratic convergence, and it is Newton’s signature. (That formula $x_{n+1} = \frac12(x_n + 2/x_n)$, by the way, is the square-root method the Babylonians used four thousand years ago. They had no calculus; they found the rule by cleverness. You just rediscovered it from a tangent line.)

Toolkit: Newton’s method (finding a root)

To solve $f(x) = 0$ when no formula will: 1. Pick a starting guess $x_0$ somewhere near the root. 2. Repeat: $\quad x_{n+1} = x_n - \dfrac{f(x_n)}{f'(x_n)}$. 3. Stop when the guesses stop changing (to as many digits as you need).

What it’s really doing: approximating the curve by its tangent line and jumping to where the line hits zero. Fast — digits roughly double each step — when it works. The next section is about when it doesn’t.

All 12 chapters of Calculus I · Foundations · All courses