When Newton-Raphson Fails

A guess at a stationary point

Newton-Raphson replaces the curve by its tangent at the guess and moves to where the tangent meets the x-axis. That needs the tangent to meet the axis.

Take $f\left(x\right)=x^2-2$f(x) = x² − 2 again, whose roots are $\pm \sqrt{2}$±√2, and guess $x_0=0$x₀ = 0, the bottom of the curve. A point where the gradient is zero is a stationary point, and its tangent is horizontal. Here f(0) = −2, so the tangent is the line y = −2. It runs parallel to the x-axis and never reaches it, so there is no next guess.

The gold curve is $y=x^2-2$y = x² − 2. Its tangent at the bottom, $x_0=0$x₀ = 0, is the dashed horizontal line y = −2, which never meets the x-axis, although the curve crosses it twice.

The formula divides by zero

The formula says the same thing. $x_{n+1}=x_n-\frac{f\left(x_n\right)}{f^{\prime }\left(x_n\right)}$xₙ₊₁ = xₙ − f(xₙ)/f'(xₙ), and f'(x) = 2x, so f'(0) = 0. The step would be $\frac{f\left(0\right)}{f^{\prime }\left(0\right)}=-\frac{2}{0}$f(0)/f'(0) = −2/0, and nothing can be divided by zero, so $x_1$x₁ has no value.

This is not because the curve has no root, and not because the guess is a root: f(0) = −2, not 0. The trouble is only where the guess was made. Any start other than 0 works for this curve.

A nearly flat tangent

A tangent that is almost horizontal does meet the axis, but a long way off. The step $\frac{f}{f^{\prime }}$f/f' is the height divided by the gradient, and dividing by a small gradient gives a large step.

Take $f\left(x\right)=x^3-3x$f(x) = x³ − 3x, whose positive root is $\sqrt{3}\approx 1.732$√3 ≈ 1.732, and whose gradient $f^{\prime }\left(x\right)=3x^2-3$f'(x) = 3x² − 3 is zero at x = 1. Guess $x_0=1.1$x₀ = 1.1, just past that stationary point. Then f(1.1) = 1.331 − 3.3 = −1.969 and f'(1.1) = 3 × 1.21 − 3 = 0.63, so the step is $\frac{1.969}{0.63}=3.125$1.969/0.63 = 3.125 and $x_1=1.1+3.125=4.225$x₁ = 1.1 + 3.125 = 4.225. The guess was 0.63 from the root, and the next one is 2.49 from it.

The gold curve is $y=x^3-3x$y = x³ − 3x, which crosses the x-axis at $\sqrt{3}\approx 1.732$√3 ≈ 1.732. Its tangent at $x_0=1.1$x₀ = 1.1, dashed, has gradient only 0.63, so it reaches the axis at $x_1=4.225$x₁ = 4.225, much further from the root than the guess.

It may come back, slowly

From 4.225 the method does return: the next guesses are 2.984, 2.241, 1.865, 1.745 and 1.7322, and only then 1.73205. Six steps were spent recovering from one bad tangent. From $x_0=1.5$x₀ = 1.5, where the curve is steeper, the guesses are 1.8, 1.7357 and 1.73206, three steps for the same accuracy.

The flatter the tangent, the worse the throw. From $x_0=1.01$x₀ = 1.01 the gradient is 0.06, and $x_1=34.17$x₁ = 34.17.

Going round in a cycle

The method can also go round in a loop. Take $f\left(x\right)=x^3-2x+2$f(x) = x³ − 2x + 2, with $f^{\prime }\left(x\right)=3x^2-2$f'(x) = 3x² − 2, and start at $x_0=0$x₀ = 0. Then f(0) = 2 and f'(0) = −2, so $x_1=0-\frac{2}{-2}=1$x₁ = 0 − 2/(−2) = 1.

From $x_1=1$x₁ = 1: f(1) = 1 − 2 + 2 = 1 and f'(1) = 3 − 2 = 1, so $x_2=1-\frac{1}{1}=0$x₂ = 1 − 1/1 = 0, which is where it began. The guesses run 0, 1, 0, 1 forever and never reach the root.

The tangent at 0 is y = 2 − 2x, which meets the axis at 1. The tangent at 1 is y = x, which meets the axis at 0. Each tangent points to the other start. The only root of this cubic is near −1.7693, on the far side of a hump, and neither tangent leads there.

The curve is $y=x^3-2x+2$y = x³ − 2x + 2. The gold straight line, its tangent at $x_0=0$x₀ = 0, meets the x-axis at 1, and the dashed line, its tangent at $x_1=1$x₁ = 1, meets it back at 0. The root, near −1.7693, is never reached from either.

A better start

The same cubic is easy from the right start. f(−2) = −8 + 4 + 2 = −2 and f(−1) = −1 + 2 + 2 = 3 have opposite signs, so the root lies between −2 and −1. From $x_0=-2$x₀ = −2: f'(−2) = 10, so $x_1=-2-\frac{-2}{10}=-1.8$x₁ = −2 − (−2)/10 = −1.8, then $x_2=-1.76995$x₂ = −1.76995 and $x_3=-1.76929$x₃ = −1.76929. Three steps give the root to 5 decimal places.

Newton-Raphson on $y=x^3-2x+2$y = x³ − 2x + 2, from $x_0=-2$x₀ = −2: the tangents close in on the root. Drag $x_0$x₀ to 0 and the four steps go 1, 0, 1, 0; drag it toward −0.82 or 0.82, where the curve is flat, and the next guess is thrown far away.

The wrong root

Even when the guesses settle, they may settle on a root nobody wanted. The method finds the root the tangent at the start points to. $x^3-3x$x³ − 3x has roots $-\sqrt{3}$−√3, 0 and $\sqrt{3}$√3; a start at 0.5 gives f(0.5) = −1.375 and f'(0.5) = −2.25, so $x_1=0.5-0.611=-0.111$x₁ = 0.5 − 0.611 = −0.111, heading for 0, not for $\sqrt{3}$√3. And a root of the equation may make no sense in the problem, such as a negative time or a depth deeper than the object.

Choosing a start

Find an interval where f changes sign, so that a root is known to be inside it. Find where f' = 0, and start well away from those points, where the curve is steep. Run the method until successive guesses agree to the accuracy wanted; if they jump about or repeat, start somewhere else. Then check the answer by a change of sign either side of it.

The usual mistakes

Thinking f is zero at a stationary point. It is the gradient that is zero there; the height, here −2, is not.

Deciding the curve has no root because the method failed. $y=x^2-2$y = x² − 2 crosses the axis twice; only the start was bad.

Thinking a shallow tangent helps. A small gradient makes $\frac{f}{f^{\prime }}$f/f' large, so the step is large too.

Giving $x_1$x₁ when $x_2$x₂ is asked for. In the cycle, $x_1=1$x₁ = 1, and one more step sends it back to $x_2=0$x₂ = 0.

Taking f(0) = 2 as a term of the sequence. It is the height of the curve at the guess, not the next guess.

Two applications

In the first application below, a walker starts Newton-Raphson at the top of a hill to find where the path reaches a lake. The summit is a stationary point, so the method has nowhere to go, and a start just beside it is thrown far beyond the lake.

In the second, a start at 1 minute sends the method back and forth between 1 and 2 forever, because the tangent at each meets the axis at the other. A start at 4 finds the landing time.