The Newton-Raphson Method

A guess and how far off it is

The positive root of f(x) = x² − 2 is √2, the number whose square is 2. Pretend it is unknown, and guess x₀ = 2. Then f(2) = 4 − 2 = 2, a long way from 0, so 2 is not the root. The height of the curve above the axis says how far off the guess is, but not which way to move or how far.

Follow the tangent to the axis

Close to a point, a smooth curve is almost the same as its tangent there, and a straight line is easy to follow to the x-axis. So replace the curve by its tangent at the guess and see where that crosses.

f'(x) = 2x, so the gradient at x = 2 is f'(2) = 4. The tangent goes through (2, 2) with gradient 4: y − 2 = 4(x − 2), which is y = 4x − 6. It meets the x-axis where 4x − 6 = 0, at x = 1.5. That is the next guess, x₁ = 1.5, much nearer √2 ≈ 1.414 than 2 was.

The gold curve is y = x² − 2, which crosses the x-axis at √2 ≈ 1.414. The dashed tangent at x₀ = 2 has gradient 4 and reaches the axis at x₁ = 1.5, just to the right of the root.

The formula

Do the same at any guess xₙ. The tangent goes through (xₙ, f(xₙ)) with gradient f'(xₙ), so its equation is y − f(xₙ) = f'(xₙ)(x − xₙ), or y = f(xₙ) + f'(xₙ)(x − xₙ).

The x-axis is where y = 0: 0 = f(xₙ) + f'(xₙ)(x − xₙ). Divide by f'(xₙ) and move the fraction across: x − xₙ = −f(xₙ)/f'(xₙ). So the next guess is xₙ₊₁ = xₙ − f(xₙ)/f'(xₙ). This is the Newton-Raphson formula.

The fraction has a meaning of its own. The tangent rises f(xₙ) over a horizontal distance from the crossing to xₙ, and its gradient is rise over run, so that horizontal distance is f(xₙ) / f'(xₙ): the height divided by the gradient. A steep tangent gives a short step and a shallow one a long step.

Three steps

From x₀ = 2: f(2) = 2 and f'(2) = 4, so x₁ = 2 − 2/4 = 1.5.

From x₁ = 1.5: f(1.5) = 2.25 − 2 = 0.25 and f'(1.5) = 3, so x₂ = 1.5 − 0.25/3 = 1.416667, which is 17/12.

From x₂ = 17/12: f(17/12) = 289/144 − 2 = 1/144 and f'(17/12) = 17/6, so x₃ = 17/12 − 1/408 = 577/408 = 1.4142157. To seven decimal places √2 = 1.4142136. So x₃ differs from √2 by only 0.0000021, and its first five decimal places, 1.41421, are those of √2.

Close up near the root. The dashed tangent at x₁ = 1.5 has gradient 3 and reaches the axis at x₂ = 1.4167. The gold curve crosses at 1.4142, only 0.0025 to the left, so near the root the curve and its tangent can hardly be told apart.

The correct digits roughly double

The errors, each guess minus √2, are 0.586, 0.0858, 0.00245 and 0.0000021. Each is roughly a third of the square of the one before: 0.0858² = 0.0074, and a third of that is 0.0025. Squaring a small error makes it far smaller, which is why the number of correct decimal places roughly doubles at every step once the guesses are close.

One more step gives x₄ = 1.41421356237469, whose first eleven decimal places are those of √2 = 1.41421356237310.

Simplify the formula first

For f(x) = x² − 2 the formula is xₙ₊₁ = xₙ − (xₙ² − 2)/(2xₙ). Put it over one denominator: (2xₙ² − xₙ² + 2)/(2xₙ) = (xₙ² + 2)/(2xₙ), which is the same as (xₙ + 2/xₙ)/2, the average of xₙ and 2/xₙ. If xₙ is too big then 2/xₙ is too small, and their average lands between them.

For the cube root of 2, f(x) = x³ − 2 and f'(x) = 3x², and the formula simplifies to xₙ₊₁ = (2xₙ³ + 2)/(3xₙ²). From 1 it gives 1.333333, 1.263889, 1.259933 and 1.259921.

Which root it finds

x² − 2 = 0 also has the root −√2. Start at x₀ = −1 and the tangent slopes the other way: x₁ = −1 − (−1)/(−2) = −1.5, then −1.416667 and −1.4142157. The method finds the root the tangent at the start points to, so the start decides which root comes out.

So locate the root first. If f changes sign between two values, a continuous f has a root between them, and a start in that interval usually reaches it. Afterwards check the answer the same way: to 4 decimal places √2 is 1.4142, and f(1.41415) is negative while f(1.41425) is positive.

$x_{n + 1} = x_{n} - \frac{f ( x _{n} )}{f ^{'} ( x _{n} )}, f (x) = x^{3} - 2 x + 2 : x_{0} = - 2.5, x_{3} = - 1.77$

x₃ = −1.769 to three places: from a start in the root's basin, Newton converges in a few steps

Start at x₀ = 0 and take four steps

The curve is y = x³ − 2x + 2, with one root near −1.7693. From x₀ = −2.5 three tangents give −1.985, −1.797 and −1.770. Drag x₀ and the number of steps; a start between about −0.8 and 1 can cycle, be thrown far away, or wander for many steps before it settles.

The usual mistakes

Adding the step. f(xₙ)/f'(xₙ) crosses the equals sign, so it is subtracted: from 2, x₁ = 2 − 2/4 = 1.5, while 2 + 2/4 = 2.5 walks away from the root.

Turning the fraction upside down. The step is the height divided by the gradient, f/f'; using f'/f gives 2 − 4/2 = 0 from x₀ = 2.

Not dividing by the gradient. 2 − f(2) = 0 moves by the full height of the curve, which is only right when the tangent has gradient 1.

Leaving out the constant. For x³ = 2 the function is x³ − 2, not x³; with f(x) = x³ the method heads for 0.

Working in degrees. The derivative of sin x is cos x only in radians.

Two applications

In the first application below, the depth a floating buoy sinks to satisfies the cubic x³ − 3x² + 2.4 = 0. A change of sign places the root between 1 and 1.5, and Newton-Raphson from 1 finds it in two steps.

In the second, the edge of a cube of volume 2 cm³ is the cube root of 2, and the simplified formula gives it to six decimal places in four steps, with the correct places going from 2 to 4 to more than 6.

Worked example: A Spherical Buoy Floating in a Harbor: The Depth It Sinks To, from a Cubic Solved by Newton-Raphson

Question A buoy is a hollow sphere of radius 1 m, and its average density is 0.6 of the density of sea water. When it floats, the depth x m of its lowest point below the surface satisfies x²(3 − x) = 2.4: the part below the water is a cap of volume π x²(3 − x)3, and it must be 0.6 of the volume of the whole sphere, 4π3. (a) Write the equation as f(x) = 0, where f(x) = x³ − 3x² + 2.4, and show that it has a root between x = 1 and x = 1.5. (b) Use the Newton-Raphson method with x₀ = 1 to find the depth to 3 decimal places.

1.Expand the bracket: 3x² − x³ = 2.4, so f(x) = x³ − 3x² + 2.4 = 0. Then f(1) = 1 − 3 + 2.4 = 0.4 and f(1.5) = 3.375 − 6.75 + 2.4 = −0.975.

Expand the bracket: f(x) = x³ − 3x² + 2.4. Then f(1) = 0.4 and f(1.5) = −0.975.
2.(a) f(x) changes sign between x = 1 and x = 1.5, and f is continuous, so there is a root between them. The cubic has two other roots, one negative and one near 2.66, but a depth must lie between 0 and the diameter, 2 m.

(a) f(x) changes sign between x = 1 and x = 1.5, so a root lies between them: the depth, which must be between 0 and 2 m.
3.Differentiate: f'(x) = 3x² − 6x. From x₀ = 1: f(1) = 0.4 and f'(1) = −3, so x₁ = 1 − 0.4−3 = 1.1333, to 4 decimal places.

The tangent at (1, 0.4) has gradient f'(1) = −3. It meets the x-axis at x₁ = 1 − 0.4−3 = 1.1333.
4.At x₁ = 1.1333: f(x₁) = 0.00237 and f'(x₁) = −2.9467, so x₂ = 1.1333 − 0.00237−2.9467 = 1.1341. The next step gives x₃ = 1.1341 again.

From x₁ = 1.1333 the next tangent lands at x₂ = 1.1341, and the one after at x₃ = 1.1341 again.
5.(b) The buoy floats with its lowest point 1.134 m below the surface, to 3 decimal places. Check: f(1.1335) = 0.0019 and f(1.1345) = −0.0011 have opposite signs, so the root lies between them. The depth is more than the radius, as it must be for a buoy that is denser than half of the water.

(b) The buoy floats with its lowest point 1.134 m below the surface, more than its radius, since it is denser than half of the water.

Answer: (a) f(1) = 0.4 > 0 and f(1.5) = −0.975 < 0, so there is a root between 1 and 1.5; (b) 1.134 m

Common mistakes

Starting at x₀ = 0 or x₀ = 2, where f'(x) = 3x² − 6x = 0. The tangent there is horizontal and never meets the x-axis, so the formula would divide by zero.
Giving the root near 2.66. It solves the cubic, but a depth of more than 2 m would put the lowest point of a sphere of diameter 2 m further down than the sphere reaches, so it does not describe the buoy.

More solving equations numerically problems, worked step by step →

Worked example: A Steel Cube of Exactly 2 Cubic Centimeters: Its Edge to Six Decimal Places in Four Steps

Question A laboratory needs the edge of a cube of volume exactly 2 cm³, worked out to six decimal places. Its edge x cm satisfies x³ = 2. (a) Show that the Newton-Raphson method for f(x) = x³ − 2 gives x_n+1 = 2x_n³ + 23x_n². (b) Starting at x₀ = 1, find x₁, x₂, x₃ and x₄ to 6 decimal places, give the edge to 6 decimal places, and say, for each of x₂, x₃ and x₄, how many decimal places it gives correctly once rounded.

1.f(x) = x³ − 2 has f'(x) = 3x², so x_n+1 = x_n − x_n³ − 23x_n² = 3x_n³ − x_n³ + 23x_n². (a) This simplifies to x_n+1 = 2x_n³ + 23x_n².

(a) With f'(x) = 3x², x_n+1 = x_n − x_n³ − 23x_n² = 2x_n³ + 23x_n².
2.From x₀ = 1: x₁ = 2 + 23 = 1.333333. Then x₂ = 2 × 2.370370 + 23 × 1.777778 = 6.7407415.333333 = 1.263889.

The tangent at x₀ = 1 lands at x₁ = 1.333333, and the tangent there lands at x₂ = 1.263889.
3.The next two steps give x₃ = 1.259933 and x₄ = 1.259921, and x₅ = 1.259921 again. (b) The edge is 1.259921 cm, to 6 decimal places.

(b) The next tangents land at x₃ = 1.259933 and x₄ = 1.259921: the edge is 1.259921 cm.
4.Compare each iterate with the edge. x₂ = 1.263889 is correct to 2 decimal places, x₃ = 1.259933 to 4, and x₄ to more than 6. Check: 1.2599205³ = 1.9999974 and 1.2599215³ = 2.0000021, so the edge rounds to 1.259921.

x₂, x₃ and x₄ are correct to 2, 4 and more than 6 decimal places: the number of correct places roughly doubles at each step.

Answer: (a) x_n+1 = 2x_n³ + 23x_n²; (b) 1.333333, 1.263889, 1.259933, 1.259921: the edge is 1.259921 cm, and x₂, x₃ and x₄ are correct to 2, 4 and more than 6 decimal places

Common mistakes

Writing f(x) = x³ and f'(x) = 3x², which leaves out the −2. The method then finds the root of x³ = 0, which is 0, not the edge of the cube.
Expecting one step for each decimal place, and so six steps for six places. Near the root the error is roughly squared at each step, so the correct places go from 2 to 4 to more than 6.

More solving equations numerically problems, worked step by step →