The Lagrange Error Bound

What is left over

A Taylor polynomial stops after a few terms, and the terms it leaves out add up to something. Write $P_n\left(x\right)$Pₙ(x) for the polynomial of degree n and $R_n\left(x\right)$Rₙ(x) for the remainder, so that $f\left(x\right)=P_n\left(x\right)+R_n\left(x\right)$f(x) = Pₙ(x) + Rₙ(x). The remainder is the error made by using the polynomial instead of the function.

For sin x, the polynomial of degree 3 is $P_3\left(x\right)=x-\frac{x^3}{6}$P₃(x) = x − x³/6. At x = 1 it gives $\frac{5}{6}=0.833333$5/6 = 0.833333, and sin 1 = 0.841471, so the remainder there is $R_3\left(1\right)=0.008138$R₃(1) = 0.008138. Without knowing sin 1 in advance, how large could that error be?

The bound

The Lagrange error bound says that $|R_n\left(x\right)|\le M|x-a{|}^{n+1}/\left(n+1\right)!$|Rₙ(x)| ≤ M|x − a|ⁿ⁺¹ / (n + 1)!, where a is the center of the series and M is any number at least as large as the size of the (n + 1)th derivative, $|f^{(n+1)}\left(t\right)|$|f⁽ⁿ⁺¹⁾(t)|, for every t between a and x.

It is an upper bound, not the error itself. The true error can be anything from 0 up to the bound, and it is usually smaller.

Where it comes from

There is an exact formula for the remainder: $R_n\left(x\right)=\frac{f^{(n+1)}\left(c\right){\left(x-a\right)}^{n+1}}{\left(n+1\right)!}$Rₙ(x) = f⁽ⁿ⁺¹⁾(c)(x − a)ⁿ⁺¹/(n + 1)! for some number c between a and x. It has the shape of the next term of the series, except that the derivative is taken at c instead of at a.

When n = 0 this is the mean value theorem: f(x) − f(a) = f'(c)(x − a), the change in f equals the gradient at some point c times the step.

The formula says that such a c exists, but not where it is. So the derivative at c is replaced by M, the largest size it could have anywhere between a and x. That is the bound: the next term, with its unknown derivative replaced by its largest size.

sin x to degree 3

Take $P_3\left(x\right)=x-\frac{x^3}{6}$P₃(x) = x − x³/6, centered at 0, on $-1\le x\le 1$−1 ≤ x ≤ 1. The bound needs the fourth derivative. The derivatives of sin x are cos x, −sin x, −cos x and then sin x again, so $f^{(4)}\left(t\right)=\sin t$f⁽⁴⁾(t) = sin t, and its size is never more than 1. So M = 1.

With n = 3, $|R_3\left(x\right)|\le 1\times \frac{|x{|}^4}{4!}=\frac{|x{|}^4}{24}$|R₃(x)| ≤ 1 × |x|⁴/4! = |x|⁴/24. The largest value on $-1\le x\le 1$−1 ≤ x ≤ 1 is at $x=\pm 1$x = ±1: $\frac{1}{24}=0.041667$1/24 = 0.041667. The error is at most 0.041667 anywhere on the interval.

The true error at x = 1 is 0.008138, about a fifth of the bound, and at x = 0.5 it is 0.000259 against a bound of $\frac{0.5^4}{24}=0.002604$0.5⁴/24 = 0.002604. The bound holds at every x, and it is never smaller than the error.

A sharper bound is free here. sin x has no $x^4$x⁴ term, so $P_3$P₃ is also $P_4$P₄, the polynomial of degree 4, and the bound with n = 4 uses the fifth derivative, cos t, again of size at most 1: $|R_4\left(x\right)|\le \frac{|x{|}^5}{5!}$|R₄(x)| ≤ |x|⁵/5!. At x = 1 that is $\frac{1}{120}=0.008333$1/120 = 0.008333, just above the true error 0.008138.

The gold curve is the true error $|\sin x-P_3\left(x\right)|$|sin x − P₃(x)|, and the plain one is the bound $\frac{|x{|}^4}{24}$|x|⁴/24, for x from −1 to 1. The bound lies above the error everywhere. At x = 1 the bound is 0.041667 and the error is 0.008138.

Choosing M

For sine every derivative is at most 1 in size, so M is easy. For other functions, M has to be found on the interval between a and x, and it depends on which side of a the point x lies.

Take $e^x$eˣ with $P_2\left(x\right)=1+x+\frac{x^2}{2}$P₂(x) = 1 + x + x²/2. The third derivative is $e^t$eᵗ, which is largest at the right-hand end of any interval. At x = 0.5 the interval is $0\le t\le 0.5$0 ≤ t ≤ 0.5, and $e^{0.5}$e^0.5 is less than 2, because e < 3 and so $e^{0.5}<\sqrt{3}<2$e^0.5 < √3 < 2. So M = 2 will do. The bound is $2\times \frac{0.5^3}{3!}=0.041667$2 × 0.5³/3! = 0.041667. The polynomial gives 1.625 and $e^{0.5}=1.648721$e^0.5 = 1.648721, so the true error is 0.023721, inside the bound.

At x = −0.5 the interval is $-0.5\le t\le 0$−0.5 ≤ t ≤ 0, where $e^t$eᵗ is at most $e^0=1$e⁰ = 1, so M = 1 and the bound is $\frac{0.5^3}{3!}=0.020833$0.5³/3! = 0.020833. The polynomial gives 0.625 and $e^{-0.5}=0.606531$e^(−0.5) = 0.606531, an error of 0.018469, again inside the bound.

How many terms are enough

The bound can be used before any calculating is done, to decide how many terms a given accuracy needs. Suppose e must be found from $1+1+\frac{1}{2!}+\frac{1}{3!}+\dots$1 + 1 + 1/2! + 1/3! + … with an error of at most 0.001. Here x = 1 and a = 0, and every derivative of $e^x$eˣ is $e^t\le e^1<3$eᵗ ≤ e¹ < 3 on $0\le t\le 1$0 ≤ t ≤ 1, so M = 3 and the bound for degree n is $\frac{3}{\left(n+1\right)!}$3/(n + 1)!.

For n = 5 that is $\frac{3}{720}=0.004167$3/720 = 0.004167, too large. For n = 6 it is $\frac{3}{5040}=0.000595$3/5040 = 0.000595, which is below 0.001, so the terms up to $\frac{1}{6!}$1/6! are enough. They add up to 2.718056, and e = 2.718282, so the true error is 0.000226.

The plain curve is y = sin x and the gold one is $x-\frac{x^3}{3!}$x − x³/3!. The shaded band is where the two differ by at most 0.01, out to |x| = 1.04. The Lagrange bound $\frac{|x{|}^5}{5!}$|x|⁵/5! is at most 0.01 for |x| up to 1.037, so it guarantees nearly all of that band before any error is measured. Drag N to 5 and the band reaches 1.76, with the bound guaranteeing 1.751; at N = 7 it reaches 2.5, with the bound guaranteeing 2.486.

The usual mistakes

Using the wrong derivative. The bound for degree n uses the (n + 1)th derivative and divides by (n + 1)!. For sin x to degree 3 that is 4! = 24, not 3! = 6.

Taking M at the center only. M must bound the derivative at every point between a and x. For $e^x$eˣ at x = 0.5, $e^0=1$e⁰ = 1 is too small; the derivative reaches $e^{0.5}$e^0.5 at the far end.

Counting terms as the degree. Three terms of the sine series, $x-\frac{x^3}{3!}+\frac{x^5}{5!}$x − x³/3! + x⁵/5!, reach degree 5, so their bound uses $\frac{x^7}{7!}$x⁷/7!, not $\frac{x^4}{4!}$x⁴/4!.

Reading the bound as the error. $\frac{1}{24}$1/24 is the most the error of $P_3$P₃ can be on $-1\le x\le 1$−1 ≤ x ≤ 1; the actual error at x = 1 is 0.008138.

A calculator’s sine key

In the application below, a calculator works out sin 0.5 from the sine series and has to be right to six decimal places. The Lagrange bound, with M = 1, shows that three terms are not enough and four are.