Taylor Series

Definition 🐧

A taylor series is an approximation of a function by capturing its $n$ derivatives. We can express a taylor approximation by the following:

Derivation of coefficients 🚀

We know that $a = f(x_0)$ and we know from derivatives that $b = f’(x_0)$. Let’s try to derive $c$.

• Derivation of $c$

  We can solve for $c$, which gets you

  $$c = \frac{f(x) - f(x_0) - f'(x_0)(x - x_0)}{(x - x_0)^2}$$

  And the function approximation should be perfectly correct as $x→x_0$, so we take the limit using l’hopital’s rule. If we do this, we get

  $$c = f''(x_0)/2$$

  which is exactly what it should be. In the next part, we will derive a general rule.

You can do this inductively for any number of terms. Just solve for that coefficient, and compute the limit as $x → x_0$ using L’hopital’s rule. The factorial in the term comes from the repeated differentiation of $(x - x_0)^n$.

Taylor’s Theorem 🚀

Theorem: A function can be expressed exactly in terms of its taylor expression as

So the second term is just the taylor approximation with $N$ terms, but this first term is actually quite magical. It tells us that we can make up the difference with just the next term, but you evalutate the $f^{(N)}$at some $c \in (x_0, x)$. Of course, this $c$ depends a lot on what $x$ is; you won’t get a perfect function just by adding another term with a singular value. We can also call this first term the remainder.

• Proof

  To help with this, we look at the case where $N = 2$ and then we use induction.

  Let’s set $h(x) = f(x_0) + f’(x_0)(x - x_0) + m_2(x - x_0)^2$. Let’s compute

  $$g(x) = f(x) - h(x)$$

  We know that $g(x_0) = 0$ by construction. Now, let’s solve for m such that $g(\tilde x) = 0$. This is possible because you have one equation and one variable. Now, let’s assume you have solved for $m_2$. Then, we have $g(\tilde x) = g(x_0) = 0$. By Rolle’s theorem, there exists some $x_1 \in (x_0, \tilde x)$ where $g’(x_1) = 0$.

  Let’s look at $h’(x)$. We have that $h’(x) = f’(x_0) + 2m_2(x - x_0)$. This means that $h’(x_0) = f’(x_0)$.

  Therefore, we have $g’(x_1) = g’(x_0) = 0$, and by Rolle’s theorem, we can find a $x_2 \in (x_0, x_1)$ that satisfies $g’’(x_2) = 0$. If we look at $h’’(x_2)$ we have $h’’(x_2) = 2m_2$. Because $g’’(x_0) = 0$, then $f’’(x_2)= 2m_2$, and we have $m_2 = f’’(x_2)/2$.

  Now, it’s critical to note here that we didn’t just define $m_2$; we defined it at the start of the problem. Therefore, what we derived is the existence of $x_2 \in (x_0, x)$ whose $f’’(x_2)/2$ is exactly equal to this remainder coefficent $m_2$!

  Ok, so this shows taylor’s theorem for 2. Can we do this inductively? Yes! Here’s a sketch

  1. Start by pinning down $m_n$ using the equality $f(\tilde x) = h(\tilde x)$

  2. Use Rolle’s theorem to get some $x_1 \in (x_0, \tilde x)$ and this yields $g’(x_1) = 0$. We always know that $f^{(k)}(x_0) = h^{(k)}(x_0)$. As such, $g’(x_0) = 0$, and we can use Rolle’s theorem again.

  3. Do this over and over, effectivley “pinching” the bounds of $x_k$. As you get to the final term, you will get $h^{(n)}(x_n) = n!*m_n$. Therefore, we have $m_n = f^{(n)}(x_n) / n!$. And this is exactly what we wanted to show! There is some $x_n \in (x_0, x_{n-1}) \in (x_0, \tilde x)$ that satisfies this coefficient.