Jacobian Matrix

Tags	Basics

The idea

The gradient $\nabla$ gives us the derivative of a function $f: \mathbb{R}^n → \mathbb{R}$ . What if we wanted to find the derivative of a function $f: \mathbb{R}^m → \mathbb{R}^n$ ??

Intuitively, that means that we need $m\times n$ derivatives, one for each pair of outputs to inputs. We can express this as the Jacobian matrix $J$ , defined as follows:

J_{i, j} = \frac{\partial}{\partial x_j}f(x)_i

Every row of the jacobian represents an element of the output, and each column represents an element of the input.

☝

Occasionally, we represent

J

Df

. The function

J(a)

is just the jacobian dependent on the point that it's evaluated at. We can also represent this with

(Df)(a)

Linear approximations with the jacobian

This is very simple: a linear approximation at a point $a$ is just

f(a) + (J)(a)\vec{p}

where $p$ is in the input domain

Intuition of the Jacobian

Here, we can start to understand what $J$ actually is. It's just a row matrix consisting of gradients of each individual component of the output. As such, if we think of matrix-vector multiplication in the row format, you can think of $J(a)$ as a bunch of dot products

J(a)\vec{p} = \begin{bmatrix} \nabla f_1 \\\nabla f_2 \\ \vdots \\ \nabla f_m\end{bmatrix} \vec{p} = \begin{bmatrix} \nabla f_1 \cdot \vec{p}\\\nabla f_2 \cdot \vec{p} \\ \vdots \\ \nabla f_m \cdot \vec{p}\end{bmatrix}

Now, each one of those $\nabla f_i \cdot \vec{p}$ is just a linear approximation WRT $f_i$ , or if you will, the derivative. Now, you can probably understand why the jacobian is a linear approximation of $f$ .

Geometric intuition of the Jacobian

The jacobian has many uses. You can also imagine the Jacobian itself as a linear transformation. But what does this tell us?

Well, you can imagine $f$ as a transformation but it isn't linear, so it might pinch and stretch unevenly. The Jacobian approximates the behavior of $f$ at the specific point, and it approximates it as a linear transformation. This linear transformation is just the Jacobian matrix!

The determinant of the Jacobian therefore tells you how space is expanded or shrunk at a certain point. This is used in multivariable change-of-variable integrations.