Interactive Explainer · Calculus Toolkit

The Jacobian is a Local Warp

A nonlinear map bends space in complicated ways. But zoom into any one point and the bending straightens out: locally the map just stretches and rotates, exactly as its Jacobian matrix says. Watch a tiny circle around your chosen point turn into an ellipse — and watch that ellipse match what the Jacobian predicts.

~9 min Deep Learning · Calculus · Linear Algebra

Prelude

The best linear picture of a map

A map $F:\mathbb{R}^2 \to \mathbb{R}^2$ sends $(x,y)$ to $\big(F_1(x,y),\, F_2(x,y)\big)$. Its Jacobian at a point is the matrix of all four partial derivatives — the multi-output, multi-input generalization of $f'(x_0)$:

J(x_0,y_0) = \begin{bmatrix} \dfrac{\partial F_1}{\partial x} & \dfrac{\partial F_1}{\partial y} \\[1.1em] \dfrac{\partial F_2}{\partial x} & \dfrac{\partial F_2}{\partial y} \end{bmatrix}, \qquad F(x_0 + \Delta) \approx F(x_0) + J\,\Delta .

That approximation is the whole story. Near the base point, a small displacement $\Delta$ is mapped by multiplying it by $J$ — a linear stretch-and-rotate. So the little circle of displacements around the point becomes an ellipse: $J$ applied to a circle.

The Lab

Circle in, ellipse out

Left is input space; right is output space, both drawn at the same scale. The orange circle around $(x_0,y_0)$ is carried through the true nonlinear map to the solid orange loop on the right. The blue dashed ellipse is the prediction from the Jacobian alone, $F(x_0) + J\cdot(\text{circle})$. Shrink the radius and the two lock together — that is local linearity.

Base $x_0$ = 0.60

Base $y_0$ = 0.50

Warp strength $s$ = 0.50

Circle radius $r$ = 0.70

Input space. Orange circle + its two coloured radii (the $x$ and $y$ directions) around the draggable base point.

Output space. Solid orange = true image of the circle; blue dashed = Jacobian's ellipse; arrows are the columns of $J$.

Jacobian $J(x_0, y_0)$

[

—— ——

]

$\det J$ (area scale) —

Ellipse vs. true gap —

Watch the gap number. With a big radius the solid orange loop bulges away from the blue ellipse — the linear model is wrong far from the point. Slide the radius toward zero and the gap collapses: the true image and $J\cdot(\text{circle})$ become indistinguishable. The Jacobian is the map, but only in the small.

The Payoff

Stretch, rotate, and the chain rule

The two columns of $J$ are where the unit $x$- and $y$-vectors land — the orange and blue arrows on the right. They span the ellipse. Their determinant is the factor by which $J$ scales area: $|\det J| > 1$ expands, $< 1$ contracts, and $\det J = 0$ collapses the ellipse to a line (the map folds, and is locally non-invertible).

\underbrace{F \circ G}_{\text{compose maps}} \;\Longrightarrow\; J_{F\circ G}(x) = J_F\big(G(x)\big)\, J_G(x) \quad\text{(chain rule = matrix product).}

Backprop is Jacobian products. A deep network is a long composition of maps; its overall Jacobian is the product of each layer's Jacobian. Reverse-mode autodiff just multiplies these matrices right-to-left, reusing vectors instead of forming the full matrices.
Circle to ellipse is the singular-value story. The ellipse's axis lengths are the singular values of $J$. When one is tiny, gradients through that layer nearly vanish; when one is huge, they explode. Same picture, different stakes.
It is $f'$, promoted. One input, one output: $J$ is the number $f'(x_0)$. Same input and output dimension: $J$ is a square matrix. The definition never changed — only the shapes did.

The toolkit, assembled. A derivative is a local line; a gradient stacks partial slopes into the steepest direction; a Jacobian stacks whole gradients into a local linear map. Every one is the same move — replace a curve by its best line — and together they are the calculus that makes gradient-based learning possible.