Interactive Explainer · Calculus Toolkit

Partial Derivatives are Slices

A surface $f(x,y)$ has no single slope — it tilts differently in every direction. A partial derivative fixes that by holding one variable still and asking the ordinary one-dimensional question. Move the point on the map and watch two slices, each with its own tangent, and the gradient arrow that collects them.

~9 min Deep Learning · Calculus · Gradients

Prelude

Freeze one axis, take a normal derivative

To measure $\partial f / \partial x$ at a point $(x_0, y_0)$, pin $y = y_0$. What remains is a curve in $x$ alone, and its ordinary slope is the partial derivative. Swap the roles for $\partial f / \partial y$:

\frac{\partial f}{\partial x}(x_0,y_0) = \lim_{h\to0}\frac{f(x_0+h,\,y_0)-f(x_0,y_0)}{h}, \qquad \frac{\partial f}{\partial y}(x_0,y_0) = \lim_{h\to0}\frac{f(x_0,\,y_0+h)-f(x_0,y_0)}{h}.

Stack the two partials into a vector and you have the gradient $\nabla f = \big(\partial f/\partial x,\; \partial f/\partial y\big)$. It points in the direction of steepest ascent, perpendicular to the contour lines — the single most important object in optimization.

The Lab

One point, two slices, one gradient

The map is a filled contour plot of $f(x,y)$: blue is low, orange is high. Click or drag on the map (or use the sliders) to move the point. The dashed lines are the two slices; each slice is redrawn below with its tangent, whose slope is the corresponding partial derivative. The green arrow is $\nabla f$ — notice it always points uphill, across the contours.

Point $x_0$ = 1.00

Point $y_0$ = 0.60

Contour map of $f(x,y)$. Orange dashed line $y=y_0$ generates the left slice; blue dashed line $x=x_0$ generates the right slice. The green arrow is the gradient $\nabla f(x_0,y_0)$.

Slice $g(x)=f(x,\,y_0)$ — hold $y$ fixed. Tangent slope $=\partial f/\partial x$.

Slice $g(y)=f(x_0,\,y)$ — hold $x$ fixed. Tangent slope $=\partial f/\partial y$.

$\partial f / \partial x$ (left slice slope) —

$\partial f / \partial y$ (right slice slope) —

$\lVert \nabla f \rVert$ (steepness) —

$f(x_0,y_0)$ (height) —

Try the saddle. Sit at the origin: both partials are $0$, so $\nabla f = 0$ — a critical point. Yet the left slice curves up and the right slice curves down. Zero gradient does not mean a minimum; here it is a saddle. Now step off-centre and watch the green arrow snap to point straight uphill.

The Payoff

Why the gradient points uphill

The two partials are the components of one vector, and that vector has a geometric job. Along any unit direction $u$, the rate of change is the directional derivative $\nabla f \cdot u$. By Cauchy–Schwarz this dot product is largest when $u$ aligns with $\nabla f$ — so the gradient points in the direction of steepest ascent, and $-\nabla f$ is the direction gradient descent actually steps.

D_u f = \nabla f \cdot u, \qquad \max_{\lVert u\rVert = 1} D_u f = \lVert \nabla f\rVert \ \text{ at } \ u = \frac{\nabla f}{\lVert \nabla f\rVert}.

Perpendicular to contours. Moving along a contour keeps $f$ constant, so that direction has zero slope; the gradient, being steepest, must be at right angles to it. Watch the arrow — it never runs along a colour band.
Every weight is an axis. A network's loss depends on millions of parameters. Its gradient has one partial per weight, each the slope of a private one-dimensional slice, and backprop computes them all in a single backward pass.
Flat spots are ambiguous. $\nabla f = 0$ marks minima, maxima, and saddles alike. The curvature of the slices — the Hessian — tells them apart.

Next up: when the output is also a vector, each output has its own gradient. Stack them as rows and you get the Jacobian — a whole matrix of partials that linearizes a map. That is the final explainer, The Jacobian is a Local Warp.