Interactive Explainer · Calculus Toolkit

The Derivative is a Local Line

A derivative is not a formula to memorize — it is the slope of the straight line a curve becomes when you zoom in far enough. Drag the base point, collapse a secant onto the tangent, and zoom until the curve straightens. The number you land on is $f'(x_0)$.

~8 min Deep Learning · Calculus · Gradients

Prelude

Slope of what, exactly?

The derivative $f'(x_0)$ is defined as a limit of slopes of secant lines. Pick a nearby point a distance $h$ away, draw the line through $(x_0, f(x_0))$ and $(x_0+h,\, f(x_0+h))$, and read off its slope. Now let the second point slide in toward the first:

f'(x_0) \;=\; \lim_{h \to 0} \frac{f(x_0+h) - f(x_0)}{h}.

Two pictures of the same fact live below. The secant slider shrinks $h$ so the average slope converges to the instantaneous one. The zoom slider does it geometrically: magnify the graph around $(x_0, f(x_0))$ and any smooth curve flattens into a straight line — the tangent. That line, and its slope, is the entire content of "derivative."

The Lab

Secant, tangent, zoom

Choose a curve, then drag left/right on the plot (or use the $x_0$ slider) to move the base point. Pull the $h$ slider toward $0$ and watch the orange secant swing onto the blue tangent. Then push the zoom slider and watch the curve itself straighten.

Base point $x_0$ = 1.00

Secant step $h$ = 1.50

Zoom ×1.0

Blue = tangent line $y = f(x_0) + f'(x_0)(x-x_0)$. Orange = secant through the two dots, slope $\big(f(x_0+h)-f(x_0)\big)/h$. Grid spacing is labelled and shrinks as you zoom.

Secant slope $\;\big(f(x_0+h)-f(x_0)\big)/h$ —

Derivative $f'(x_0)$ —

Gap (secant − tangent) —

Height $f(x_0)$ —

Do this now. Set $x_0$ somewhere interesting, then drag the $h$ slider from $2.5$ down toward $0$. The big orange number slides toward the blue derivative, and the gap shrinks — for a smooth curve it shrinks roughly like $h$ itself. Separately, leave $h$ alone and crank the zoom: the curve peels away from your eye as a perfect line. Same limit, two lenses.

The Payoff

Differentiable means locally linear

Zoom into a differentiable function anywhere and it looks like a line. That is the whole reason calculus powers deep learning: near the current weights, a loss surface is well approximated by its tangent (its gradient), so we can take a confident step downhill. Formally, the tangent is the best linear approximation, with error that dies faster than $h$:

f(x_0 + h) = \underbrace{f(x_0) + f'(x_0)\,h}_{\text{tangent line}} + o(h).

Corners break it. Try $f(x)=|x|$ in your head at $x_0=0$: zooming never straightens the kink, and the left/right secant slopes disagree. No single tangent, no derivative — the same reason $\mathrm{ReLU}$ needs a subgradient at $0$.
The slope is the derivative. There is nothing else in the definition: it is literally the number the secant slopes converge to.
Backprop is this, everywhere at once. Each layer contributes its local slope; the chain rule multiplies them. Every arrow in a computation graph is a tiny version of this picture.

Next up: functions of many variables do not have one slope but a whole vector of them — the gradient. Each entry is exactly this one-dimensional picture, taken along one axis. That is the next explainer, Partial Derivatives are Slices.