ONE-DIMENSIONAL WITNESS
Build a function from ReLU hinges.
Construct an interpolant that is guaranteed by its knots, then train the same kind of network from a deterministic random start. Keep existence and optimization as separate claims.
CONSTRUCTED WITNESS
Sine · fixed-knot interpolation
Training-sample MSE—not trained
Dense-grid MSE0.0000401 evaluation points
Max grid gap0.000max of 401 gaps
Gradient steps019 parameters
TARGET + MODEL
Can the blue curve stay inside the ε band?
ĝ(x) = c + Σ vⱼ ReLU(wⱼx + bⱼ)
Target f(x)
Approximation ĝ(x)
f(x) ± ε
Interpolation knots
Training samples
Drag an orange interior knot left or right; endpoint knots stay fixed.
ERROR
Pointwise gap |f(x) − ĝ(x)|
MODEL CONTRACT
One input → six ReLUs → one output
- Source
- Fixed knots
- Output bias c
- 0.000
- Objective
- No optimization
- Evaluation
- 401-point grid
This construction is a witness for this target and width. It does not show that gradient descent would discover the same weights.
SIGNED HINGE FUNCTIONS
The output is the bias plus every contribution below.
All cards use the same zero-centred y-scale. A small curve is genuinely a small contribution.
USE IN CLASS
Change one thing. Predict first.
- Capacity: add hinges; predict where the largest gap will move.
- Optimization: keep width fixed; compare two initialization seeds.
- Theorem: say what existence promises, then name what this optimizer run does not prove.