ReLU Classification Playground

Data → ReLU features → score

See the function your classifier is using.

Change the evidence, architecture, or initialization. Then inspect how signed ReLU contributions assemble the logit surface and its z = 0 decision boundary.

Dataset

Evidence shown to the model

Four labels constrain the corners, not the space between them.

Data seed

Model

Architecture Hidden width2

Widen without resetting. Move right: added ReLUs begin with zero output contribution, so the current function stays exactly the same. Move left: a narrower model cannot preserve the same function, so the lab loads a fresh seeded model and training starts fresh. More capacity expands what the network can represent; an optimization run still determines which solution training finds.

Learning rate

Initialization

Hold data and architecture fixed; change only where optimization starts.

Weight seed

Filled XOR: Constructed · 4 ReLUs proves the capacity. Recommended is the reliable training route from a balanced seeded start.

Optimize

Constructed rule loaded for inspection. Choose Recommended or Seeded before training.

Training minimizes BCE. Accuracy uses a 0.5 threshold, so it can briefly fall even while BCE improves.

CURRENT DECISION FUNCTION

constructed

Four points do not specify four regions

Training accuracy100%

Balanced field accuracy75%

Training BCE0.31

Gradient steps0

Class 0Class 1z = 0linked probe

Shape and outline keep the data visible over every field.

ARCHITECTURE

Two inputs → two ReLUs → one score

9 parameters

OPTIMIZATION TRACE

Cross-entropy over steps

log scale

SIGNED OUTPUT TERMS

The pieces that add up to z(x₁, x₂)

shared scale · zero centered

Each map shows vⱼhⱼ, not merely activation. Blue raises the class-1 score, rust lowers it, and the hinge line marks where its ReLU turns on.

RAW LOGIT SURFACE

Orbit the learned function

Drag to orbit · wheel to zoom · orange sheet is z = 0

x₁ 0.25x₂ 0.75

z = 0.50 · p = 0.62 · class 1

Run the experiment

Change one thing. Predict first.

Evidence Compare four XOR corners with the filled field. Which unseen regions change?

Capacity Train a small model, then widen it. The boundary should not jump until optimization uses the new ReLUs.

Optimization Hold data and width fixed. Do different seeds learn the same surface?