FROM PAIRS TO A LOSS
One calculation at a time
Choose a pair. Follow its cosine through the row or column calculation.
First use downloads model files (WebGPU fp32: roughly 600 MB; WASM q8: roughly 150 MB). Later runs use the browser cache.
ILLUSTRATIVE 2D WORKSHEET · chosen vectors, not image embeddings
Checking WebGPU…
Rows = imagesColumns = text
See the vectors and the training code
U = F.normalize(image_encoder(images) @ W_I, dim=-1)
V = F.normalize(text_encoder(tokens) @ W_T, dim=-1)
S = log_scale.exp() * (U @ V.T)
y = torch.arange(len(images), device=S.device)
loss = (F.cross_entropy(S, y) + F.cross_entropy(S.T, y)) / 2
loss.backward()
optimizer.step()The notebook contains the full runnable setup. The browser worksheet inspects the loss; it does not train the pretrained CLIP checkpoint.