# Shared mathematics and diagram conventions

## Roles stay fixed

| Role | Colour | Typical objects |
|---|---|---|
| Data | Blue `#245EDB` | Observations, image states, data points |
| Inference / q | Purple `#8B2CDE` | Variational distributions, encoder; explicitly fixed forward q in diffusion |
| Model / p | Teal `#0F766E` | Target density, decoder, generator, learned reverse distribution |
| Latent values and randomness | Amber `#AA4E08` | u, r, z, epsilon, explicit draws |
| Objective and gradients | Coral `#BE123C` | Loss, bound gap, derivative, return gradient path |
| Discriminator | Green `#147737` | D and its source-classification score |

Colour is accompanied by labels and arrows. A whole density expression can take its owning q or p colour. A probabilistic dependency diagram is labelled separately from a neural computation graph. Solid arrows show computations or explicitly labelled conditional transitions. Inference relationships use labelled dashed purple arrows. Gradients use labelled coral arrows in a separate return lane.

## Notation

`u` is a generic unknown, `r` is the coin's probability of heads, `z` is a latent variable (a deterministic code in an ordinary AE), `x` is an observation and `𝒟` is the dataset. `θ` belongs to the generative model, `φ` to the shared encoder, `ψ` to free variational parameters, and `ω` to the discriminator. `η` is a learning rate.

`σ` is a standard deviation; `logvar` means `log σ²`, so `std = exp(0.5 * logvar)`. PyTorch Normal uses `(loc, scale)`, while mathematical `N(μ,σ²)` uses variance in the second position. Logs are natural, and KL is measured in nats. Beta shapes `a,b` are positive. Diffusion uses `αₜ=1−βₜ` and `ᾱₜ=∏ₛ₌₁ᵗαₛ`; a VAE weighting factor is named `β_KL`.

## Distinctions that must remain visible

- Exact identities use `=`; rounded computed values use `≈`.
- `KL(q || posterior)` is the evidence-bound gap. The VAE objective also contains `KL(q || prior)`. They are different quantities.
- Maximise the ELBO; minimise its negative. A negative ELBO is not an error.
- The evidence is constant with respect to variational parameters for a fixed model. It need not be constant with respect to learned generative parameters.
- A finite Monte Carlo estimate of KL can be negative. Exact KL is nonnegative under the usual support and integrability conditions.
- A Gaussian location-scale reparameterisation does not explain Beta sampling; the Beta example uses PyTorch's differentiable sampler.
- The decoder outputs likelihood parameters. Displaying a Bernoulli probability image differs from drawing binary pixels.
- Reconstruction loss sums over event dimensions and then averages over the batch. A mean must name its axes.
- Freezing discriminator parameters preserves the input-gradient path needed by the generator. Detaching that path does not.
- Diffusion's clean estimate and reverse mean differ. The tractable training posterior conditions on the clean image. The simplified noise MSE omits weights from the variational objective.

## Visual and computational checks

Titles are 40 logical pixels, main prose 28, principal equations 32 and code at least 26. Essential captions are 22. Long equations and diagrams can scroll in phone reading mode; their projection layout stays fixed. Nonessential provenance is smaller and repeated in the notes.

Fixed-axis comparisons keep the same scale and the same samples where the lesson depends on controlling those quantities. Checkpoint outputs, fresh updates, toy values and schematic arrows are labelled separately. A successful test supports its stated claim; it does not establish exact inference, model fidelity or classroom pacing.

Numerical fixtures and notebook computations supply the displayed values. Original editable SVG diagrams use the shared helpers; measured charts are generated by the lecture scripts. No generated raster illustration is used to typeset equations or fabricate numerical results.

## Primary references

- [PML 2023 schedule and recordings](https://nipunbatra.github.io/pml2023/schedule.html), [VI notebook](https://github.com/nipunbatra/pml-teaching/blob/master/notebooks/variational-inference.ipynb), [VAE notebook](https://github.com/nipunbatra/pml-teaching/blob/master/notebooks/vae.ipynb).
- [Attention lecture series](https://nipunbatra.github.io/attention/).
- Kingma and Welling, [Auto-Encoding Variational Bayes](https://arxiv.org/abs/1312.6114).
- Goodfellow et al., [Generative Adversarial Nets](https://arxiv.org/abs/1406.2661).
- Ho et al., [Denoising Diffusion Probabilistic Models](https://arxiv.org/abs/2006.11239).
- Lilian Weng, [From Autoencoder to Beta-VAE](https://lilianweng.github.io/posts/2018-08-12-vae/) and [What are Diffusion Models?](https://lilianweng.github.io/posts/2021-07-11-diffusion-models/), diagram and exposition references.

Per-lecture provenance files record model/data sources, seeds and copied checkpoint hashes.