Lecture library
Slides are the complete handouts; Present opens the classroom build, and each Cheatsheet is a two-page review guide.
Resource key: ▤ handout · ▶ classroom presentation · ★ cheatsheet · ⌨ notebook · ◉ recording
Foundations
| No. | Lecture | Resources |
|---|---|---|
| 1 | Where Deep Learning Losses Come From I — distributions, likelihood, and loss | ▤ Slides · ▶ Present · ★ Cheatsheet |
| 2 | Where Deep Learning Losses Come From II — MLE, regression, and classification | ▤ Slides · ▶ Present · ★ Cheatsheet |
| 3 | Likelihood to Loss in Practice — MAP, regularization, and robust observation models | ▤ Slides · ★ Cheatsheet · ⌨ Likelihood notebook · ⌨ Robust regression |
| 4 | From Linear Models to Neural Networks — neurons, activations, multilayer networks, XOR, and approximation | ▤ Slides · ▶ Present · ★ Cheatsheet · ⌨ Linear GD · ⌨ XOR |
Gradients & optimization
| No. | Lecture | Resources |
|---|---|---|
| 5 | Optimization I — Gradient Estimates & Step Sizes — batches, stochastic gradients, and optimizer geometry | ▤ Slides · ▶ Present · ★ Cheatsheet · ◉ Recording |
| 6 | Backpropagation & Autograd from Scratch — computation graphs, reverse mode, and a scalar engine | ▤ Slides · ▶ Present · ★ Cheatsheet · ⌨ Colab · ◉ Recording |
| 7 | Optimization II — Momentum, RMSProp & Adam — moving averages and adaptive coordinate scaling | ▤ Slides · ▶ Present · ★ Cheatsheet · ◉ Momentum · ◉ RMSProp & Adam |
| 8 | Vector, Matrix & Affine Autograd — gradients, Jacobians, dense layers, and batched VJPs | ▤ Calculus · ▤ Autograd · ★ Cheatsheet · ⌨ Colab |
| 9 | Valid Outputs by Design & Learning-Rate Schedules — constrained parameters, warmup, decay, and cosine | ▤ Valid outputs · ▤ LR schedules · ★ Cheatsheet |
Generalization
| No. | Lecture | Resources |
|---|---|---|
| 10 | Generalization & Regularization — learning curves, weight decay, early stopping, dropout, augmentation, Mixup, and label smoothing | ▤ Slides · ▶ Present · ★ Cheatsheet · ⌨ Learning curves · ⌨ Mechanisms |
Attention and language
Read the interactive lecture at your own pace, or choose Present for the classroom slides. Each cheat sheet fills two printable A4 pages.
| Part | Lecture | Resources |
|---|---|---|
| 1 | From characters to next-token prediction - tokenization, learned embeddings, a hidden-layer MLP, training, and generation | ▤ Interactive lecture · ▶ Present · ★ A4 cheat sheet |
| 2 | Self-attention, from first principles - queries, keys, values, scaling, causal masking, and residual updates | ▤ Interactive lecture · ▶ Present · ★ A4 cheat sheet |
| 3 | From Attention to Applications - encoder, decoder-only and encoder-decoder models; token labels, CLS and English-to-Hindi cross-attention | ▤ Interactive lecture · ▶ Present · ▤ PDF |
| CLIP | CLIP: From Fixed Labels to Language-Defined Vision — image–text applications, two encoders, contrastive loss and architecture summaries | ▤ HTML lecture · ▶ Present · ▤ PDF · 168 pages · ⌘ Applications Colab · ⌘ EmbeddingGemma · WebGPU lab · Ask the course · Search course code |
| VLMs | From CLIP to Vision-Language Models — visual prefixes, cross-attention, compression, training and image-conditioned generation | ▤ HTML lecture · ▶ Present · ▤ PDF · 178 pages |
Object detection and beyond boxes
Two connected lectures: first predict a set of objects, then choose richer outputs when boxes are not enough.
| Part | Lecture | Resources |
|---|---|---|
| I | Object Detection: From One Label to a Set of Objects — candidates, assignment, worked losses, NMS, evaluation and learned queries | HTML lecture · Present · PDF · 133 pages · Detector lab |
| II | Beyond Boxes: Dense, Structured, and Promptable Vision — semantic and instance segmentation, pose, depth, promptable masks and grounding | HTML lecture · Present · PDF · 60 pages · Beyond Boxes lab |
Convolutional neural networks
One complete interactive deck built on the ML course CNN lecture and its tutorials: original filtering examples, a step-by-step LeNet exercise, real MNIST training and layer inspection, plus the DL course’s gradients, receptive fields, computational accounting, modern blocks, and transfer.
Read the lecture · Present · Sources and teaching choices · PyTorch companion
Generative modelling
Making images from random numbers. First the maths toolkit (KL divergence and variational inference, built from information theory and a tiny Bayesian network); then variational autoencoders built around one handwritten 3; then a generator trained against a learned judge; then learning to remove noise a little at a time. Every figure and number comes from small models trained for the lectures, except where a slide credits a published model.
| Part | Lecture | Resources |
|---|---|---|
| KL and VI | KL Divergence and Variational Inference — Bayes for a tiny neural network and its intractable evidence; surprise, entropy, Huffman codes, cross-entropy and KL as wasted bits; KL at work: classification and knowledge distillation as one KL; closed forms and Monte Carlo; grid search, gradient descent and covariance families; mixtures and the reparameterisation trick; the ELBO one rule at a time; mean-field and full-covariance VI for the network. Twenty-two numbered sections and three sub-sections, each linked to its notebook cell | ▤ Interactive lecture · ▶ Present · ▤ PDF · 152 pages · ▤ Every click · 331 pages · ⌘ Notebook (Colab) |
| VAEs | Learning to Generate: Autoencoders and VAEs — fitting and sampling distributions, Gaussian mixtures on MNIST, from PCA to autoencoders (MLP, CNN, ViT), the probabilistic view, the ELBO and the VAE on one running digit | ▤ Interactive lecture · ▶ Session 1 · ▶ Session 2 · ▤ PDF · 146 pages · ▤ Every click · 367 pages |
| GANs | GANs: Learn the Judge, Not the Likelihood — two networks, the D and G steps, why G’s gradient flows through D, the minimax and non-saturating losses, mode collapse and conditioning | ▤ Interactive lecture · ▶ Present · ▤ PDF · 37 pages · ▤ Every click · 96 pages |
| Diffusion | Diffusion Models: Create by Learning to Undo Noise — in the order of Paras Chopra’s explainer: the forward process with every number (a five-step toy schedule, β → α → ᾱ by hand) and the shortcut; the reverse step on one pixel; what the network learns and the eight-line training loop; sampling; the score; noise schedules, DDIM and one-step models; classifier-free guidance, latents and text, with real outputs of the original DDPM model and Stable Diffusion. Thirty-four numbered sections, each linked to its notebook cell | ▤ Interactive lecture · ▶ Present · ▤ PDF · 102 pages · ▤ Every click · 246 pages · ⌘ Notebook (Colab) |