ES 667 IIT Gandhinagar

Home Lectures Assessment Assignments FAQ Slack

Lectures

Each lecture lists its reading material first, then its videos, then practice. Every kind of resource has its own icon.

NotesSlidesCheat sheetPDF handoutFull recording3-minute summaryNotebookInteractive lab

Foundations1–4Gradients and optimization5–9Generalization10Attention and language11–15Vision and language16–23Generative modelling24–28

Full lecture playlist3-minute summaries playlist

Foundations

Lectures 1–4

Lecture 01	Where Deep Learning Losses Come From I Distributions, likelihood, and loss.	NotesSlidesCheat sheet Where do loss functions come from?2:54
Lecture 02	Where Deep Learning Losses Come From II MLE, regression, and classification.	NotesSlidesCheat sheet Where MSE and cross-entropy come from2:53
Lecture 03	Likelihood to Loss in Practice MAP, regularization, and robust observation models.	NotesSlidesCheat sheet Outliers: how the noise model picks the loss2:55Priors: where regularization comes from2:54 Likelihood notebookRobust regression
Lecture 04	From Linear Models to Neural Networks Neurons, activations, multilayer networks, XOR, and approximation.	NotesSlidesCheat sheet From weighted sums to neurons: solving XOR2:59ReLU hinges and universal approximation2:53 Linear gradient descentXOR

Gradients and optimization

Lectures 5–9

Lecture 05	Optimization I — Gradient Estimates & Step Sizes Batches, stochastic gradients, and optimizer geometry.	NotesSlidesCheat sheet Full recordingSGD vs minibatch: how much data per step?3:00Why gradient descent zigzags: step sizes2:52
Lecture 06	Backpropagation & Autograd from Scratch Computation graphs, reverse mode, and a scalar engine.	NotesSlidesCheat sheet Full recordingBackprop by hand: one local rule3:03Build a tiny autograd engine from scratch2:59 Colab notebook
Lecture 07	Optimization II — Momentum, RMSProp & Adam Moving averages and adaptive coordinate scaling.	NotesSlidesCheat sheet Full recording 1Full recording 2Momentum: averaging away the zigzag3:03RMSProp and Adam: a step size per coordinate3:14
Lecture 08	Vector, Matrix & Affine Autograd Gradients, Jacobians, dense layers, and batched VJPs.	Calculus notesAutograd notesCalculus slidesAutograd slidesCheat sheet Backprop through layers: Jacobians to batches3:04 Colab notebook
Lecture 09	Valid Outputs by Design & Learning-Rate Schedules Constrained parameters, warmup, decay, and cosine.	Valid outputs notesLR schedule notesValid outputs slidesLR schedule slidesCheat sheet Valid outputs by design: softplus to softmax2:59Learning-rate schedules: warmup, decay, cosine2:53

Generalization

Lectures 10

Lecture 10

Generalization & Regularization Learning curves, weight decay, early stopping, dropout, augmentation, Mixup, and label smoothing.

NotesSlidesCheat sheet Overfitting, weight decay and early stopping3:08Dropout, augmentation, Mixup and label smoothing3:08 Learning curvesDropout and regularization

Attention and language

Lectures 11–15

Lecture 11	From characters to next-token prediction Tokenization, learned embeddings, a hidden-layer MLP, training, and generation.	NotesSlidesCheat sheet Full recordingNext-token prediction from scratch3:01
Lecture 12	Self-attention, from first principles	NotesSlidesCheat sheet Full recordingWhy attention?2:32Query, key, value3:05
Lecture 13	Self-attention, continued Embedding updates, next-token prediction, and positional encoding.	NotesSlidesCheat sheet From attention to prediction3:12Position and order2:56
Lecture 14	Self-attention and multi-head attention	Self-attention notesMulti-head attention notesSelf-attention slidesMulti-head attention slides
Lecture 15	From Attention to Applications Encoder, decoder-only and encoder–decoder Transformers; token and sentence classification; causal self-attention and cross-attention.	NotesPDFSlides Full recording

Vision and language

Lectures 16–23

Lecture 16	Vision Transformer for image classification Patch embeddings, full attention, embedding updates and CLS readout; CNN comparisons and image-covering experiments.	NotesSlidesPDF · 55 pp. Full recording
Lecture 17	CLIP: From Fixed Labels to Language-Defined Vision Applications, two encoders, a step-by-step contrastive-loss calculation, zero-shot classification and a full architecture summary.	NotesSlidesPDF · 168 pp.Addendum slidesAddendum PDF Full recording Applications ColabEmbeddingGemma labTrain a classifier
Lecture 18	From CLIP to Vision-Language Models Visual prefixes, cross-attention and learned summaries; architecture diagrams, code walkthroughs, training and generation.	NotesSlidesPDF · 178 pp. Full recording Three connections Colab
Lecture 19	Vision-Language Models, Part 2 Cross-attention and learned visual summaries; tensor shapes, training and generation, multiple-image questions and practical embedding demos.	NotesSlidesPDF · 178 pp. Full recording Three connections Colab
Lecture 20	Concept Bottleneck Models: I Interpretable-by-design prediction through named concepts; CUB examples, concept supervision, training schemes and interventions.	NotesSlidesPDF · 262 pp. Full recording
Lecture 21	Concept Bottleneck Models: II Concept sufficiency, concept embedding models, post-hoc bottlenecks and classification by description; supervision costs, evaluation and failure modes.	NotesSlidesPDF · 262 pp. Full recording
Session+	Learning ViTs with AI Tools Socratic live coding of a Vision Transformer in PyTorch: using AI tools to understand patches, attention, tensor shapes and MNIST training.	Full recording
Lecture 22	Object Detection: From One Label to a Set of Objects Spatial candidates, target assignment, worked losses, decoding, NMS, evaluation and DETR-style set prediction.	NotesSlidesPDF · 133 pp. Detector lab: score thresholds and NMS
Lecture 23	Beyond Boxes: Dense, Structured, and Promptable Vision Semantic, instance and panoptic segmentation; keypoints, depth, promptable masks and language grounding.	NotesSlidesPDF · 60 pp. Beyond Boxes lab: masks, pose, depth and grounding

Generative modelling

Lectures 24–28

Lecture 24	KL Divergence and Variational Inference Bayes for a tiny neural network and its intractable evidence; surprise, entropy, codes, cross-entropy and KL; classification loss and knowledge distillation as KL; Monte Carlo and the reparameterisation trick; the ELBO step by step; VI for the network.	NotesSlidesPDF · 152 pp.Notebook
Lecture 25	Learning to Generate I: Distributions and Autoencoders Fit and sample distributions, Gaussian mixtures on MNIST, from PCA to autoencoders (MLP, CNN, ViT), why autoencoders cannot generate, and the probabilistic view.	NotesSlidesPDF · 95 pp.
Lecture 26	Learning to Generate II: The Variational Autoencoder KL and the ELBO applied to a decoder, variational inference for one digit, amortisation and the encoder, reparameterisation, and the VAE trained on MNIST.	NotesSlidesPDF · 51 pp.
Lecture 27	GANs: Learn the Judge, Not the Likelihood A generator trained against a learned discriminator: the D and G steps, why G’s gradient flows through D, the minimax and non-saturating losses, mode collapse and conditioning.	NotesSlidesPDF · 37 pp.
Lecture 28	From Noise to Images: Denoising Diffusion Corrupt a digit step by step, train a network to predict the noise and sample backwards; the ELBO behind the loss, U-Nets and Transformers, guidance, and diffusion for text.	NotesSlidesPDF · 96 pp.

Notes are the complete PDF handouts or interactive lectures. Slides open the classroom presentation. Cheat sheets are two-page reviews.