ES 661 IIT Gandhinagar

Home Lectures Assessment Assignments Interactive apps Slack

Lectures

Each lecture lists its slides, cheat sheets and reading first, then its recording, then its notebooks. Most slides and notebooks come from pml-teaching. Cheat sheets are two-page summaries.

SlidesCheat sheetFull recordingNotebookInteractive labReading

Probability and likelihood1–4Estimation and Bayesian regression5–8Sampling and MCMC9–14Uncertainty in deep learning15–16Hierarchical models, meta-learning and self-supervision17–20Active learning and Bayesian optimisation21–22Variational inference23–25

Lecture playlist

Probability and likelihood

Lectures 1–4

Lecture 01	Introduction and logistics	SlidesCheat sheet Recording
Lecture 02	Bayes' rule for ML	SlidesCheat sheet Recording Notebook: Bayes' rule
Lecture 03	Distributions, sampling and likelihood	SlidesCheat sheet Recording Notebook: distributionsNotebook: likelihoodNotebook: MLE
Lecture 04	Maximum likelihood estimation for the univariate normal	SlidesCheat sheetSlides: bias and variance of estimators (revision)Reading: SGD is unbiased Recording Notebook: likelihoodNotebook: MLENotebook: biased estimators

Estimation and Bayesian regression

Lectures 5–8

Lecture 05	MLE for linear and logistic regression	SlidesCheat sheet Recording Notebook: log-likelihood for linear regression
Lecture 06	MAP for coin tosses; the multivariate normal	SlidesCheat sheet Recording Notebook: MAP
Lecture 07	Bayesian linear regression	SlidesCheat sheet Recording Notebook: multivariate normal
Lecture 08	Laplace approximation; calculus	Slides: Laplace approximationSlides: calculusCheat sheet Recording Notebook: Laplace approximationNotebook: calculus

Sampling and MCMC

Lectures 9–14

Lecture 09	Monte Carlo sampling	SlidesCheat sheet Recording
Lecture 10	Monte Carlo sampling; sampling from common distributions	SlidesCheat sheet Recording
Lecture 11	Uncertainty in neural networks	Recording Notebook: uncertainty in neural networks
Lecture 12	Rejection and importance sampling	SlidesCheat sheet Recording Notebook: rejection and importance sampling
Lecture 13	Practical MCMC	Cheat sheet Recording Notebook: MCMC with hamiltorch
Lecture 14	MCMC theory	Slides: Markov chainsSlides: MCMCCheat sheet: MCMCCheat sheet: hidden Markov models Notebook: Markov chains

Uncertainty in deep learning

Lectures 15–16

Lecture 15	Calibration	Reading: tutorial on classifier calibration Recording Notebook: reliability diagrams
Lecture 16	MC dropout and deep ensembles	Recording Notebook: MC dropout

Hierarchical models, meta-learning and self-supervision

Lectures 17–20

Lecture 17	Hierarchical models	Recording Notebook: hierarchical linear regression in NumPyro
Lecture 18	Meta-learning: hypernetworks and neural processes	Cheat sheet Recording Notebook: hypernetworks and neural processes
Lecture 19	Meta-learning: SIREN, coordinate networks, hypernetworks and neural processes	Cheat sheet Recording Notebook: SIREN
Lecture 20	Self-supervised learning	Reading: illustrated survey of self-supervised learning Recording Notebook: self-supervised learning

Active learning and Bayesian optimisation

Lectures 21–22

Lecture 21	Active learning	Cheat sheetReading: active learning survey (Settles)Reading: deep active learning Recording Notebook: active learningInteractive article: active learning
Lecture 22	Bayesian optimisation	Cheat sheetReading: Distill article on Bayesian optimisationReading: acquisition functions Recording Notebook: motivationNotebook: implementation from scratch

Variational inference

Lectures 23–25

Lecture 23	KL divergence, variational inference, Monte Carlo sampling and reparameterisation	Cheat sheet: information theoryCheat sheet: variational inference Recording Notebook: variational inference
Lecture 24	Variational inference II: the ELBO and coin tosses	Cheat sheet Recording Notebook: variational inference
Lecture 25	Variational inference III: variational autoencoders	Reading: from autoencoder to beta-VAE (Lilian Weng)Reading: VAE tutorial (Jaan Altosaar) Recording Notebook: VAE

Notebooks hosted on this site open as pages; the rest open on GitHub. Assignment and project dates are under Assessment.