ES 335 IIT Gandhinagar

Home Lectures Assessment FAQ Slack

Lectures

Each lecture lists its reading first (slides and cheat sheet), then its one-minute videos, then practice: notebooks and interactive apps. Slides and notebooks come from ml-teaching, which also lists all slides and all notebooks.

NotesSlidesCheat sheetShort videoNotebookInteractive labReading

Foundations1–2Trees, model fit and ensembles3–7Linear models and optimisation8–15Classification16–19Neural networks20–26Unsupervised and reinforcement learning27–29Neighbours and support vector machines30–33Also in the series

Machine learning in 1 minute playlistAll Shorts in course order

Foundations

Lectures 1–2

·	Prerequisite test An open-book test on Python, linear algebra, probability and calculus, required to enrol.	Prerequisite test
Lecture 01	Introduction and logistics What machine learning is, how the course runs, and a tour of what current models can do.	Slides How does a computer learn to sort good tomatoes from bad?Same fruits, no labels. Can a computer still group them? Transcription and local LLMsSegmentation and localizationPrompt to image
Lecture 02	Conventions, metrics, classification and regression Notation for data, train and test sets, the confusion matrix, accuracy, precision and recall, and errors for regression.	SlidesCheat sheet Could you write rules that spot every handwritten 4?Is the sample number a useful feature for predicting a tomato's quality?Spam, cat or dog, house prices, tumors: which is a regression problem?A decision tree gets 100% on its training data. Does that tell you its test score?A test is 99% accurate. Is it a good test?Test A is 99% accurate, Test B is 95.8%. Which would you use?A model says 'good' but the tomato is bad. What is that mistake called?A test flags 48 people as possibly sick. How many are really sick?2% of 10,000 defective, precision 0.60, recall 0.90. How many get flagged?Model 1: precision 0.70, recall 0.85. Model 2: the reverse. Which job needs Model 1?Precision 100%, recall 10%, average 55%. Is that a fair score?A program spots handwritten 4s with 88.56% accuracy. Is it any good?Why might MAE be preferred over MSE?A model's two errors are +5 and -5. Its mean error is 0. Is it perfect?A tree fits its 400 training points exactly. Can it do worse than guessing the average? Rule-based vs MLConfusion matrixDummy baselines

Trees, model fit and ensembles

Lectures 3–7

Lecture 03	Decision trees I Entropy, information gain, and growing a tree one greedy question at a time.	SlidesCheat sheet What does entropy measure in a decision tree?Tennis data, 14 days. Why does 'Outlook' give the biggest information gain?Temperatures 40, 48, 60, 72, 80, 90. Where should a decision tree try to split?A regression tree leaf holds 30, 32 and 40 kWh. What does it predict? EntropyReal outputReal input, discrete output
Lecture 04	Decision trees II Real-valued inputs and outputs, Gini impurity, and how depth leads to overfitting.	SlidesCheat sheet A group is half yes, half no. What is its Gini impurity?Does a deeper decision tree always do better on new data?A tree with 3 levels of questions fits a sine wave. How many values can it predict? Overfitting and underfitting
Lecture 05	Bias and variance I; cross-validation Underfitting and overfitting, the bias-variance decomposition, k-fold cross-validation and tuning hyperparameters.	Bias and variance ICross-validationCheat sheet: Bias and varianceCheat sheet: Cross-validation and hyperparameters Three curves fit the same 16 points. Which will predict new points best?Refit a line and a wiggly curve on 5 new samples. Whose predictions jump more?100 data points, 5-fold cross-validation. How many train the model each round?You scale with the mean of all the data, then cross-validate. Is the test score honest?100 points, 10 positive, 5 random folds of 20. What is the chance a fold gets none?Rows sorted: 80 Yes, 20 No. Train on the first 80, test on the last 20. Will it work? Tuning hyperparametersBias-variance analysisBias-variance charts
Lecture 06	Ensemble learning Bagging, random forests and boosting, and why a vote of many models beats one.	SlidesCheat sheet Draw 100 times from 100 items, putting each back. How many different items?3 models, each wrong 30% of the time, take a vote. How often is the vote wrong?A model gets 2 of 10 points wrong. In boosting, which does the next one focus on?In AdaBoost, a small model is wrong 90% of the time. How much say should it get?A random forest split sees 2 of 4 measurements. How often is the best one hidden? Boosting explainedFeature importanceEnsemble representation
Lecture 07	Decision tree pruning; bias and variance II Cutting a grown tree back, and a second look at the bias-variance trade-off.	Tree pruningBias and variance IICheat sheet: Decision trees IICheat sheet: Bias and variance A fully grown decision tree is cut back to fewer leaves. Can it do better on new data?Add a column of dice rolls to a linear regression. Can its training error go up? Tree pruning

Linear models and optimisation

Lectures 8–15

Lecture 08	Maths for ML Gradients of linear and quadratic forms, rank, contour plots, and the time complexity of decision trees.	Maths for MLContours and gradientsTree time complexityCheat sheet u = (1, 2, 3), v = (1, 0, 2). What is the rank of the 3 by 3 matrix u vᵀ? Contour plotsMeshgrid and contours
Lecture 09	Linear regression I Least squares, the normal equation, and what one outlier does to the fit.	SlidesCheat sheet Line 1 passes through 3 of 4 points, line 2 through none. Which has less squared error?Four points lie on a rising line. Move one. Can the best-fit slope turn negative?Four data sets share one best-fit line and the same average miss. Do they look alike? Tree pruning
Lecture 10	Linear regression II The geometric view, basis functions, multicollinearity and dummy variables.	SlidesCheat sheet: Linear regression ICheat sheet: Linear regression II Linear regression fits straight lines. Can it fit a curve?Colour is red, green or blue. Code it as 1, 2, 3 for a linear model? Geometric linear regressionBasis expansionBasis expansion 2Dummy variables
Lecture 11	Gradient descent From Taylor series to the update rule, the step size, and batch, mini-batch and stochastic gradient descent.	SlidesCheat sheet f(x) = x², start at x = 4, step size 0.1. Where is x after one gradient descent step?f(x) = x², start at 4, step size 1.1. Does gradient descent reach the bottom?Each step nudges a line toward one random point. Does the error drop every step?Gradient descent stops where the slope is 0. Is that the lowest point? Taylor seriesContours and gradient descent
Lecture 12	Movie recommendation and matrix factorisation Filling in a ratings matrix with low-rank factors, fit by alternating least squares or gradient descent.	SlidesCheat sheet Priya never rated Sholay. Can the other ratings predict hers? Movie recommendation
Lecture 13	Ridge regression A penalty on large weights: the closed form, shrinkage and choosing the penalty.	SlidesCheat sheet Make a curve fit its training points worse on purpose. Can it do better on new points?Ridge with penalty 100 is fit to (1, 101) and (2, 102). What does it predict at x = 1? Ridge regression
Lecture 14	Lasso regression The L1 penalty and sparse weights, subgradients, coordinate descent and feature selection.	SlidesCheat sheet Ridge and lasso both shrink weights. Which one can make a weight exactly 0?California housing has 8 columns. Trying every set, how many models do you train? Lasso regression
Lecture 15	Constrained optimisation Lagrange multipliers and the KKT conditions.	SlidesCheat sheet

Classification

Lectures 16–19

Lecture 16	Logistic regression I The sigmoid, odds and the decision boundary, and why linear regression fails for classes.	SlidesCheat sheet A model's score is 0. What probability does the sigmoid turn it into?Logistic regression with two inputs: what shape is the border between yes and no?A straight line sorts tomatoes from oranges. Add one huge orange. Does the cut move? Why use logitsLogistic regression in Torch
Lecture 17	Logistic regression II Cross-entropy from maximum likelihood, its gradient, and softmax for many classes.	SlidesCheat sheet The true class gets probability 0.01. Which loss punishes it more?Softmax turns the scores 2, 1, 0 into probabilities. What does the 0 class get? Why use logitsLogistic regression in Torch
Lecture 18	Convexity, Hessians and IRLS Tutorial: testing convexity, Newton's method, and iteratively reweighted least squares.	SlidesCheat sheet Gradient descent needs 27 steps down x² from x = 4. How many does Newton's method need? Second-order logistic regression
Lecture 19	Precision-recall curves Thresholds, the precision-recall trade-off, average precision and ROC curves.	SlidesCheat sheet A detector flags scores of 0.5 or more. Raise the bar to 0.7. What usually happens?A perfect model has an ROC AUC of 1. What does a coin-flipping model get? PR curves

Neural networks

Lectures 20–26

Lecture 20	Multi-layer perceptrons Perceptrons, XOR and hidden layers, activations, and the forward pass as matrices.	SlidesCheat sheet Weights 1 and 1, bias -1.5: which logic gate is this neuron?Can one straight line put the blue points on one side and the amber on the other?What is the fewest number of straight lines that separates XOR?Two layers with no activation, y = 3 · (2x). More powerful than one layer?784 inputs, 100 hidden units, 1 output. How many numbers does the network learn? Perceptron learning algorithm
Lecture 21	Automatic differentiation Computation graphs, the chain rule, and forward versus reverse mode.	SlidesCheat sheet f = (x + y) × z at x = 2, y = 3, z = 4. Nudge x by 0.01: how much does f go up?LeNet-5 has 61,706 weights. How many passes does backpropagation need for every slope?The slope of 3x² + 2x + 1 at x = 2 is 14. Nudge x by 10⁻²⁰: what slope do you get? Autodiff
Lecture 22	Autograd from scratch Building a small reverse-mode autograd engine in Python.	Cheat sheet Autograd from scratch
Lecture 23	Next-token prediction A character-level model of names: embeddings, an MLP, and sampling.	SlidesCheat sheet A language model reads "the cat sat on the". What does it output? NamesNext-token demo
Lecture 24	Convolutional networks I Convolution as a sliding dot product, stride, padding, pooling, weight sharing and LeNet.	SlidesCheat sheet A 5 by 5 filter slides over a 28 by 28 image. How big is the output?LeNet-5 learns 61,706 numbers in 5 layers. Where are most of them? 1D CNNEdge detectionLeNetConvolutionMulti-channel convolution
Lecture 25	Grad-CAM, equivariance and invariance Seeing which pixels a CNN relies on, and how convolution treats shifted inputs.	Notes Grad-CAMGrad-CAM enhanced
Lecture 26	Pretrained features with DINOv3 Using a frozen vision model's features for search, classification and regression.	Freeze a trained image model, train only a 7,690-number last layer. Test accuracy? ClassificationRegressionDataVisual fashion searchPatch similarity

Unsupervised and reinforcement learning

Lectures 27–29

Lecture 27	Unsupervised learning: clustering K-means and choosing k, and clustering by density.	SlidesCheat sheet Points 1, 2, 3, 7, 8, 9, centres at 0 and 5: where are they after one k-means step?More centres always lower k-means' error. So is the k with the lowest error the best?Six points in one clump, three in another, one alone. Must the lone point join a group? Simple image clusteringImage clusteringDocument clustering
Lecture 28	Principal component analysis Tutorial: directions of maximum variance, eigenvectors of the covariance, and explained variance.	SlidesCheat sheet PCA 1PCA 2
Lecture 29	Reinforcement learning Agents and rewards, returns, value functions, Q-learning and exploration.	SlidesCheat sheet No labels, only a reward at the end. What does the agent learn from?A car too weak to climb the hill pushes right all the time. How many steps to the flag? Flappy BirdRL demoExploration

Neighbours and support vector machines

Lectures 30–33

Lecture 30	K-nearest neighbours Distance metrics, choosing k, the curse of dimensionality and approximate search.	KNNApproximate KNNCheat sheet New fruit: orange or apple? Its 3 nearest neighbours vote.k-NN uses height (metres) and income (rupees). Which one decides who is nearest?Point A is at (3, 0), point B at (2, 2). Which is nearer to (0, 0)?A ball just fits inside a cube. In 10 dimensions, how much of the cube does it fill?A million stored points: must k-NN compare a new point with every one? Curse of dimensionality
Lecture 31	Support vector machines I The maximum-margin classifier, its dual, and support vectors.	SlidesCheat sheet Both lines make 0 mistakes. Which is safer for new points?An SVM's border sits at 0. Remove the blue point at 2. Where is the border now? SVMSVM with CVXOPT
Lecture 32	Support vector machines II: kernels Feature maps, the kernel trick, and polynomial and RBF kernels.	SlidesCheat sheet Blue points sit inside a ring of amber. Can a straight line split them? Understanding kernelsKernelsPrimal and dual
Lecture 33	Support vector machines III: soft margin Slack variables, the C parameter and the hinge loss.	SlidesCheat sheet One amber point sits among the blue ones. Must the dividing line twist to fit it? Soft margin

Also in the series

3 topics

·	Naive Bayes and Bayesian networks Bayes' rule for classification, conditional independence, and smoothing counts.	Cheat sheet Tennis on a sunny, cool, humid, windy day: what does naive Bayes predict?All 4 overcast days were "yes". What chance does naive Bayes give "no" to the next one?
·	Time series and sequence models Forecasting baselines, autoregression, and recurrent networks.	Cheat sheet Sales 3, 5, 4, 6, 5, 7, 6, 8. A model uses the last 2 days. What does it predict next?What happens to gradients in a simple RNN during backpropagation?
·	Object detection Bounding boxes and how much two of them overlap.	Two 70 x 70 boxes, one shifted 20 right and 20 down. How much do they overlap?

Slides are the PDF handouts used in class. Cheat sheets are two-page A4 summaries. Each one-minute video answers one question, given as its title.