ES 654 IIT Gandhinagar

Home Lectures Assessment Feedback

Lectures

Every class of the January–April 2019 offering, numbered in order, with its slides, readings and notebooks. Notebooks open in Colab.

SlidesNotebookReading

Foundations and decision trees1–4Linear models and optimisation5–14Neighbours, Bayes and active learning15–19SVMs, logistic regression and neural networks20–26

Foundations and decision trees

Lectures 1–4

Lecture 01	Introduction and logistics	Slides
Lecture 02	Accuracy metrics; decision trees I Reading: Tom Mitchell, Machine Learning, chapter 3.	Accuracy metrics, classification and regressionDecision trees: discrete input, discrete output
Lecture 03	Decision trees II and III; bias and variance	Decision trees IIDecision trees III: regressionBias–variance and cross-validation
Lecture 04	Ensemble learning Bagging, boosting and random forests.	SlidesResearch methods (RM101)

Linear models and optimisation

Lectures 5–14

Lecture 05	Linear regression I; maths for ML I Reading: An Introduction to Statistical Learning, chapter 3.	Linear regressionMaths for ML I
Lecture 06	Paper presentations I
Lecture 07	Linear regression II Slides as in lecture 5.	Linear regression
Lecture 08	Maths for ML II; feature selection; gradient descent	Maths for ML II: contours, gradients, LagrangianFeature selectionLinear regression III: gradient descentVideo: Constrained optimisation (Khan Academy)
Lecture 09	Paper presentations II
Lecture 10	Gradient descent and SGD: a worked example	SlidesReading: An overview of gradient descent algorithms Notebook: motivating regularisation
Lecture 11	Paper presentations III
Lecture 12	GD versus SGD; KKT conditions; convexity; ridge regression	GD versus SGD: summaryShufflingKKT conditionsConvexityRidge regressionReading: KKT conditions Notebook: contours for GD and SGD
Lecture 13	Subgradients, lasso and coordinate descent	SubgradientsLassoCoordinate descentCoordinate descent for regularised and unregularised regression
Lecture 14	Project phase I presentations

Neighbours, Bayes and active learning

Lectures 15–19

Lecture 15	K-nearest neighbours Parametric versus non-parametric methods. Reading: An Introduction to Statistical Learning, section 2.1.2.	Slides
Lecture 16	Curse of dimensionality; naive Bayes Why KNN is non-parametric.	Naive BayesCourse: Bayesian methods in machine learning (Coursera) Notebook: curse of dimensionality
Lecture 17	Categories of ML; active learning	ML categorisationActive learningReading: Active learning literature survey Notebook: active learning
Lecture 18	MLE, MAP and Bayesian estimation I	Slides
Lecture 19	MLE, MAP and Bayesian estimation II; SVM I	MLE, MAP and Bayesian estimation IISVM IVideo: Multivariate GaussianReading: SVM without tears Notebook: Bayesian linear regression

SVMs, logistic regression and neural networks

Lectures 20–26

Lecture 20	SVM II Followed by ML demo presentations.	SlidesReading: SVM tutorial
Lecture 21	SVM III	Slides
Lecture 22	ML demos
Lecture 23	SVM IV; logistic regression	SVM IVLogistic regression
Lecture 24	SVM duality; neural networks I	SVM: dual and primalNeural networks I
Lecture 25	Neural networks II Slides as in lecture 24. Quiz 3 in class.	Neural networks
Lecture 26	Dropout; unsupervised learning	Unsupervised learning Notebook: dropout

Slides are the PDFs used in class. Paper presentations and project presentations were student sessions.