Link Search Menu Expand Document
Probabilistic Machine Learning

Probabilistic ML course

Table of contents

  1. Probabilistic ML course
    1. Class Timings
    2. Google Classroom Link
    3. Prerequisites
    4. Course contents:
    5. Textbooks
    6. Other similar courses
    7. Grading Policy
    8. Useful YT channels/Playlists

Class Timings

Thursday, 12 PM to 1:30 PM, 7-102 Friday, 2 PM to 3:30 PM, 7-105

u7jjtu2

Prerequisites

  • Mathematics for ML: Recommended reading is MML book
  • A prior ML course: ES654 course at IIT Gandhinagar or equivalent
  • Good programming skills in Python. Familiarity with: numpy, Pandas, matplotlib, sklearn. Recommended reading is Python Data Science Handbook

Course contents:

  • Probability refresher: Probability theory, discrete distributions, continuous distributions, joint probability distributions, sampling from different distributions (e.g. using Box-Muller transform), uncertainty modelling, information theoretic concepts: (KL-divergence, entropy)
  • Bayesian concept learning: Likelihood, Prior, Posterior, Maximum Likelihood Estimate (MLE), Maximum A-Posteriori estimation (MAP), Full Bayesian Estimation with Conjugate Priors (Beta-Bernoulli model for the coin toss example, Normal-Normal model for estimating parameters of Normal distribution)
  • Bayesian supervised methods:
    • Regression: Bayesian linear regression, Robust linear regression via alternative likelihood (e.g. Laplace, Student-T)
    • Classification: Bayesian Logistic regression, Bayesian Naive Bayes
  • Latent Variable modelling: Gaussian Mixture Model, Probabilistic principal component analysis (PPCA)
  • Approximate Inference:
    • Sampling based strategies: Rejection sampling, Importance sampling, Markov Chain Monte Carlo (MCMC), Metropolis Hastings, Gibbs sampling, No U-Turn sampler (NUTS)
    • Variational inference: Mean field approach, Evidence Lower Bound (ELBO), Reparameterization trick, Stochastic Variational Inference, Automatic Differentiation Variational Inference (ADVI)
  • Gaussian Processes (GP): Multivariate Normal distribution and its properties, Kernels, GP regression, GP classification, Approximate Inference and Sparse GPs
  • Bayesian Optimization (BO) and Active Learning (AL): AL - Query by committee, Uncertainty sampling, Expected model change, BO - Acquisition functions, GP based BO, Random Forests based BO
  • Probabilistic Deep Learning: MC Dropout, Deep Ensembles, Bayesian neural networks (BNNs), Deep GPs

Textbooks

  • Kevin Murphy. Machine Learning, A Probabilistic Perspective. The MIT Press, 2012.
  • Kevin Murphy. Probabilistic Machine Learning: An Introduction. The MIT Press, 2022
  • Kevin Murphy. Probabilistic Machine Learning: Advanced Topics. The MIT Press, 2023.
  • Chris Bishop. Pattern Recognition and Machine Learning.
  • Allen Downey. Think Bayes: Bayesian Statistics in Python. Green Tea Apress, 2012
  • David Barber. Bayesian Reasoning and Machine Learning. Cambridge University Press, 2012
  • Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. The MIT Press, 2006
  • Richard McLearth. Statistical Rethinking. CRC Press, 2020.

Other similar courses

Grading Policy

  • Assignments: 20%
  • Quizzes: 20%
    • Quiz syllabus will be everything from previous quiz to present day
  • Projects: 60%

Useful YT channels/Playlists

  • https://www.youtube.com/user/mathematicalmonk
  • https://www.youtube.com/c/MutualInformation0
  • https://www.youtube.com/c/ritvikmath
  • https://www.youtube.com/c/MachineLearningSimulation
  • https://www.youtube.com/watch?v=UbaVGD4Lfis&list=PL05umP7R6ij1tHaOFY96m5uX3J21a6yNd
  • https://www.youtube.com/watch?v=BYUykHScxj8&list=PLDcUM9US4XdMROZ57-OIRtIK0aOynbgZN