Link Search Menu Expand Document
Probabilistic Machine Learning

Assignment 3 Due September 15, 6:00pm

Instructions

  1. Total marks: 4
  2. Use TFP with Jax substrate for distributions
  3. Use Jax for autograd functionality and vector programming
  4. The assignment has to be done in groups of two
  5. The assignment should be a single Jupyter notebook.
  6. It is important that you are able to get insights from your results and not just present answers. As an example – if the question asks you to try a different likelihood, then you should be able to observe how this likelihood compares to the default likelihood.

Dataset

Create the following dataset to be used in the following questions

from sklearn.datasets import make_classification
import numpy as np
X, y = make_classification(
    n_features=2, n_redundant=0, n_informative=2, random_state=1, n_clusters_per_class=1
)
rng = np.random.RandomState(2)
X += 2 * rng.uniform(size=X.shape)

Questions

For all of the following questions, please consider binary classification as the model.

  1. Find the approximate posterior distribution using the Laplace approximation. Use a P(theta) = N(0, b^2 I) prior [1]
  2. Using the above Laplace approximation for the posterior distribution, find the predictive posterior distribution using MC sampling [1]
  3. Use probit approximation to find the closed form distribution for the posterior [1]
  4. Using the above, find the closed form posterior predictive using probit approximation and compare against the MC sampling + Laplace approximation. [1]

References

  1. https://www.tensorflow.org/probability/api_docs/python/tfp/distributions/ProbitBernoulli
  2. https://www.cse.iitk.ac.in/users/piyush/courses/tpmi_winter19/tpmi_w19_lec7_slides_print.pdf