Assignment 2 Due Tuesday Aug 23, 6:00pm

Instructions

1. Total marks: 6
2. Use TFP with Jax substrate for distributions
3. Use Jax for autograd functionality and vector programming
4. The assignment has to be done in groups of two
5. The assignment should be a single Jupyter notebook.
6. It is important that you are able to get insights from your results and not just present answers. As an example – if the question asks you to try a different likelihood, then you should be able to observe how this likelihood compares to the default likelihood.

Dataset

Create the following datasets to be used in the following questions

• Dataset 1

import jax.numpy as jnp
import jax
x = jnp.linspace(-2, 2, 100)
f = 3*x + 2.0
eps = 0.5*jax.random.normal(key=jax.random.PRNGKey(0), shape=(100, ))
y = f + eps

• Dataset 2

import jax.numpy as jnp
import jax
x = jnp.linspace(-2, 2, 100)
f = 3*x + 2.0
eps = 0.5*jax.random.normal(key=jax.random.PRNGKey(0), shape=(100, ))
y = f + eps
y = y.at[1].set(y[1] + 3)
y = y.at[30].set(y[30] + 1)

• Dataset 3

from sklearn.datasets import make_classification
import numpy as np
X, y = make_classification(
    n_features=2, n_redundant=0, n_informative=2, random_state=1, n_clusters_per_class=1
)
rng = np.random.RandomState(2)
X += 2 * rng.uniform(size=X.shape)

Questions

1. Assume scale=1 for the likelihood noise, learn the MLE parameters for Dataset 1 assuming Normal likelihood. [0.5 marks]
2. Assume scale=1 for the likelihood noise, learn the MLE parameters for Dataset 2 assuming Normal likelihood. [0.5 marks]
3. Learn the MLE parameters for Dataset 1 assuming Normal likelihood. Do not assume scale and also estimate it using MLE. You may prefer to learn log(scale) as a part of your optimisation procedure. Plot predicted function along with two scale (𝜇 ± 2𝜎) band to show likelihood noise. [0.5 marks]
4. Repeat question 3 for Dataset 2. [0.5 marks]
5. Assume scale=1 for the likelihood noise, learn the MAP parameters for Dataset 1 assuming Normal likelihood and a Normal prior: P(theta) = N(0, b^2 I). Show the effect of varying b on the MAP parameters and the learnt function. [1 marks]
6. Repeat question 5, but instead of Normal prior choose a Laplace prior [0.5 marks]
7. Repeat question 3 for Dataset 2 but instead of Normal likelihood, Student-T likelihood with varying degrees of freedom. You should manually vary the degree of freedom in a range (you should choose the range based on the insights). [1 mark]
8. Implement Logistic regression using Bernoulli likelihood and learn the MLE and MAP parameters for Dataset 3. For MAP, assume normal prior. Also plot contours/surfaces of predictive mean and variance.[1.5 marks]