Probabilistic Generative Models: Lab Exercises

Submission: Lab exercises must be submitted on the following ecampus page:

ecampus page for submission

You must submit only two files: your report as a PDF file and your code as a ipynb file. Please, only submit one time per group, and put all group members name at the beginning of each document.

Deadlines (2022):

  • Lab exercise 1: October 12, 23:59

  • Lab exercise 2: October 19, 23:59 October 23, 23:59

  • Lab exercise 3: November 2, 23:59 November 4, 23:59

Deadlines are hard, the submission website automatically close after them, so I strongly advise to not submit during the last minutes in order to avoid problems.

Groups: There should be 1, 2 or 3 students per group. You can not change groups between submission (except if you have problems with other students in your group, please send me an email in this case). I expect a stronger work from a group of three students, e.g. an average assessment (i.e. grade of 10/20) for a group of 1-2 will be considered as bad (below 10/20) for a group of 3.

Report instructions

Lab exercises comprise two parts:

  1. code to complete;

  2. a report to write.

So the question is: what do you need to write in the report? There are no specific instruction! You must think about the report as an essay: the objective of the report is that you convince me that you understand the theoretical foundation of the model and how to implement it in practice. Use your own word and notations, try to process the course and the lab exercise and explain them to me. Do not write handwavy explanation. You should probably use Latex for this.

Length: 3-6 pages, but these are not hard limits - you can do less, you can do a little more. Just don’t write too much, go to the essential (this is not a paper, do not try to write a related work section or anything like this  —  but you can try to build bridges between course concepts or concepts outside the course). Formal notations with minimum writing to be understandable. Just convince me that you know what you are talking about. :)

Advice: do not postpone this to the last minute. It is not something that you can do at the last minute.

Scoring: as long as you do the work seriously, that you commented the code you wrote and you submit a nice report, I will give you a good grade. Do not worry if you did not succeed to do everything or if you didn’t understand something. If you explain in the report what you did not succeed, and I can see you did some effort, I will give you a good grade.

Lab exercise 1: Variational Auto-Encoders

If the link for the link to download the data does not work, try to download it from here.

Report guidelines:

  • What is a Variational Auto-Encoders?

  • How does it compare with Sigmoid Belief Networks?

  • How does VAEs deal with different kind of distribution family? (observed, latent) How is it compared to SBN trained via mean field theory/variational methods? (think about a SBN with different family distributions than Bernoulli - so that won't be called SBN anymore, but you get the idea!)

  • How are VAE implemented? What should you take care about? What kind of distributions can be problematic? Why?

  • Highlight some results from your experiments! Can you describe the neural architecture? (mathematical description! - this a scientific report)

Lab exercise 2: Restricted Boltzmann Machine with continuous observations

Report guidelines:

  • Why are Restricted Boltzmann Machines interesting as globally normalized models?

  • When using a single Markov Chain to sample from the model, what do you observe with respect to the model space exploration?

  • In practice, what advice could you give to someone who needs many samples from the model?

Lab exercise 3: Real NVP normalizing flow

Report guidelines:

  • What problem does normalizing flows solve? What is their benefit compared to other model we saw in the course? (think about the training loss)

  • What is the intuition behing NICE and Real NVP models?