## Deep learning: Lab exercisesThere should be 2-4 students per group. Deadlines are hard and you cannot change groups. Lab exercises comprise two parts: code, and a report to write.
So the question is: what do you need to write in the report? There are no specific instruction! You must think about the report as an essay: the objective of the report is that you convince us that you understand the theoretical foundation of the model and how to implement it in practice. Use your own word and notations, try to process the course and the lab exercise and explain them to us. Do not write handwavy explanation. You should probably use Latex for this.
Goal of the lab exercise Neural network(s) architecture description Implementation details Experiment results: hyperparameters, learning curves, analysis, model comparison.
## Frequently asked questions?
## Lab exercises 1 and 2You only need to submit the code of the second lab exercise, but to succeed the second one you must succeed the first one… Submission: Hard deadline: April 6, 23:59:00 Submission website: ecampus You must submit one ipynb (Lab exercise 2) and one PDF file — there will be a penalty is you submit a zip/rar/tar.gz/etc file
## Lab exercises 3
## Lab exercise 4
The VAE that we will develop is based on the following generative story: \(z \sim p(z)\) \(x \sim p(x | z ; \theta)\)
where the latent representations z take value in \(R^n\). The prior ditribution p(z) is a multivariate Gaussian where each coordinate is independent. We fix the mean and variance of each coordinate to 0 and 1, respectively. The conditional distribution \(p(x | z ; \theta)\) is parameterized by a neural network: it is the decoder! The generated pixels x are independent Gaussians with a fixed variance. Note: this kind of VAE will be quite bad at generating MNIST picture. Therefore, when you do you experiments, you should both generate picture and display the mean parameters of the output distributions. This is a well known problem of VAE, you can try to play with the network architecture and the parameters to improve generation. Although the decoder is similar to the auto-encoder decoder, the encoder is different: it must return two tensors, the tensor of means and the tensor of variances. As the variance of a Gaussian distribution is constrained to be strictly positive, it is usual to instead return the log-variance (or log squared variance), which is unconstrained. If you exponentiate the log-variance, you get the variance which will be strictly positive as the exponential function only returns positive values. Similarly to the auto-encoder, there are several hyperparameters you can try to tune. However, for the VAE I strongly advise you to: set the latent space dim to 2 use gradient clipping
class VAEEncoder(nn.Module): def __init__(self, dim_input, dim_latent): super().__init__() # TODO def forward(self, inputs): # TODO # mu = mean # log_sigma_squared = log variance # The idea is that you use two different output projection: # one for the mean, one for the log_sigma_squared # but all other layers are shared return mu, log_sigma_squared ## Training lossTo compute the training loss, you must compute two terms: a Monte-Carlo estimation of the reconstruction loss the KL divergence between the distributions computed by the encoder and the prior
For the reconstruction loss, you can use the mean square error loss. To sample values, you can use the reparameterization trick as follows: e = torch.normal(0, 1., mu.shape) z = mu + e * torch.sqrt(torch.exp(log_sigma_squared)) The formula of the KL divergence with the prior is as follows: -0.5 * torch.sum(1 + log_sigma_squared - mu.pow(2) - log_sigma_squared.exp()) see Appendix B of the original paper.
## Generating new imagese = torch.normal(0, 1., (10, 2)) images = decoder(e) for i in range(10): picture = images[i].clone().detach().numpy() plt.imshow(picture.reshape(28,28), cmap='Greys') plt.show() ## Latent space visualizationIt is quite useful to visualize the latent space the variational auto-encoder. You can visualize it either for the training data or the dev data. Note that if you want to visualize a latent space when its dimension is greater than two (useful for the first part!), you could project it in 2 dimensions using PCA (its already implemented in scikit-learn!) decoder.eval() # this code assume that: # data is a tensor of shape (number of images, input dim) # and datalabels a tensor of shape (number of images) # adapt it to your code! # tensor that will contain all latent points points = np.empty((data.shape[0], dim_latent)) with torch.no_grad(): for i in range(0, data.shape[0], batch_dim): batch = train_data_pixels[i:i+batch_dim] # auto-encoder mu = encoder(batch) # VAE, keep only the mean #mu, _ = encoder(batch) points[i:i+batch_dim] = mu.numpy() plt.scatter( points[:,0], points[:, 1], # colormap is between 0 and 1, and we have 10 classes # so we just divide by 10 :) # https://matplotlib.org/3.1.1/tutorials/colors/colormaps.html c=matplotlib.cm.get_cmap("tab10")(data_labels / 10.) ) plt.show() |