Deep Learning

Description

Labs of the Deep Learning course of the MSc in Artificial Intelligence at the University of Amsterdam.

Lab 1

Problem statement - Solution

Goal of this assignment is to learn how to implement and train basic neural architectures like MLPs and CNNs for classification tasks. The assignment consists of four parts:

• First, to gain an in-depth understanding we focused on a basic implementation of a MLP in NumPy. This requires to understand backpropagation in detail and to derive the necessary equations first.
• Next, we implemented MLP in PyTorch and add tuned its performance by adding additional layers provided by the
• In order to learn how to implement custom operations in PyTorch, in next task we reimplemented a batch-normalization layer.
• Last part aims at implementing a simple CNN in PyTorch.

Lab 2

Problem statement - Solution

Goal of this assignment is to study and implement recurrent neural networks (RNNs). The assignment consists of three parts:

• Getting familiar with vanilla RNNs and LSTMs on a simple toy problem. This allows for understanding the fundamentals of recurrent networks.
• Studying a more advanced LSTM cell to emphasize the wide variety in LSTM cells.
• Using LSTMs for learning and generating text.

Samples during training on Vanity Fair

(The bold characters are picked randomly and used the initialize the LSTM state.)

Train Step	Sampled Text
0	**#**jx!%dYikK.vXonWIc(Qw"@_AyeOwX
17600	ved and discovered herse. His
35200	Unce\nin him home and going, th
52800	\'s long money," reC.B.n that t
70400	y must have dreads everything

Lab 3

Problem statement - Solution

Goal of this assignment is to study and implement Deep Generative Models. The assignment focuses on two most popular models:

• Variational Auto Encoders (VAE)
• Generative Adversarial Networks (GAN)

We have implemented both of them in PyTorch as part of this assignment.

Figure: Data manifold learned by VAE after 40 iterations

Figure: Samples for DCGAN at steps 1, 5, 20 and 40 (left to right, top to bottom)

Dependencies

• NumPy
• PyTorch
• Matplotlib
• Scipy

Copyright

Copyright © 2018 Andrii Skliar.

This project is distributed under the MIT license. This was developed as part of the Deep Learning course taught by Efstratios Gavves at the University of Amsterdam. Please follow the UvA regulations governing Fraud and Plagiarism in case you are a student.