README.md

Super resolution in Tensorflow

Super resolution of an image using Efficient subpixel Phase Shift CNNs.

• DCGAN
  • The final architecture is a modified version of the traditional DCGAN, with 3 1x1 convolutional layers (leaky ReLU activation) followed by a phase shift deconvolution with ratio 2, followed by 2 1x1 convolutional layers, another phase shift deconvolution with ratio 2, and finally 2 more 1x1 convolutional layers. The final output is passed through a tanh to force it in the [0,1] range.
  • Multiple architectures and their checkpoints have been tried in the past, including the vanilla DCGAN architecture, vanilla architecture with an extra 1x1 convolutional layer post the phase shift, extra up-sampling followed by maxpooling, etc. To obtain their checkpoints, kindly refer to older commits
  • Since the training is performed on 32 x 32 images, to perform super resolution on an image of different dimensions, the image is broken down into a grid with cells of dimension 32x32 (zero padding done on dimensions to make them a multiple of 32)
• VAE Work in progress
• SegNet Work in progress

Results

Training

Input	Output	Ground Truth

Testing

Input	Output

Requirements

• Tensorflow 1.0+
• Python 3.5+
• Scipy, Numpy

Usage

1. Training: (assuming the celebA dataset (split into train, test and valid) are located in the folder data in the repository folder)

   python main.py --dataset celebA --is_train True --is_crop True

2. Testing (suppose you want to test on abc.png)

   python main.py --test_image abc.png

To-Do

• Rather than non overlapping sections of 32x32, use overlapping sections and take the center part alone
• Stop storing the partial outputs in the RAM - limits the size of the picture that can be tested upon
• Train model for different resolutions
• Add ability to do it to a subsection of the image that the user defines

References

1. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. By Shi et. al.
2. DCGAN
3. Phase Shift implementation
4. Visualizing and Understanding Convolutional Networks. By Zeiler and Fergus.
5. A guide to convolution arithmetic for deep learning. By Dumoulin and Visin.