This is a Deep Learning project that implements Neural Style Transfer (transferring a given style to an image) from the paper A Neural Algorithm of Artistic Style by Leon A. Gatys, Alexander S. Ecker, Matthias Bethge. The pre-trained VGG-19 Model is used in this project to extract and work on the deep level features of input and style images.
The project has two Jupyter notebooks :
-
neural_single_style_transfer.ipynb : This transfers the style of one image to a given image.
-
neural_multiple_style_transfer.ipynb : This transfers the styles of multiple images to a given image. The notebook is currently configured to transfer styles of two images to a given image but can be conveniently modified to transfer more than two styles to a given input image.
Specify the path for content and style images (for single style transfer) :
# specify paths to content and style images
CONTENT_IMG_PATH = "images/emma.jpg"
STYLE_IMG_PATH = "images/style4.jpg"
Specify the path for content image and two style images (for multi style transfer) :
# specify paths to content and style images
CONTENT_IMG_PATH = "dancing girl.jpg"
STYLE_IMG1_PATH = "red-canna.jpg"
STYLE_IMG2_PATH = "seated-nude.jpg"
Set hyperparameters ALPHA and BETA accordingly (for single style transfer):
# content and style weights for computing losses
ALPHA = 10.0
BETA = 3000.0
Set hyperparameters ALPHA, BETA1, BETA2 accordingly (for multi style transfer):
# content and style weights for computing losses
ALPHA = 10.0
BETA1 = 1500.0
BETA2 = 1500.0
-
The notebooks were run on GPU-accelerated run-time of Google Colab where each run of about 2000 iterations used to take nearly 10 minutes. On a CPU, it could take about 3-4 hours or more depending on the clock speed. So, it is preferable to run these notebooks on a GPU.
-
The process of Style Transfer requires a lot of Hyperparameter Tuning. Each style and input image requires a different set of hyperparameters that may or may not work on a different style-image pair.
Tip : Start with APLHA = 10 and BETA = 5000. Check after 500 iterations. If the outcome is not desirable, change accordingly. If the content of input image is too much distorted, try increasing the value of ALPHA. If the image is not attaining the desired style, increase BETA and if the input is loosing its original content a lot, decrease the value of BETA. If nothing works, change ALPHA-BETA combinations altogether such as ALPHA = 5 and BETA = 100 and see whatever works best for the given combination of input and style image.
For Multi-Style Transfer, same guidelines apply along with an additional step. Instead of just single BETA value in case of Single Style Transfer, now two values need to be considered - BETA1 and BETA2. BETA1 and BETA2 needs to be modified depending on whether the final image should have more style of first style image ( BETA1 > BETA2), more style of second style image ( BETA1 < BETA2) or should have equal styles of both style images ( BETA1 = BETA2 ).
Below are some examples of the images on which Neural Style Transfer was applied and the corresponding ALPHA-BETA combinations are also given along with the effect they have on a given input image.
Following are the input images used in the given project :
Following are the style images used in the given project :
Following are the images styled using Neural Style Transfer technique :
ALPHA - 10 and BETA - 100
+ = 
ALPHA - 10 and BETA - 5000
+ = 
ALPHA - 10 and BETA - 5000
+ = 
ALPHA - 10 and BETA - 5000
+ = 
ALPHA - 10 and BETA - 5000
+ = 
ALPHA - 10 and BETA - 20
+ = 
ALPHA - 10 and BETA - 250
+ = 
ALPHA - 10 and BETA - 500
+ = 
ALPHA - 10 and BETA - 6000
+ = 
+ =
ALPHA - 10 and BETA - 3000
ALPHA - 10 and BETA - 2500
ALPHA - 10 and BETA - 1000
As can be seen in these images, when we decrease the value of BETA, the overpowering effect of Style image is being reduced and generated image is looking more pleasant.
+ + =
ALPHA - 10 BETA1 - 1500 BETA2 - 1500
ALPHA - 10 BETA1 - 2200 BETA2 - 800
As is apparent from above two images when BETA1 is increased and BETA2 is decreased for the second image, style of first image increases considerably while the style of second image somewhat fades in.
ALPHA - 10 BETA1 - 1500 BETA2 - 15000
+ + =
ALPHA - 10 BETA1 - 50 BETA2 - 100
+ + =
+ + =
ALPHA - 10 BETA1 - 2500 BETA2 - 1750
ALPHA - 10 BETA1 - 3500 BETA2 - 1750
ALPHA - 10 BETA1 - 5000 BETA2 - 1500
As can be seen in these images, when we increase the value of BETA1, the effect of First Style image increases on the final input image while the effect of second style image decreases.