Our Implementation
The following explains the Style Transfer procedure using the VGG19 CNN
A pre-trained VGG19 CNN is divided into 6 slices and the output feature maps from each slice is taken as the network output. The content image is passed through this custom VGG19 and the 5th feature map is taken as the content target. Then, the style image is also passed through this custom VGG19 and the Gram matrices for each set of feature maps is calculated and treated as the taret gram matrices.
Once the target content feature maps and target style gram matrices have been calculated, we initialize an optimizing image (either same as the content image, or random noise) which will be optimized to acquire the target content and style.
At every step, this optimizing image is passed through the custom CNN and the difference between
its content feature map & style gram matrices and the target content & style is calculated as a weighted loss.
Once the loss is calculated, we backpropagate it through the network to the image pixel values themselves. Then, we change the values of the pixels in the image itself in order to minimize the loss that we have calculated.
This optimization process can be carried out through some standard optimization techniques like Adam or LBFGS. At each iteration of the optimization procedure, we change the values of the pixels in the optimizing image and the proces is repeated for a certain number of epochs or until we reach a certain optimal value for the loss function.
