Loving Vincent is a movie very each frame was paintstakingly painted by hand. The film is from 2017, two years before the tool EbSynth entered the stage. Ebsynth lets you stylise keyframes and interpolates the rest of the frames for you. The film would’ve been finished much earlier if it were used, although it would be less impressive or even lazy.
How can we be even lazier? There is a technique called style transfer. The name is quite illuminating. It lets you transfer the style, let’s say that of Vincent van Gogh, to any picture you like. This means you wouldn’t even have to paint each keyframe by hand!
Loving Vincent is about the death of Vincent van Gogh. The last painting ever painted by van Gogh was the painting ‘Boomwortels’ or Tree Roots. Recently the location of the subject of this painting was rediscovered on a postcard. The photo of the tree roots on the postcard was taken from a different perspective. What would happen when we transfer the style of the painting to the postcard? Could this be a scene that Vincent could have painted if he were still alive?
Looks pretty good! Now note that style transfer does not take semantics into account. It does not know what a tree or a guy with a bike is. Guys with bikes should be the same colour as bushes in neural network world. What about something more relevant? Masks are everywhere these days. Naturally, Vincent would also have worn one in our time. I’m not a Photoshop buff, and as stated before, I’m definitely not a painter. So how about style transfer once more?
Not bad. Here I applied style transfer to the Photoshopped image using the style of the original. You can unfortunately still see my bad Photoshopping, especially on the upper right part of the mask. Luckily there’s a paper that blends the image with its surrounding context. It does require you to make mattes of the photoshopped part and define a feathering area. Realistic results nonetheless, although I like the colour of the simple style transfer version better.
The astute may have noticed that the portrait above was made in 1887 while Vincent cut off his ear in 1888. For those that value historical accuracy, I’ve made another style transfer image of a self-portrait of Vincent in 1889.
