Advertisement

Artistic Style Transfer for Videos

  • Manuel RuderEmail author
  • Alexey Dosovitskiy
  • Thomas Brox
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9796)

Abstract

In the past, manually re-drawing an image in a certain artistic style required a professional artist and a long time. Doing this for a video sequence single-handed was beyond imagination. Nowadays computers provide new possibilities. We present an approach that transfers the style from one image (for example, a painting) to a whole video sequence. We make use of recent advances in style transfer in still images and propose new initializations and loss functions applicable to videos. This allows us to generate consistent and stable stylized video sequences, even in cases with large motion and strong occlusion. We show that the proposed method clearly outperforms simpler baselines both qualitatively and quantitatively.

Keywords

Loss Function Optical Flow Temporal Constraint Deep Neural Network Brush Stroke 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

419026_1_En_3_MOESM1_ESM.pdf (5.1 mb)
Supplementary material 1 (pdf 5190 KB)

References

  1. 1.
    Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part VI. LNCS, vol. 7577, pp. 611–625. Springer, Heidelberg (2012)Google Scholar
  2. 2.
    Collobert, R., Kavukcuoglu, K., Farabet, C.: Torch7: A matlab-like environment for machine learning. In: BigLearn, NIPS Workshop (2011)Google Scholar
  3. 3.
    Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style (2015). CoRR abs/1508.06576. http://arxiv.org/abs/1508.06576
  4. 4.
    Hays, J., Essa, I.: Image and video based painterly animation. In:Proceedings of the 3rd International Symposium on Non-photorealistic Animation and Rendering, NPAR 2004, pp. 113–120. ACM, New York, NY, USA (2004). http://doi.acm.org/10.1145/987657.987676
  5. 5.
    Li, C., Wand, M.: Combining Markov random fields and convolutional neural networks for image synthesis (2016). CoRR abs/1601.04589. http://arxiv.org/abs/1601.04589
  6. 6.
    Litwinowicz, P.: Processing images and video for an impressionist effect. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1997, pp. 407–414. ACMPress/Addison-Wesley Publishing Co., New York, NY, USA (1997). http://dx.doi.org/10.1145/258734.258893
  7. 7.
    Nikulin, Y., Novak, R.: Exploring the neural algorithm of artisticstyle (2016). CoRR abs/1602.07188. http://arxiv.org/abs/1602.07188
  8. 8.
    O’Donovan, P., Hertzmann, A.: Anipaint: interactive painterly animation from video. IEEE Trans. Vis. Comput. Graph. 18(3), 475–487 (2012)CrossRefGoogle Scholar
  9. 9.
    Revaud, J., Weinzaepfel, P., Harchaoui, Z., Schmid, C.: EpicFlow: edge-preserving interpolation of correspondences for optical flow. In: CVPR2015 - IEEE Conference on Computer Vision & Pattern Recognition, Boston, United States, June 2015. https://hal.inria.fr/hal-01142656
  10. 10.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014). CoRR abs/1409.1556. http://arxiv.org/abs/1409.1556
  11. 11.
    Sundaram, N., Brox, T., Keutzer, K.: Dense point trajectories by GPU-accelerated large displacement optical flow, September 2010. http://lmb.informatik.uni-freiburg.de//Publications/2010/Bro10e
  12. 12.
    Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C.: DeepFlow: Large displacement optical flow with deep matching. In: ICCV 2013 - IEEE International Conference on Computer Vision, pp. 1385–1392. IEEE, Sydney, Australia, December 2013. https://hal.inria.fr/hal-00873592

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of FreiburgFreiburg im BreisgauGermany

Personalised recommendations