Advertisement

DeepStyleCam: A Real-Time Style Transfer App on iOS

  • Ryosuke Tanno
  • Shin Matsuo
  • Wataru Shimoda
  • Keiji YanaiEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10133)

Abstract

In this demo, we present a very fast CNN-based style transfer system running on normal iPhones. The proposed app can transfer multiple pre-trained styles to the video stream captured from the built-in camera of an iPhone around 140ms (7fps). We extended the network proposed as a real-time neural style transfer network by Johnson et al. [1] so that the network can learn multiple styles at the same time. In addition, we modified the CNN network so that the amount of computation is reduced one tenth compared to the original network. The very fast mobile implementation of the app are based on our paper [2] which describes several new ideas to implement CNN on mobile devices efficiently. Figure 1 shows an example usage of DeepStyleCam which is running on an iPhone SE.

References

  1. 1.
    Johnson, J., Alahi, A., Fei, L.F.: Perceptual losses for real-time style transfer and super-resolution. In: Proceedings of European Conference on Computer Vision (2016)Google Scholar
  2. 2.
    Yanai, K., Tanno, R., Okamoto, K.: Efficient mobile implementation of a CNN-based object recognition system. In: Proceedings of ACM Multimedia (2016)Google Scholar
  3. 3.
    Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style. arXiv:1508.06576 (2015)
  4. 4.
    Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of IEEE Computer Vision and Pattern Recognition (2016)Google Scholar
  5. 5.
    Dong, C., Loy, C.C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 184–199. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-10593-2_13 Google Scholar
  6. 6.
    Iizuka, S., Simo-Serra, E., Ishikawa, H.: Let there be color!: joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification. ACM Trans. Graph. 35(4), 110 (2016). (Proceedings of SIGGRAPH 2016)CrossRefGoogle Scholar
  7. 7.
    Gatys, L.A., Bethge, M., Hertzmann, A., Shechtman, E.: Preserving color in neural artistic style transfer. arXiv:1606.05897 (2016)

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Ryosuke Tanno
    • 1
  • Shin Matsuo
    • 1
  • Wataru Shimoda
    • 1
  • Keiji Yanai
    • 1
    Email author
  1. 1.Department of InformaticsThe University of Electro-CommunicationsChofu-shiJapan

Personalised recommendations