Multimedia Tools and Applications

, Volume 78, Issue 1, pp 799–816 | Cite as

Web-based SBLR method of multimedia tools for computer-aided drawing

  • Ning Xie
  • Tingting Zhao
  • Yang Yang
  • Heng Tao Shen


As one of the most successful multimedia tools for digital media and creative industry, computer-aided drawing system assists users to convert the input real photos into painterly style images. Nowadays, it is widely developed as cloud brush engine service in many creative software tools and applications of artistic rendering such as Prisma, Photoshop Cloud, and Meitu, because the machine learning server has more powerful than the stand-alone version. In this paper, we propose a web collaborative Stroke-based Learning and Rendering (WebSBLR) system. Different from the existing methods that are mainly focused on the artistic filters, we concentrate on the stroke realistic rendering engine for browser on client using WebGL and HTML5. Moreover, we implement the learning-based stroke drawing path generation module on the server. By this way, we enable to achieve the computer-supported cooperative work (CSCW), especially for multi-screen synchronous interaction. The experiments demonstrated our method are efficient to web-based multi-screen painting simulation.It can successfully learn artists’ styles and render pictures with consistent and smooth brush strokes.


Multimedia tools CSCW SBR Artistic stylization PGPE 



We firstly thank anonymous reviewers for their helpful comments. This work was supported in part by the National Natural Science Foundation of China under Project 61602088, Project 61572108, Project 61632007, and Project 61502339, the National Thousand-Young-Talents Program of China, the Fundamental Research Funds for the Central Universities under Project ZYGX2014Z007, Project ZYGX2015J055 and Project ZYGX2016J212.


  1. 1.
    Abbeel P, Ng AY (2004) Apprenticeship learning via inverse reinforcement learning. In: Machine learning, proceedings of the twenty-first international conference(ICML 2004). Banff, Alberta, Canada, July 4-8 2004Google Scholar
  2. 2.
    Baran I, Lehtinen J, Popovic J (2010) Sketching clothoid splines using shortest paths. Comput Graph Forum 29(2):655–664CrossRefGoogle Scholar
  3. 3.
    Bousseau A, Neyret F, Thollot J, Salesin D (2007) Video watercolorization using bidirectional texture advection. ACM Trans Graph 26(3):104CrossRefGoogle Scholar
  4. 4.
    Chu NS, Tai C (2004) Real-time painting with an expressive virtual chinese brush. IEEE Comput Graph Appl 24(5):76–85CrossRefGoogle Scholar
  5. 5.
    Chu NS, Tai C (2005) Moxi: real-time ink dispersion in absorbent paper. ACM Trans Graph 24(3):504–511CrossRefGoogle Scholar
  6. 6.
    Chu N, Baxter B, Wei LY, Govindaraju NK (2010) Detail-preserving paint modeling for 3D brushes Proceedings of the 8th international symposium on NPAR. ACM, New York, pp 27–34Google Scholar
  7. 7.
    Davies E (2005) Machine vision: theory algorithms, practicalities, 3rd edn. ElsevierGoogle Scholar
  8. 8.
    Eric KJ, Kang H (2011) Image and video abstraction by coherence enhancing filtering. Comput Graph Forum, 593–602Google Scholar
  9. 9.
    Hertzmann A (2003) A survey of stroke-based rendering. IEEE Comput Graph Appl 23:70–81CrossRefGoogle Scholar
  10. 10.
    Hu MQ, Yang Y, Shen F, Xie N, Shen HT (2018) Hashing with angular reconstructive embeddings. IEEE Trans Image Process 27(2):545–555MathSciNetCrossRefGoogle Scholar
  11. 11.
    Huang Z, Han L, Gong GH (2015) A local adaptive catmull-rom to reduce numerical dissipation of semi-lagrangian advection. J Visual Comp Animat 26 (2):141–146Google Scholar
  12. 12.
    Jolliffe IT (2002) Principal component analysis. Springer Verlag, New YorkzbMATHGoogle Scholar
  13. 13.
    Kalogerakis E, Nowrouzezahrai D, Breslav S, Hertzmann A (2012) Learning hatching for pen-and-ink illustration of surfaces. ACM Trans Graph 31(1):1,1–1,17CrossRefGoogle Scholar
  14. 14.
    Kang H, Lee S (2008) Shape-simplifying image abstraction. Comput Graph Forum 27(7):1773–1780CrossRefGoogle Scholar
  15. 15.
    Kyprianidis JE, Kang H, Döllner J (2009) Image and video abstraction by anisotropic kuwahara filtering. Comput Graph Forum 28(7):1955–1963CrossRefGoogle Scholar
  16. 16.
    Kyprianidis JE, Semmo A, Kang H, Döllner J (2010) Anisotropic kuwahara filtering with polynomial weighting functions. In: Theory and practice of computer graphics. Sheffield, United Kingdom, 2010. Proceedings, pp 25–30Google Scholar
  17. 17.
    Lu JW, Yu F, Finkelstein A, DiVerdi S (2012) Helpinghand: example-based stroke stylization. ACM Trans Graph 31(4):46,1–46,10CrossRefGoogle Scholar
  18. 18.
    Lu J, Barnes C, Diverdi S, Finkelstein A (2013) Realbrush: painting with examples of physical media. Acm Trans Graph 32(4):117CrossRefGoogle Scholar
  19. 19.
    Orbay G, Kara LB (2011) Beautification of design sketches using trainable stroke clustering and curve fitting. IEEE Trans Vis Comput Graph 17(5):694–708CrossRefGoogle Scholar
  20. 20.
    Pham TQ, van Vliet LJ (2005) Separable bilateral filtering for fast video preprocessing. In: Proceedings of the 2005 IEEE international conference on multimedia and expo, ICME 2005, July 6-9, 2005. Amsterdam, pp 454–457Google Scholar
  21. 21.
    Sehnke F, Osendorfer C, Ru̇ckstieß T, Graves A, Peters J, Schmidhuber J (2010) Parameter-exploring policy gradients. Neur Netw 23(4):551–559CrossRefGoogle Scholar
  22. 22.
    Semmo A, Trapp M, Dürschmid T, Döllner J, Pasewaldt S (2016) Interactive multi-scale oil paint filtering on mobile devices. In: SIGGRAPH ’16. Anaheim, CA, USA, July 24-28, 2016, pp 42:1– 42:2Google Scholar
  23. 23.
    Shen F, Xu Y, Liu L, Yang Y, Huang Z, Shen HT (2018) Unsupervised deep hashing with similarity-adaptive and discrete optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)Google Scholar
  24. 24.
    Sýkora D, Buriánek J, ára J (2005) Colorization of black-and-white cartoons. Image Vis Comput 23(9):767–782CrossRefGoogle Scholar
  25. 25.
    Williams RJ (1992) Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learn 8:229–256zbMATHGoogle Scholar
  26. 26.
    Xie N, Laga H, Saito S, Nakajima M (2010) IR2s: interactive real photo to sumi-e. In: Proceedings of the 8th international symposium on non-photorealistic animation and rendering 2010. Annecy, France, June 7-10, 2010, pp 63–71Google Scholar
  27. 27.
    Xie N, Hachiya H, Sugiyama M (2012) Artist agent: a reinforcement learning approach to automatic stroke generation in oriental ink painting. In: Proceedings of the 29th international conference on machine learning, ICML 2012. Edinburgh, Scotland, UK, June 26 - July 1 2012Google Scholar
  28. 28.
    Xu S, Xu Y, Kang SB, Salesin D, Pan Y, Shum H (2006) Animating Chinese paintings through stroke-based decomposition. ACM Trans Graph 25(2):239–267CrossRefGoogle Scholar
  29. 29.
    Zeng K, Zhao M, Xiong C, Zhu SC (2010) From image parsing to painterly rendering. ACM Trans Graph 29(1):2,1–2,11Google Scholar
  30. 30.
    Zhao T, Hachiya H, Niu G, Sugiyama M (2011) Analysis and improvement of policy gradient estimation. Adv Neural Inf Process Syst 24:262–270zbMATHGoogle Scholar
  31. 31.
    Zhao Y, Jin X, Xu Y, Zhao H, Ai M, Zhou K (2015) Parallel style-aware image cloning for artworks. IEEE Trans Visual Comput Graph 21(2):229CrossRefGoogle Scholar
  32. 32.
    Zitnick CL (2013) Handwriting beautification using token means. ACM Trans Graph 32(4):53,1–53,8CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Ning Xie
    • 1
  • Tingting Zhao
    • 1
  • Yang Yang
    • 1
  • Heng Tao Shen
    • 1
  1. 1.Chengdu CityChina

Personalised recommendations