Skip to main content

Interactive Generation of Calligraphic Trajectories from Gaussian Mixtures

  • Chapter
  • First Online:

Part of the book series: Unsupervised and Semi-Supervised Learning ((UNSESUL))

Abstract

The chapter presents an approach for the interactive definition of curves and motion paths based on Gaussian mixture model (GMM) and optimal control. The input of our method is a mixture of multivariate Gaussians defined by the user, whose centers define a sparse sequence of key-points, and whose covariances define the precision required to pass through these key-points. The output is a dynamical system generating curves that are natural looking and reflect the kinematics of a movement, similar to that produced by human drawing or writing. In particular, the stochastic nature of the GMM combined with optimal control is exploited to generate paths with natural variations, which are defined by the user within a simple interactive interface. Several properties of the Gaussian mixture are exploited in this application. First, there is a direct link between multivariate Gaussian distributions and optimal control formulations based on quadratic objective functions (linear quadratic tracking), which is exploited to extend the GMM representation to a controller. We then exploit the option of tying the covariances in the GMM to modulate the style of the calligraphic trajectories. The approach is tested to generate curves and traces that are geometrically and dynamically similar to the ones that can be seen in art forms such as calligraphy or graffiti.

This is a preview of subscription content, log in via an institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    We refer the reader to the chapter by O.E. Parsons in this same book [Chapter 1] for an introduction and in-depth description of GMMs and relevant estimation methods.

References

  1. AlMeraj, Z., Wyvill, B., Isenberg, T., Gooch, A., Guy, R.: Automatically mimicking unique hand-drawn pencil lines. Comput. Graph. 33(4), 496–508 (2009)

    Article  Google Scholar 

  2. Baran, I., Lehtinen, J., Popović, J.: Sketching clothoid splines using shortest paths. In: Computer Graphics Forum, vol. 29, pp. 655–664. Wiley, London (2010)

    Article  Google Scholar 

  3. Berio, D., Calinon, S., Fol Leymarie, F.: Learning dynamic graffiti strokes with a compliant robot. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3981–3986. IEEE, Piscataway (2016)

    Google Scholar 

  4. Berio, D., Calinon, S., Fol Leymarie, F.: Generating calligraphic trajectories with model predictive control. In: Proceedings of the 43rd Conference on Graphics Interface, pp. 132–139. Canadian Human-Computer Communications Society School of Computer Science, Waterloo (2017)

    Google Scholar 

  5. Berio, D., Leymarie, F.F., Plamondon, R.: Expressive curve editing with the sigma lognormal model. In: Diamanti, O., Vaxman, A. (eds.) EG 2018—Short Papers. The Eurographics Association (2018)

    Google Scholar 

  6. Bernstein, N.A., Latash, M.L., Turvey, M.: Dexterity and Its Development. Taylor & Francis, London (1996)

    Google Scholar 

  7. Calinon, S.: A tutorial on task-parameterized movement learning and retrieval. Intell. Serv. Robot. 9(1), 1–29 (2016)

    Article  Google Scholar 

  8. Cooper, M., Chalfant, H.: Subway Art. Rinehart and Winston, Holt (1984)

    Google Scholar 

  9. d’Avella, A., Saltiel, P., Bizzi, E.: Combinations of muscle synergies in the construction of a natural motor behavior. Nat. Neurosci. 6(3), 300–308 (2003)

    Article  Google Scholar 

  10. De Boor, C.: A Practical Guide to Splines, vol. 27. Springer, New York (1978)

    Book  MATH  Google Scholar 

  11. Dooijes, E.: Analysis of handwriting movements. Acta Psychol. 54(1), 99–114 (1983)

    Article  Google Scholar 

  12. Edelman, S., Flash, T.: A model of handwriting. Biol. Cybern. 57(1–2), 25–36 (1987)

    Article  Google Scholar 

  13. Egerstedt, M., Martin, C.: Control Theoretic Splines: Optimal Control, Statistics, and Path Planning. Princeton University Press; Princeton Oxford, Princeton (2009)

    Google Scholar 

  14. Ferrer, M.A., Diaz, M., Carmona-Duarte, C., Morales, A.: A behavioral handwriting model for static and dynamic signature synthesis. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1041–1053 (2017)

    Article  Google Scholar 

  15. Flash, T., Handzel, A.: Affine differential geometry analysis of human arm movements. Biol. Cybern. 96(6), 577–601 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  16. Flash, T., Henis, E.: Arm trajectory modifications during reaching towards visual targets. J. Cogn. Neurosci. 3(3), 220–230 (1991)

    Article  Google Scholar 

  17. Flash, T., Hochner, B.: Motor primitives in vertebrates and invertebrates. Curr. Opin. Neurobiol. 15(6), 660–666 (2005)

    Article  Google Scholar 

  18. Flash, T., Hogan, N.: The coordination of arm movements. J. Neurosci. 5(7), 1688–1703 (1985)

    Article  Google Scholar 

  19. Flash, T., Hogan, N.: Optimization principles in motor control. In: The Handbook of Brain Theory and Neural Networks, pp. 682–685. MIT Press, Cambridge, MA (1998)

    Google Scholar 

  20. Freedberg, D., Gallese, V.: Motion, emotion and empathy in esthetic experience. Trends Cogn. Sci. 11(5), 197–203 (2007)

    Article  Google Scholar 

  21. Freeman, F.: Experimental analysis of the writing movement. Psychol. Monogr. Gen. Appl. 17(4), 1–57 (1914)

    Article  MathSciNet  Google Scholar 

  22. Fujioka, H., Kano, H., Nakata, H., Shinoda, H.: Constructing and reconstructing characters, words, and sentences by synthesizing writing motions. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 36(4), 661–670 (2006)

    Article  Google Scholar 

  23. Haeberli, P.: Dynadraw: A dynamic drawing technique. www.graficaobscura.com/dyna/ (1989)

  24. House, D., Singh, M.: Line drawing as a dynamic process. In: Proceedings of the 15th Pacific Conference on Computer Graphics and Applications, pp. 351–60. IEEE, Piscataway (2007)

    Google Scholar 

  25. Jordan, M., Wolpert, D.: Computational motor control. In: Gazzaniga, M. (ed.) The Cognitive Neurosciences, 2nd edn. MIT Press, Cambridge, MA (1999)

    Google Scholar 

  26. Kyprianidis, J., Collomosse, J., Wang, T., Isenberg, T.: State of the “art”: A taxonomy of artistic stylization techniques for images and video. IEEE Trans. Vis. Comput. Graph. 19(5), 866–885 (2013)

    Article  Google Scholar 

  27. Lacquaniti, F., Terzuolo, C., Viviani, P.: The law relating the kinematic and figural aspects of drawing movements. Acta Psychol. 54(1), 115–130 (1983)

    Article  Google Scholar 

  28. Leder, H., Bär, S., Topolinski, S.: Covert painting simulations influence aesthetic appreciation of artworks. Psychol. Sci. 23(12), 1479–1481 (2012)

    Article  Google Scholar 

  29. Longcamp, M., Anton, J.L., Roth, M., Velay, J.L.: Visual presentation of single letters activates a premotor area involved in writing. NeuroImage 19(4), 1492–1500 (2003)

    Article  Google Scholar 

  30. Lu, J., Yu, F., Finkelstein, A., DiVerdi, S.: Helpinghand: Example-based stroke stylization. ACM Trans. Graph. 31(4), 46 (2012)

    Article  Google Scholar 

  31. McCrae, J., Singh, K.: Sketching piecewise clothoid curves. Comput. Graph. 33(4), 452–461 (2009)

    Article  Google Scholar 

  32. Mediavilla, C., Marshall, A., van Stone, M., Xuriguera, G., Jackson, D.: Calligraphy: from calligraphy to abstract painting. Scirpus (1996)

    Google Scholar 

  33. Meirovitch, Y., Bennequin, D., Flash, T.: Geometrical invariance and smoothness maximization for task-space movement generation. IEEE Trans. Robot. 32(4), 837–853 (2016)

    Article  Google Scholar 

  34. Morasso, P.: Spatial control of arm movements. Exp. Brain Res. 42(2), 223–7 (1981)

    Article  Google Scholar 

  35. Pignocchi, A.: How the intentions of the draftsman shape perception of a drawing. Conscious. Cogn. 19(4), 887–898 (2010)

    Article  Google Scholar 

  36. Plamondon, R.: A kinematic theory of rapid human movements. Part I. Biol. Cybern. 72(4), 295–307 (1995)

    Article  MATH  Google Scholar 

  37. Plamondon, R., O’Reilly, C., Galbally, J., Almaksour, A., Anquetil, É.: Recent developments in the study of rapid human movements with the kinematic theory. Pattern Recogn. Lett. 35, 225–35 (2014)

    Article  Google Scholar 

  38. Shoemake, K.: Arcball: A user interface for specifying three-dimensional orientation using a mouse. In: Graphics Interface, vol. 92, pp. 151–156 (1992)

    Google Scholar 

  39. Tanwani, A., Calinon, S.: Learning robot manipulation tasks with task-parameterized semitied hidden semi-Markov model. IEEE Robot. Autom. Lett. 1(1), 235–242 (2016)

    Article  Google Scholar 

  40. Thiel, Y., Singh, K., Balakrishnan, R.: Elasticurves: Exploiting stroke dynamics and inertia for the real-time neatening of sketched 2D curves. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, pp. 383–392. ACM, New York (2011)

    Google Scholar 

  41. Todorov, E., Jordan, M.: Optimal feedback control as a theory of motor coordination. Nat. Neurosci. 5(11), 1226–1235 (2002)

    Article  Google Scholar 

  42. Todorov, E., Jordan, M.I.: Smoothness maximization along a predefined path accurately predicts the speed profiles of complex arm movements. J. Neurophysiol. 80(2), 696–714 (1998)

    Article  Google Scholar 

  43. Uno, Y., Kawato, M., Suzuki, R.: Formation and control of optimal trajectory in human multijoint arm movement. Biol. Cybern. 61(2), 89–101 (1989)

    Article  Google Scholar 

  44. Viviani, P., Schneider, R.: A developmental study of the relationship between geometry and kinematics in drawing movements. J. Exp. Psychol. Hum. Percept. Perform. 17(1), 198–218 (1991)

    Article  Google Scholar 

  45. Zeestraten, M., Calinon, S., Caldwell, D.G.: Variable duration movement encoding with minimal intervention control. In: Proceedings of the International Conference on Robotics and Automation (ICRA), pp. 497–503. IEEE, Stockholm, Sweden (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Daniel Berio .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Berio, D., Leymarie, F.F., Calinon, S. (2020). Interactive Generation of Calligraphic Trajectories from Gaussian Mixtures. In: Bouguila, N., Fan, W. (eds) Mixture Models and Applications. Unsupervised and Semi-Supervised Learning. Springer, Cham. https://doi.org/10.1007/978-3-030-23876-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-23876-6_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-23875-9

  • Online ISBN: 978-3-030-23876-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics