Abstract
The chapter presents an approach for the interactive definition of curves and motion paths based on Gaussian mixture model (GMM) and optimal control. The input of our method is a mixture of multivariate Gaussians defined by the user, whose centers define a sparse sequence of key-points, and whose covariances define the precision required to pass through these key-points. The output is a dynamical system generating curves that are natural looking and reflect the kinematics of a movement, similar to that produced by human drawing or writing. In particular, the stochastic nature of the GMM combined with optimal control is exploited to generate paths with natural variations, which are defined by the user within a simple interactive interface. Several properties of the Gaussian mixture are exploited in this application. First, there is a direct link between multivariate Gaussian distributions and optimal control formulations based on quadratic objective functions (linear quadratic tracking), which is exploited to extend the GMM representation to a controller. We then exploit the option of tying the covariances in the GMM to modulate the style of the calligraphic trajectories. The approach is tested to generate curves and traces that are geometrically and dynamically similar to the ones that can be seen in art forms such as calligraphy or graffiti.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
We refer the reader to the chapter by O.E. Parsons in this same book [Chapter 1] for an introduction and in-depth description of GMMs and relevant estimation methods.
References
AlMeraj, Z., Wyvill, B., Isenberg, T., Gooch, A., Guy, R.: Automatically mimicking unique hand-drawn pencil lines. Comput. Graph. 33(4), 496–508 (2009)
Baran, I., Lehtinen, J., Popović, J.: Sketching clothoid splines using shortest paths. In: Computer Graphics Forum, vol. 29, pp. 655–664. Wiley, London (2010)
Berio, D., Calinon, S., Fol Leymarie, F.: Learning dynamic graffiti strokes with a compliant robot. In: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3981–3986. IEEE, Piscataway (2016)
Berio, D., Calinon, S., Fol Leymarie, F.: Generating calligraphic trajectories with model predictive control. In: Proceedings of the 43rd Conference on Graphics Interface, pp. 132–139. Canadian Human-Computer Communications Society School of Computer Science, Waterloo (2017)
Berio, D., Leymarie, F.F., Plamondon, R.: Expressive curve editing with the sigma lognormal model. In: Diamanti, O., Vaxman, A. (eds.) EG 2018—Short Papers. The Eurographics Association (2018)
Bernstein, N.A., Latash, M.L., Turvey, M.: Dexterity and Its Development. Taylor & Francis, London (1996)
Calinon, S.: A tutorial on task-parameterized movement learning and retrieval. Intell. Serv. Robot. 9(1), 1–29 (2016)
Cooper, M., Chalfant, H.: Subway Art. Rinehart and Winston, Holt (1984)
d’Avella, A., Saltiel, P., Bizzi, E.: Combinations of muscle synergies in the construction of a natural motor behavior. Nat. Neurosci. 6(3), 300–308 (2003)
De Boor, C.: A Practical Guide to Splines, vol. 27. Springer, New York (1978)
Dooijes, E.: Analysis of handwriting movements. Acta Psychol. 54(1), 99–114 (1983)
Edelman, S., Flash, T.: A model of handwriting. Biol. Cybern. 57(1–2), 25–36 (1987)
Egerstedt, M., Martin, C.: Control Theoretic Splines: Optimal Control, Statistics, and Path Planning. Princeton University Press; Princeton Oxford, Princeton (2009)
Ferrer, M.A., Diaz, M., Carmona-Duarte, C., Morales, A.: A behavioral handwriting model for static and dynamic signature synthesis. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1041–1053 (2017)
Flash, T., Handzel, A.: Affine differential geometry analysis of human arm movements. Biol. Cybern. 96(6), 577–601 (2007)
Flash, T., Henis, E.: Arm trajectory modifications during reaching towards visual targets. J. Cogn. Neurosci. 3(3), 220–230 (1991)
Flash, T., Hochner, B.: Motor primitives in vertebrates and invertebrates. Curr. Opin. Neurobiol. 15(6), 660–666 (2005)
Flash, T., Hogan, N.: The coordination of arm movements. J. Neurosci. 5(7), 1688–1703 (1985)
Flash, T., Hogan, N.: Optimization principles in motor control. In: The Handbook of Brain Theory and Neural Networks, pp. 682–685. MIT Press, Cambridge, MA (1998)
Freedberg, D., Gallese, V.: Motion, emotion and empathy in esthetic experience. Trends Cogn. Sci. 11(5), 197–203 (2007)
Freeman, F.: Experimental analysis of the writing movement. Psychol. Monogr. Gen. Appl. 17(4), 1–57 (1914)
Fujioka, H., Kano, H., Nakata, H., Shinoda, H.: Constructing and reconstructing characters, words, and sentences by synthesizing writing motions. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 36(4), 661–670 (2006)
Haeberli, P.: Dynadraw: A dynamic drawing technique. www.graficaobscura.com/dyna/ (1989)
House, D., Singh, M.: Line drawing as a dynamic process. In: Proceedings of the 15th Pacific Conference on Computer Graphics and Applications, pp. 351–60. IEEE, Piscataway (2007)
Jordan, M., Wolpert, D.: Computational motor control. In: Gazzaniga, M. (ed.) The Cognitive Neurosciences, 2nd edn. MIT Press, Cambridge, MA (1999)
Kyprianidis, J., Collomosse, J., Wang, T., Isenberg, T.: State of the “art”: A taxonomy of artistic stylization techniques for images and video. IEEE Trans. Vis. Comput. Graph. 19(5), 866–885 (2013)
Lacquaniti, F., Terzuolo, C., Viviani, P.: The law relating the kinematic and figural aspects of drawing movements. Acta Psychol. 54(1), 115–130 (1983)
Leder, H., Bär, S., Topolinski, S.: Covert painting simulations influence aesthetic appreciation of artworks. Psychol. Sci. 23(12), 1479–1481 (2012)
Longcamp, M., Anton, J.L., Roth, M., Velay, J.L.: Visual presentation of single letters activates a premotor area involved in writing. NeuroImage 19(4), 1492–1500 (2003)
Lu, J., Yu, F., Finkelstein, A., DiVerdi, S.: Helpinghand: Example-based stroke stylization. ACM Trans. Graph. 31(4), 46 (2012)
McCrae, J., Singh, K.: Sketching piecewise clothoid curves. Comput. Graph. 33(4), 452–461 (2009)
Mediavilla, C., Marshall, A., van Stone, M., Xuriguera, G., Jackson, D.: Calligraphy: from calligraphy to abstract painting. Scirpus (1996)
Meirovitch, Y., Bennequin, D., Flash, T.: Geometrical invariance and smoothness maximization for task-space movement generation. IEEE Trans. Robot. 32(4), 837–853 (2016)
Morasso, P.: Spatial control of arm movements. Exp. Brain Res. 42(2), 223–7 (1981)
Pignocchi, A.: How the intentions of the draftsman shape perception of a drawing. Conscious. Cogn. 19(4), 887–898 (2010)
Plamondon, R.: A kinematic theory of rapid human movements. Part I. Biol. Cybern. 72(4), 295–307 (1995)
Plamondon, R., O’Reilly, C., Galbally, J., Almaksour, A., Anquetil, É.: Recent developments in the study of rapid human movements with the kinematic theory. Pattern Recogn. Lett. 35, 225–35 (2014)
Shoemake, K.: Arcball: A user interface for specifying three-dimensional orientation using a mouse. In: Graphics Interface, vol. 92, pp. 151–156 (1992)
Tanwani, A., Calinon, S.: Learning robot manipulation tasks with task-parameterized semitied hidden semi-Markov model. IEEE Robot. Autom. Lett. 1(1), 235–242 (2016)
Thiel, Y., Singh, K., Balakrishnan, R.: Elasticurves: Exploiting stroke dynamics and inertia for the real-time neatening of sketched 2D curves. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, pp. 383–392. ACM, New York (2011)
Todorov, E., Jordan, M.: Optimal feedback control as a theory of motor coordination. Nat. Neurosci. 5(11), 1226–1235 (2002)
Todorov, E., Jordan, M.I.: Smoothness maximization along a predefined path accurately predicts the speed profiles of complex arm movements. J. Neurophysiol. 80(2), 696–714 (1998)
Uno, Y., Kawato, M., Suzuki, R.: Formation and control of optimal trajectory in human multijoint arm movement. Biol. Cybern. 61(2), 89–101 (1989)
Viviani, P., Schneider, R.: A developmental study of the relationship between geometry and kinematics in drawing movements. J. Exp. Psychol. Hum. Percept. Perform. 17(1), 198–218 (1991)
Zeestraten, M., Calinon, S., Caldwell, D.G.: Variable duration movement encoding with minimal intervention control. In: Proceedings of the International Conference on Robotics and Automation (ICRA), pp. 497–503. IEEE, Stockholm, Sweden (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Berio, D., Leymarie, F.F., Calinon, S. (2020). Interactive Generation of Calligraphic Trajectories from Gaussian Mixtures. In: Bouguila, N., Fan, W. (eds) Mixture Models and Applications. Unsupervised and Semi-Supervised Learning. Springer, Cham. https://doi.org/10.1007/978-3-030-23876-6_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-23876-6_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-23875-9
Online ISBN: 978-3-030-23876-6
eBook Packages: EngineeringEngineering (R0)