Skip to main content
SpringerLink
Log in

Latent motion spaces for full-body motion editing

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

We explore an approach to full-body motion editing with linear motion models, prioritized constraint-based optimization and latent-space interpolation. By exploiting the mathematical connections between linear motion models and prioritized inverse kinematics (PIK), we formulate and solve the motion editing problem as an optimization function whose differential structure is rich enough to efficiently optimize user-specified constraints within the latent motion space. Performing motion editing within latent motion spaces has the advantage of handling pose transitions and consequently motion flow by construction from single key-frame editing. To handle motion adjustments from multiple key-frame and trajectory constraints, we developed a latent-space interpolation technique by exploiting spline functions. Such an approach handles per-frame adjustments generating smooth animations, while avoiding the computational expense of joint space interpolations. We demonstrate the usefulness of this approach by editing and generating full-body reaching and walking jump animations in challenging environment scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Algorithm 2
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. Only a single frame is extracted per sample to generate the latent edited pose (see Fig. 4(b) and Eq. 5).

  2. We also may refer to this constraint type as key-frame constraints.

  3. We also may refer to this constraint type as key-trajectory constraints.

  4. By feasible motions we mean solutions near the training data [14].

References

  1. Arikan, O., Forsyth, D.: Interactive motion generation from examples. In: SIGGRAPH’02: Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, pp. 483–490. ACM, New York (2002). doi:http://doi.acm.org/10.1145/566570.566606

    Chapter  Google Scholar 

  2. Baerlocher, P., Boulic, R.: An inverse kinematic architecture enforcing an arbitrary number of strict priority levels. Vis. Comput. 20(6) (2004)

  3. Callennec, B.L., Boulic, R.: Interactive motion deformation with prioritized constraints. Graph. Models 68, 175–193 (2006). Special Issue on SCA 2004

    Article  Google Scholar 

  4. Carvalho, S.R.: Data-driven constraint-based motion editing. Ph.D. thesis, École Polytechnique Fédéral de Lausanne (EPFL)—IC School of Computer and Communication Sciences, Lausanne (2009). doi:10.5075/epfl-thesis-4558. URL http://library.epfl.ch/theses/?nr=4558

  5. Carvalho, S., Boulic, R., Thalmann, D.: Interactive low-dimensional human motion synthesis by combining motion models and pik. Comput. Animat. Virtual Worlds 18 (2007). Special Issue of Computer Animation and Social Agents (CASA2007)

  6. Carvalho, S., Boulic, R., Thalmann, D.: Motion pattern preserving ik operating in the motion principal coefficients space. In: Proceedings of 15-th WSCG, pp. 97–104 (2007)

    Google Scholar 

  7. Carvalho, S., Boulic, R., Thalmann, D.: Motion pattern encapsulation for data-driven constraint-based motion editing. In: Egges, A., Geraerts, R., Overmars, M. (eds.) Motion in Games. Lecture Notes in Computer Science, vol. 5884, pp. 116–127. Springer, Berlin (2009)

    Chapter  Google Scholar 

  8. Carvalho, S.R., Vidal, C.A., Boulic, R., Talmann, D.: Propagating latent edited poses across eigen-motions. In: Proceedings of the Computer Graphics International, Ottawa, Canada (2011)

    Google Scholar 

  9. Chai, J., Hodgins, J.: Constraint-based motion optimization using a statistical dynamic model. ACM Trans. Graph. 26(3), 8 (2007). doi:http://doi.acm.org/10.1145/1276377.1276387

    Article  Google Scholar 

  10. Choi, K., Ko, H.: On-line motion retargetting. J. Vis. Comput. Animat. 11, 223–235 (2000)

    Article  MATH  Google Scholar 

  11. Glardon, P., Boulic, R., Thalmann, D.: Robust on-line adaptive footplant detection and enforcement for locomotion. Vis. Comput. 22(3), 194–209 (2006). doi:10.1007/s00371-006-0376-9

    Article  Google Scholar 

  12. Gleicher, M.: Comparing constraint-based motion editing methods. Graph. Models 63(2), 107–134 (2001)

    Article  MATH  Google Scholar 

  13. Grassia, F.S.: Practical parameterization of rotations using the exponential map. J. Graph. Tools 3(3), 29–48 (1998)

    Article  Google Scholar 

  14. Grochow, K., Martin, S.L., Hertzmann, A., Popovi, Z.: Style-based inverse kinematics. ACM Trans. Graph. 23(3), 522–531 (2004). doi:http://doi.acm.org/10.1145/1015706.1015755

    Article  Google Scholar 

  15. Hanafusa, H., Yoshikawa, T., Nakamura, Y.: Analysis and control of articulated robot with redundancy. In: IFAC, 8th Triennal World Congress, vol. 4, pp. 1927–1932 (1981)

    Google Scholar 

  16. Ikemoto, L., Arikan, O., Forsyth, D.: Generalizing motion edits with gaussian processes. ACM Trans. Graph. 28(1), 1–12 (2009). doi:http://doi.acm.org/10.1145/1477926.1477927

    Article  Google Scholar 

  17. Jain, A.K., Murty, M.N., Flynn, P.J.: Data clustering: a review. ACM Comput. Surv. 31(3), 264–323 (1999). doi:http://doi.acm.org/10.1145/331499.331504

    Article  Google Scholar 

  18. Jolliffe, I.T.: Principal Component Analysis. Springer, Berlin (1986)

    Book  Google Scholar 

  19. Kochanek, D.H.U., Bartels, R.H.: Interpolating splines with local tension, continuity, and bias control. SIGGRAPH Comput. Graph. 18(3), 33–41 (1984). doi:http://doi.acm.org/10.1145/964965.808575

    Article  Google Scholar 

  20. Kovar, L., Gleicher, M., Pighin, F.: Motion graphs. ACM Trans. Graph. 21(3), 473–482 (2002). doi:http://doi.acm.org/10.1145/566654.566605

    Article  Google Scholar 

  21. Krüger, B., Tautges, J., Weber, A., Zinke, A.: Fast local and global similarity searches in large motion capture databases. In: Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA’10, pp. 1–10. Eurographics Association, Geneva (2010)

    Google Scholar 

  22. Kulpa, R., Multon, F., Arnaldi, B.: Morphology-independent representation of motions for interactive human-like animation. In: EUROGRAPHICS, vol. 24, pp. 343–352 (2005)

    Google Scholar 

  23. Lee, J., Shin, S.: A hierarchical approach to interactive motion editing for human-like figures. In: Proceedings of ACM SIGGRAPH, pp. 39–48 (1999)

    Google Scholar 

  24. Maciejewski, A.: Dealing with the ill-conditioned equations of motion forarticulated figures. In: Computer Graphics and Applications, vol. 10, pp. 63–71. IEEE Press, New York (1990)

    Google Scholar 

  25. Monzani, J.S., Baerlocher, P., Boulic, R., Thalmann, D.: Using an intermediate skeleton and inverse kinematics for motion retargeting. Comput. Graph. Forum 19(3), 11–19 (2000). doi:10.1111/1467-8659.00393

    Article  Google Scholar 

  26. Mukai, T., Kuriyama, S.: Geostatistical motion interpolation. ACM Trans. Graph. 24(3), 1062–1070 (2005). doi:http://doi.acm.org/10.1145/1073204.1073313

    Article  Google Scholar 

  27. Rasmussen, C.E., Williams, K.I.C.: Gaussian Processes for Machine Learning. Adaptive Computation and Machine Learning. MIT Press, Cambridge (2005)

    Google Scholar 

  28. Raunhardt, D., Boulic, R.: Motion constraint. Vis. Comput. 25(5–7), 509–518 (2009). doi:10.1007/s00371-009-0336-2

    Article  Google Scholar 

  29. Safonova, A., Hodgins, J.K., Pollard, N.S.: Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces. ACM Trans. Graph. 23(3), 514–521 (2004). doi:http://doi.acm.org/10.1145/1015706.1015754

    Article  Google Scholar 

  30. Shin, H., Lee, J.: Motion synthesis and editing in low-dimensional spaces. Comput. Animat. Virtual Worlds 17(3–4), 219–227 (2006). doi:10.1002/cav.v17:3/4

    Article  Google Scholar 

  31. Shoemake, K.: Animating rotation with quaternion curves. In: SIGGRAPH’85, pp. 245–254. ACM Press, New York (1985). doi:http://doi.acm.org/10.1145/325334.325242

    Google Scholar 

  32. Urtasun, R., Fua, P.: 3d human body tracking using deterministic temporal motion models. In: European Conference on Computer Vision, Prague, Czech Republic (2004)

    Google Scholar 

  33. VAL: Visual Agent Laboratory (VAL), Department of Information and Computer Sciences, Toyohashi University of Technology. http://www.val.ics.tut.ac.jp/project/geostat/ (2009)

  34. van Basten, B., Egges, A.: Flexible splicing of upper-body motion spaces on locomotion. Comput. Graph. Forum 30(7), 1963–1971 (2011)

    Article  Google Scholar 

  35. Whitney, D.E.: Resolved motion rate control of manipulators and human prostheses. In: IEEE Trans. Man-Mach. Syst, vol. 10, pp. 47–53 (1969)

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank Mireille Clavien for the video production; Autodesk/Maya for their donation of Maya software; Benoît Le Callennec for providing access to his motion editing system (with the support of the SNF grant n o 200020-109989); and the valuable suggestions of all the anonymous reviewers. This work was supported by the EPFL—Sport and Rehabilitation Engineering program. The third author would like to acknowledge CAPES/Brazil for the grant 4557/06-9 that helped support him in VRlab-EPFL Switzerland during the academic year 2007–2008.

Author information

Authors and Affiliations

  1. Laboratory for Computational Sensorimotor Neuroscience, CNS, Botterell Hall, 18, Stuart Street, K7L 3N6, Kingston, ON, Canada

    Schubert R. Carvalho, Ronan Boulic, Creto A. Vidal & Daniel Thalmann

Authors
  1. Schubert R. Carvalho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ronan Boulic
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Creto A. Vidal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniel Thalmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Schubert R. Carvalho.

Electronic Supplementary Material

Below are the links to the electronic supplementary material.

(MP4 13.2 MB)

(MP4 13.0 MB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Carvalho, S.R., Boulic, R., Vidal, C.A. et al. Latent motion spaces for full-body motion editing. Vis Comput 29, 171–188 (2013). https://doi.org/10.1007/s00371-012-0678-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-012-0678-z

Keywords

Search

Navigation