Advertisement

Latent motion spaces for full-body motion editing

Abstract

We explore an approach to full-body motion editing with linear motion models, prioritized constraint-based optimization and latent-space interpolation. By exploiting the mathematical connections between linear motion models and prioritized inverse kinematics (PIK), we formulate and solve the motion editing problem as an optimization function whose differential structure is rich enough to efficiently optimize user-specified constraints within the latent motion space. Performing motion editing within latent motion spaces has the advantage of handling pose transitions and consequently motion flow by construction from single key-frame editing. To handle motion adjustments from multiple key-frame and trajectory constraints, we developed a latent-space interpolation technique by exploiting spline functions. Such an approach handles per-frame adjustments generating smooth animations, while avoiding the computational expense of joint space interpolations. We demonstrate the usefulness of this approach by editing and generating full-body reaching and walking jump animations in challenging environment scenarios.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 199

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Algorithm 2
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Notes

  1. 1.

    Only a single frame is extracted per sample to generate the latent edited pose (see Fig. 4(b) and Eq. 5).

  2. 2.

    We also may refer to this constraint type as key-frame constraints.

  3. 3.

    We also may refer to this constraint type as key-trajectory constraints.

  4. 4.

    By feasible motions we mean solutions near the training data [14].

References

  1. 1.

    Arikan, O., Forsyth, D.: Interactive motion generation from examples. In: SIGGRAPH’02: Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, pp. 483–490. ACM, New York (2002). doi:http://doi.acm.org/10.1145/566570.566606

  2. 2.

    Baerlocher, P., Boulic, R.: An inverse kinematic architecture enforcing an arbitrary number of strict priority levels. Vis. Comput. 20(6) (2004)

  3. 3.

    Callennec, B.L., Boulic, R.: Interactive motion deformation with prioritized constraints. Graph. Models 68, 175–193 (2006). Special Issue on SCA 2004

  4. 4.

    Carvalho, S.R.: Data-driven constraint-based motion editing. Ph.D. thesis, École Polytechnique Fédéral de Lausanne (EPFL)—IC School of Computer and Communication Sciences, Lausanne (2009). doi:10.5075/epfl-thesis-4558. URL http://library.epfl.ch/theses/?nr=4558

  5. 5.

    Carvalho, S., Boulic, R., Thalmann, D.: Interactive low-dimensional human motion synthesis by combining motion models and pik. Comput. Animat. Virtual Worlds 18 (2007). Special Issue of Computer Animation and Social Agents (CASA2007)

  6. 6.

    Carvalho, S., Boulic, R., Thalmann, D.: Motion pattern preserving ik operating in the motion principal coefficients space. In: Proceedings of 15-th WSCG, pp. 97–104 (2007)

  7. 7.

    Carvalho, S., Boulic, R., Thalmann, D.: Motion pattern encapsulation for data-driven constraint-based motion editing. In: Egges, A., Geraerts, R., Overmars, M. (eds.) Motion in Games. Lecture Notes in Computer Science, vol. 5884, pp. 116–127. Springer, Berlin (2009)

  8. 8.

    Carvalho, S.R., Vidal, C.A., Boulic, R., Talmann, D.: Propagating latent edited poses across eigen-motions. In: Proceedings of the Computer Graphics International, Ottawa, Canada (2011)

  9. 9.

    Chai, J., Hodgins, J.: Constraint-based motion optimization using a statistical dynamic model. ACM Trans. Graph. 26(3), 8 (2007). doi:http://doi.acm.org/10.1145/1276377.1276387

  10. 10.

    Choi, K., Ko, H.: On-line motion retargetting. J. Vis. Comput. Animat. 11, 223–235 (2000)

  11. 11.

    Glardon, P., Boulic, R., Thalmann, D.: Robust on-line adaptive footplant detection and enforcement for locomotion. Vis. Comput. 22(3), 194–209 (2006). doi:10.1007/s00371-006-0376-9

  12. 12.

    Gleicher, M.: Comparing constraint-based motion editing methods. Graph. Models 63(2), 107–134 (2001)

  13. 13.

    Grassia, F.S.: Practical parameterization of rotations using the exponential map. J. Graph. Tools 3(3), 29–48 (1998)

  14. 14.

    Grochow, K., Martin, S.L., Hertzmann, A., Popovi, Z.: Style-based inverse kinematics. ACM Trans. Graph. 23(3), 522–531 (2004). doi:http://doi.acm.org/10.1145/1015706.1015755

  15. 15.

    Hanafusa, H., Yoshikawa, T., Nakamura, Y.: Analysis and control of articulated robot with redundancy. In: IFAC, 8th Triennal World Congress, vol. 4, pp. 1927–1932 (1981)

  16. 16.

    Ikemoto, L., Arikan, O., Forsyth, D.: Generalizing motion edits with gaussian processes. ACM Trans. Graph. 28(1), 1–12 (2009). doi:http://doi.acm.org/10.1145/1477926.1477927

  17. 17.

    Jain, A.K., Murty, M.N., Flynn, P.J.: Data clustering: a review. ACM Comput. Surv. 31(3), 264–323 (1999). doi:http://doi.acm.org/10.1145/331499.331504

  18. 18.

    Jolliffe, I.T.: Principal Component Analysis. Springer, Berlin (1986)

  19. 19.

    Kochanek, D.H.U., Bartels, R.H.: Interpolating splines with local tension, continuity, and bias control. SIGGRAPH Comput. Graph. 18(3), 33–41 (1984). doi:http://doi.acm.org/10.1145/964965.808575

  20. 20.

    Kovar, L., Gleicher, M., Pighin, F.: Motion graphs. ACM Trans. Graph. 21(3), 473–482 (2002). doi:http://doi.acm.org/10.1145/566654.566605

  21. 21.

    Krüger, B., Tautges, J., Weber, A., Zinke, A.: Fast local and global similarity searches in large motion capture databases. In: Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, SCA’10, pp. 1–10. Eurographics Association, Geneva (2010)

  22. 22.

    Kulpa, R., Multon, F., Arnaldi, B.: Morphology-independent representation of motions for interactive human-like animation. In: EUROGRAPHICS, vol. 24, pp. 343–352 (2005)

  23. 23.

    Lee, J., Shin, S.: A hierarchical approach to interactive motion editing for human-like figures. In: Proceedings of ACM SIGGRAPH, pp. 39–48 (1999)

  24. 24.

    Maciejewski, A.: Dealing with the ill-conditioned equations of motion forarticulated figures. In: Computer Graphics and Applications, vol. 10, pp. 63–71. IEEE Press, New York (1990)

  25. 25.

    Monzani, J.S., Baerlocher, P., Boulic, R., Thalmann, D.: Using an intermediate skeleton and inverse kinematics for motion retargeting. Comput. Graph. Forum 19(3), 11–19 (2000). doi:10.1111/1467-8659.00393

  26. 26.

    Mukai, T., Kuriyama, S.: Geostatistical motion interpolation. ACM Trans. Graph. 24(3), 1062–1070 (2005). doi:http://doi.acm.org/10.1145/1073204.1073313

  27. 27.

    Rasmussen, C.E., Williams, K.I.C.: Gaussian Processes for Machine Learning. Adaptive Computation and Machine Learning. MIT Press, Cambridge (2005)

  28. 28.

    Raunhardt, D., Boulic, R.: Motion constraint. Vis. Comput. 25(5–7), 509–518 (2009). doi:10.1007/s00371-009-0336-2

  29. 29.

    Safonova, A., Hodgins, J.K., Pollard, N.S.: Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces. ACM Trans. Graph. 23(3), 514–521 (2004). doi:http://doi.acm.org/10.1145/1015706.1015754

  30. 30.

    Shin, H., Lee, J.: Motion synthesis and editing in low-dimensional spaces. Comput. Animat. Virtual Worlds 17(3–4), 219–227 (2006). doi:10.1002/cav.v17:3/4

  31. 31.

    Shoemake, K.: Animating rotation with quaternion curves. In: SIGGRAPH’85, pp. 245–254. ACM Press, New York (1985). doi:http://doi.acm.org/10.1145/325334.325242

  32. 32.

    Urtasun, R., Fua, P.: 3d human body tracking using deterministic temporal motion models. In: European Conference on Computer Vision, Prague, Czech Republic (2004)

  33. 33.

    VAL: Visual Agent Laboratory (VAL), Department of Information and Computer Sciences, Toyohashi University of Technology. http://www.val.ics.tut.ac.jp/project/geostat/ (2009)

  34. 34.

    van Basten, B., Egges, A.: Flexible splicing of upper-body motion spaces on locomotion. Comput. Graph. Forum 30(7), 1963–1971 (2011)

  35. 35.

    Whitney, D.E.: Resolved motion rate control of manipulators and human prostheses. In: IEEE Trans. Man-Mach. Syst, vol. 10, pp. 47–53 (1969)

Download references

Acknowledgements

The authors would like to thank Mireille Clavien for the video production; Autodesk/Maya for their donation of Maya software; Benoît Le Callennec for providing access to his motion editing system (with the support of the SNF grant n o 200020-109989); and the valuable suggestions of all the anonymous reviewers. This work was supported by the EPFL—Sport and Rehabilitation Engineering program. The third author would like to acknowledge CAPES/Brazil for the grant 4557/06-9 that helped support him in VRlab-EPFL Switzerland during the academic year 2007–2008.

Author information

Correspondence to Schubert R. Carvalho.

Electronic Supplementary Material

Below are the links to the electronic supplementary material.

(MP4 13.2 MB)

(MP4 13.0 MB)

(MP4 13.2 MB)

(MP4 13.0 MB)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Carvalho, S.R., Boulic, R., Vidal, C.A. et al. Latent motion spaces for full-body motion editing. Vis Comput 29, 171–188 (2013). https://doi.org/10.1007/s00371-012-0678-z

Download citation

Keywords

  • Linear motion models
  • Constraint-based optimization
  • Latent interpolation
  • Motion editing