Skip to main content
SpringerLink
Log in

Real-time marker prediction and CoR estimation in optical motion capture

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Optical motion capture systems suffer from marker occlusions resulting in loss of useful information. This paper addresses the problem of real-time joint localisation of legged skeletons in the presence of such missing data. The data is assumed to be labelled 3d marker positions from a motion capture system. An integrated framework is presented which predicts the occluded marker positions using a Variable Turn Model within an Unscented Kalman filter. Inferred information from neighbouring markers is used as observation states; these constraints are efficient, simple, and real-time implementable. This work also takes advantage of the common case that missing markers are still visible to a single camera, by combining predictions with under-determined positions, resulting in more accurate predictions. An Inverse Kinematics technique is then applied ensuring that the bone lengths remain constant over time; the system can thereby maintain a continuous data-flow. The marker and Centre of Rotation (CoR) positions can be calculated with high accuracy even in cases where markers are occluded for a long period of time. Our methodology is tested against some of the most popular methods for marker prediction and the results confirm that our approach outperforms these methods in estimating both marker and CoR positions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Algorithm 1
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. Neighbours are the markers belonging to the same limb segment.

  2. Markers in a clique have constant distances between each other.

  3. A sub-base joint is a joint which connects 2 or more chains.

References

  1. Aristidou, A., Cameron, J., Lasenby, J.: Predicting missing markers to drive real-time centre of rotation estimation. In: Proceedings of the V Conference on Articulated Motion and Deformable Objects, Mallorca, Spain. LNCS, vol. 5098, pp. 238–247 (2008)

    Chapter  Google Scholar 

  2. Aristidou, A., Lasenby, J.: Inverse kinematics: a review of existing techniques and introduction of a new fast iterative solver. Tech. Rep. F-INFENG/TR. 632, CUED (2009)

  3. Aristidou, A., Lasenby, J.: FABRIK: a fast, iterative solver for the inverse kinematics problem. Graphical Models 73(5), 243–260 (2011)

    Article  Google Scholar 

  4. Aristidou, A., Lasenby, J., Cameron, J.: Methods for real-time restoration and estimation in optical motion capture. Tech. Rep. F-INFENG/TR. 619, CUED (2009)

  5. Asseo, S.J., Ardila, R.J.: Sensor independent target state estimator design and evaluation. In: Proceedings of the National Aerospace and Electronics Conference (NAECON), pp. 916–924 (1982)

    Google Scholar 

  6. Backer, A.S.: Estimating missing motion capture data with accelerometers. MPhil thesis, Cambridge University Engineering Department. Cambridge, UK (August 2009)

  7. Baillieul, J.: Kinematic programming alternatives for redundant manipulators. In: Proceedings of the IEEE International Conference on Robotics and Automation, vol. 2, pp. 722–728 (1985)

    Google Scholar 

  8. Balestrino, A., De Maria, G., Sciavicco, L.: Robust control of robotic manipulators. In: Proceedings of the 9th IFAC World Congress, vol. 5, pp. 2435–2440 (1984)

    Google Scholar 

  9. Broeren, J., Sunnerhagen, K.S., Rydmark, M.: A kinematic analysis of a haptic handheld stylus in a virtual environment: a study in healthy subjects. Journal of NeuroEngineering and Rehabilitation 4, 13 (2007)

    Article  Google Scholar 

  10. Brown, J., Latombe, J.C., Montgomery, K.: Real-time knot-tying simulation. The Visual Computer 20(2), 165–179 (2004)

    Article  Google Scholar 

  11. Buss, S.R., Kim, J.S.: Selectively damped least squares for inverse kinematics. Journal of Graphics Tools 10(3), 37–49 (2005)

    Google Scholar 

  12. Cameron, J., Lasenby, J.: A real-time sequential algorithm for human joint localization. In: ACM SIGGRAPH Posters, p. 107. ACM, New York (2005)

    Chapter  Google Scholar 

  13. Chai, J., Hodgins, J.K.: Performance animation from low-dimensional control signals. ACM Transactions on Graphics (TOG) 24(3), 686–696 (2005)

    Article  Google Scholar 

  14. Chang, L.Y., Pollard, N.: Constrained least-squares optimization for robust estimation of center of rotation. Journal of Biomechanics 40(1), 1392–1400 (2007)

    Article  Google Scholar 

  15. Courty, N., Arnaud, E.: Inverse kinematics using sequential Monte Carlo methods. In: Proceedings of the V Conference on Articulated Motion and Deformable Objects, Mallorca, Spain. LNCS, vol. 5098, pp. 1–10 (2008)

    Chapter  Google Scholar 

  16. Courty, N., Cuzol, A.: Conditional stochastic simulation for character animation. Computer Animation and Virtual Worlds—CASA’ 2010 21(3–4), 443–452 (2010)

    Google Scholar 

  17. Der, K.G., Sumner, R.W., Popović, J.: Inverse kinematics for reduced deformable models. In: ACM SIGGRAPH Papers, pp. 1174–1179. ACM, New York (2006)

    Chapter  Google Scholar 

  18. Dessai, S.S., Hornung, A., Kobbelt, L.: Automatic data normalization and parameterization for optical motion tracking. J. Virtual Real. Broadcast. 3(3) (2006)

  19. Doran, C., Lasenby, A.: Geometric Algebra for Physicists. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  20. Dorfmüller-Ulhaas, K.: Robust optical user motion tracking using a Kalman filter. Tech. Rep. TR-2003-6, Institut fuer Informatik, Universitatsstr. 2, 86159 Augsburg (2003)

  21. Doucet, A., De Freitas, N., Gordon, N.: Sequential Monte Carlo Methods in Practice. Springer, New York (2001)

    MATH  Google Scholar 

  22. Ehrig, R.M., Taylor, W.R., Duda, G.N., Heller, M.O.: A survey of formal methods for determining the centre of rotation of ball joints. Journal of Biomechanics 39(15), 2798–2809 (2006)

    Article  Google Scholar 

  23. Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley-Interscience, New York (1987)

    MATH  Google Scholar 

  24. Gamage, S.S.H.U., Lasenby, J.: New least squares solutions for estimating the average centre of rotation and the axis of rotation. Journal of Biomechanics 35(1), 87–93 (2002)

    Article  Google Scholar 

  25. Grochow, K., Martin, S.L., Hertzmann, A., Popović, Z.: Style-based inverse kinematics. In: ACM Transactions on Graphics (TOG), pp. 522–531. ACM, New York (2004)

    Google Scholar 

  26. Halvorsen, K.: Bias compensated least squares estimate of the center of rotation. Journal of Biomechanics 36, 999–1008 (2003)

    Article  Google Scholar 

  27. Halvorsen, K., Lesser, M., Lundberg, A.: A new method for estimating the axis of rotation and the center of rotation. Journal of Biomechanics 32, 1221–1227 (1999)

    Article  Google Scholar 

  28. Hashiguchi, J., Nivomiya, H., Tanaka, H., Nakamura, M., Nobuhara, K.: Biomechanical analysis of a golf swing using motion capture system. In: Proceedings of Annual Meeting of Japanese Society for Orthopaedic Biomechanics, vol. 27, pp. 325–330 (2006)

    Google Scholar 

  29. Hecker, C., Raabe, B., Enslow, R.W., Deweese, J., Maynard, J., van Prooijen, K.: Real-time motion retargeting to highly varied user-created morphologies. ACM Transactions on Graphics (TOG) 27(3), 1–11 (2008)

    Article  Google Scholar 

  30. Herda, L., Fua, P., Plänkers, R., Boulic, R., Thalmann, D.: Skeleton-based motion capture for robust reconstruction of human motion. In: Proceedings of the IEEE Computer Animation (CA’00), pp. 77–86 (2000)

    Chapter  Google Scholar 

  31. Herda, L., Fua, P., Plänkers, R., Boulic, R., Thalmann, D.: Using skeleton-based tracking to increase the reliability of optical motion capture. Human Movement Science Journal 20(3), 313–341 (2001)

    Article  Google Scholar 

  32. Hestenes, D., Sobczyk, G.: Clifford Algebra to Geometric Calculus: A Unified Language for Mathematics and Physics. Reidel, Dordrecht (1984)

    Book  MATH  Google Scholar 

  33. Holzreiter, S.: Calculation of the instantaneous centre of rotation for a rigid body. Journal of Biomechanics 24(7), 643–647 (1991)

    Article  Google Scholar 

  34. Horn, B.: Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America A 4, 629–642 (1987)

    Article  Google Scholar 

  35. Hornung, A., Sar-Dessai, S.: Self-calibrating optical motion tracking for articulated bodies. In: Proceedings of the IEEE Conference on Virtual Reality, VR’05, pp. 75–82. IEEE Computer Society, Washington (2005)

    Google Scholar 

  36. Hsu, E., Gentry, S., Popović, J.: Example-based control of human motion. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA’04), pp. 69–77. Eurographics Association, Grenoble (2004)

    Chapter  Google Scholar 

  37. Ishigaki, S., White, T., Zordan, V.B., Liu, C.K.: Performance-based control interface for character animation. ACM Transaction on Graphics (TOG) 28(3), 1–8 (2009)

    Article  Google Scholar 

  38. Jazwinski, A.H.: Stochastic Processes and Filtering Theory. Academic Press, San Diego (1970)

    MATH  Google Scholar 

  39. Julier, S.J., Uhlmann, J.K.: A new extension of the kalman filter to nonlinear systems. In: Proceedings of the International Symposium on Aerospace/Defense Sensing, Simululation and Controls, vol. Acquisition, Tracking and Pointing XI, Florida, USA, pp. 182–193 (1997)

    Google Scholar 

  40. Kalman, R.E.: A new approach to linear filtering and prediction problems. Trans. ASME, J. Basic Eng., 35–45 (1960)

  41. Li, L., McCann, J., Pollard, N.S., Faloutsos, C.: Dynammo: mining and summarization of coevolving sequences with missing values. In: Proceedings of the 15th International Conference on Knowledge Discovery and Data Mining, pp. 507–516. ACM, Paris (2009)

    Chapter  Google Scholar 

  42. Li, L., McCann, J., Pollard, N.S., Faloutsos, C.: Bolero: a principled technique for including bone length constraints in motion capture occlusion filling. In: Proceedings of the ACM Symposium on Computer Animation, Madrid, Spain (2010)

    Google Scholar 

  43. Li, X.R., Jilkov, V.P.: Survey of maneuvering target tracking. Part I: Dynamic models. IEEE Transactions on Aerospace and Electronic Systems 39(4), 1333–1364 (2003)

    Article  Google Scholar 

  44. Lin, M.C., Gottschalk, S.: Collision detection between geometric models: a survey. In: Proceedings of IMA Conference on Mathematics of Surfaces, pp. 37–56 (1998)

    Google Scholar 

  45. Liu, G., McMillan, L.: Estimation of missing markers in human motion capture. The Visual Computer 22(9–11), 721–728 (2006)

    Article  Google Scholar 

  46. Liu, G., Zhang, J., Wang, W., McMillan, L.: Human motion estimation from a reduced marker set. In: I3D’06: Proceedings of the Symposium on Interactive 3D Graphics and Games, pp. 35–42. ACM, New York (2006)

    Chapter  Google Scholar 

  47. Maidi, M., Ababsa, F., Mallem, M.: Handling occlusions for robust augmented reality systems. EURASIP J. Image Video Process. 2010 (2010)

  48. Menache, A.: Understanding Motion Capture for Computer Animation and Video Games. Morgan Kaufmann, San Francisco (1999)

    Google Scholar 

  49. Merwe, R.V.D., Doucet, A., Freitas, N.D., Wan, E.: The unscented particle filter. Tech. Rep. F-INFENG/TR. 380, CUED (2000)

  50. Nakamura, Y., Hanafusa, H.: Inverse kinematic solutions with singularity robustness for robot manipulator control. Trans. ASME Journal of Dynamic Systems, Measurement, and Control 108(3), 163–171 (1986)

    Article  MATH  Google Scholar 

  51. O’Brien, J.F., Bodenheimer, R.E., Brostow, G.J., Hodgins, J.K.: Automatic joint parameter estimation from magnetic motion capture data. In: Proceedings of Graphic Interface, pp. 53–60 (2000)

    Google Scholar 

  52. Park, S.I., Hodgins, J.K.: Capturing and animating skin deformation in human motion. ACM Transaction on Graphics (TOG) 25(3), 881–889 (2006)

    Article  Google Scholar 

  53. Pechev, A.N.: Inverse kinematics without matrix invertion. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’08), Pasadena, CA, USA, pp. 2005–2012 (2008)

    Google Scholar 

  54. Phasespace inc: Optical motion capture systems. http://www.phasespace.com

  55. Piazza, T., Lundström, J., Hugestrand, A., Kunz, A., Fjeld, M.: Towards solving the missing marker problem in realtime motion capture. In: Proceedings of the International Design Engineering Technical Conference (2009)

    Google Scholar 

  56. Rhijn, A.V., Mulder, J.D.: Optical tracking and automatic model estimation of composite interaction devices. In: IEEE Virtual Reality Conference, pp. 135–142 (2006)

    Google Scholar 

  57. Ringer, M., Lasenby, J.: A procedure for automatically estimating model parameters in optical motion capture. In: Proceedings of the British Machine Vision Conference, pp. 747–756 (2002)

    Google Scholar 

  58. Rose, C., Cohen, M., Bodenheimer, B.: Verbs and adverbs: multidimensional motion interpolation. IEEE Computer Graphics and Applications 18(5), 32–40 (1998)

    Article  Google Scholar 

  59. Silaghi, M.C., Plänkers, R., Boulic, R., Fua, P., Thalmann, D.: Local and global skeleton fitting techniques for optical motion capture. In: Proceedings of the International Workshop on Modelling and Motion Capture Techniques for Virtual Environments, London, UK, pp. 26–40 (1998)

    Chapter  Google Scholar 

  60. Singer, R.A.: Estimating optimal tracking filter performance for manned maneuvering targets. IEEE Transactions on Aerospace and Electronic Systems 6(1), 473–483 (1970)

    Article  Google Scholar 

  61. Singer, R.A., Behnke, K.W.: Real-time tracking filter evaluation and selection for tactical applications. IEEE Transactions on Aerospace and Electronic Systems 7(1), 100–110 (1971)

    Article  Google Scholar 

  62. Sumner, R.W., Zwicker, M., Gotsman, C., Popović, J.: Mesh-based inverse kinematics. ACM Transactions on Graphics (TOG) 24(3), 488–495 (2005)

    Article  Google Scholar 

  63. Tak, S., Ko, H.S.: A physically-based motion retargeting filter. ACM Transactions on Graphics (TOG) 24(1), 98–117 (2005)

    Article  Google Scholar 

  64. Taylor, G.W., Hinton, G.E., Roweis, S.T.: Modeling human motion using binary latent variables. In: Advances in Neural Information Processing Systems, pp. 1345–1352. MIT Press, Cambridge (2007)

    Google Scholar 

  65. Wampler, C.W.: Manipulator inverse kinematic solutions based on vector formulations and damped least-squares methods. IEEE Transactions on Systems, Man and Cybernetics 16(1), 93–101 (1986)

    Article  MATH  Google Scholar 

  66. Wang, J.M., Fleet, D.J., Hertzmann, A.: Gaussian process dynamical models for human motion. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(2), 283–298 (2008)

    Article  Google Scholar 

  67. Wang, L.C.T., Chen, C.C.: A combined optimization method for solving the inverse kinematics problems of mechanical manipulators. IEEE Transactions on Robotics and Automation 7(4), 489–499 (1991)

    Article  Google Scholar 

  68. Welch, G., Bishop, G., Vicci, L., Brumback, S., Keller, K., Colucci, D.: The HiBall tracker: high-performance wide-area tracking for virtual and augmented environments. In: Virtual Reality Software and Technology, VRST, pp. 1–10. ACM, New York (1999)

    Google Scholar 

  69. Wiley, D.J., Hahn, J.K.: Interpolation synthesis of articulated figure motion. IEEE Computer Graphics and Applications 17(6), 39–45 (1997)

    Article  Google Scholar 

  70. Wolovich, W.A., Elliott, H.: A computational technique for inverse kinematics. IEEE Conference on Decision and Control 23, 1359–1363 (1984)

    Article  Google Scholar 

  71. Yu, Q., Li, Q., Deng, Z.: Online motion capture marker labeling for multiple interacting articulated targets. Computer Graphics Forum (Proceedings of Eurographics) 27(7), 477–483 (2007)

    Article  Google Scholar 

  72. Zordan, V.B., Van Der Horst, N.C.: Mapping optical motion capture data to skeletal motion using a physical model. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA’03), pp. 245–250. Eurographics Association, San Diego (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Engineering, University of Cambridge, Cambridge, CB2 1PZ, UK

    Andreas Aristidou & Joan Lasenby

Authors
  1. Andreas Aristidou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joan Lasenby
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andreas Aristidou.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

(AVI 21.9 MB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Aristidou, A., Lasenby, J. Real-time marker prediction and CoR estimation in optical motion capture. Vis Comput 29, 7–26 (2013). https://doi.org/10.1007/s00371-011-0671-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-011-0671-y

Keywords

Search

Navigation