Robust Spectral 3D-Bodypart Segmentation Along Time

  • Fabio Cuzzolin
  • Diana Mateus
  • Edmond Boyer
  • Radu Horaud
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4814)


In this paper we present a novel tool for body-part segmentation and tracking in the context of multiple camera systems. Our goal is to produce robust motion cues over time sequences, as required by human motion analysis applications. Given time sequences of 3D body shapes, body-parts are consistently identified over time without any supervision or a priori knowledge. The approach first maps shape representations of a moving body to an embedding space using locally linear embedding. While this map is updated at each time step, the shape of the embedded body remains stable. Robust clustering of body parts can then be performed in the embedding space by k-wise clustering, and temporal consistency is achieved by propagation of cluster centroids. The contribution with respect to methods proposed in the literature is a totally unsupervised spectral approach that takes advantage of temporal correlation to consistently segment body-parts over time. Comparisons on real data are run with direct segmentation in 3D by EM clustering and ISOMAP-based clustering: the way different approaches cope with topology transitions is discussed.


Cluster Centroid Locally Linear Embedding Topology Transition Linear Embedding Embedding Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Moeslund, T., Hilton, A., Krüger, V.: A survey of advances in vision based human motion capture and analysis. Computer Vision and Image Understanding 103(2-3), 90–126 (2006)CrossRefGoogle Scholar
  2. 2.
    Hogg, D.: Model based vision: a program to see a walking person. Image and Vision Computing 1(1), 5–20 (1983)CrossRefGoogle Scholar
  3. 3.
    Gavrila, D., Davis, L.: 3-D model-based tracking of humans in action: A multi-view approach. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, USA, pp. 73–80. IEEE Computer Society Press, Los Alamitos (1996)Google Scholar
  4. 4.
    Deutscher, J., Blake, A., Reid, I.: Articulated body motion capture by annealed particle filtering. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, USA, vol. 2, pp. 126–133 (2000)Google Scholar
  5. 5.
    Brand, M.: Shadow puppetry. In: Proceedings of the 7th International Conference on Computer Vision, Kerkyra, Greece, vol. 2, pp. 1237–1244 (1999)Google Scholar
  6. 6.
    Grauman, K., Shakhnarovich, G., Darrell, T.: Inferring 3D structure with a statistical image-based shape model. In: Proceedings of the 9th International Conference on Computer Vision, Nice, France, pp. 641–648 (2003)Google Scholar
  7. 7.
    Elgammal, A., Lee, C.: Inferring 3D body pose from silhouettes using activity manifold learning. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Washington, USA, pp. 681–688. IEEE Computer Society Press, Los Alamitos (2004)Google Scholar
  8. 8.
    Peursum, P., Venkatesh, S., West, G.: Tracking-as-recognition for articulated full-body human motion analysis. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, USA, pp. 1–8. IEEE Computer Society Press, Los Alamitos (2007)Google Scholar
  9. 9.
    Cheung, G., Kanade, T., Bouguet, J.Y., Holler, M.: A real time system for robust 3D voxel reconstruction of human motions. In: Proceedings of CVPR 2000, pp. 2714–2720 (2000)Google Scholar
  10. 10.
    Mukasa, T., Nobuhara, S., Maki, A., Matsuyama, T.: Finding articulated body in time-series volume data. In: Perales, F.J., Fisher, R.B. (eds.) AMDO 2006. LNCS, vol. 4069, pp. 395–404. Springer, Heidelberg (2006)Google Scholar
  11. 11.
    de Aguiar, E., Theobalt, C., Magnor, M., Theisel, H., Seidel, H.P.: M3: Marker-free model reconstruction and motion tracking from 3D voxel data. In: Cohen-Or, D., Ko, H.S., Terzopoulos, D., Warren, J. (eds.) PG 2004. 12th Pacific Conference on Computer Graphics and Applications, Seoul, Korea, pp. 101–110. IEEE Computer Society Press, Los Alamitos (2004)Google Scholar
  12. 12.
    Brostow, G.J., Essa, I., Steedly, D., Kwatra, V.: Novel skeletal representation for articulated creatures. In: Proceedings of the 8th European Conference on Computer Vision, Prague, Czech Republic, vol. 3, pp. 66–78 (2004)Google Scholar
  13. 13.
    Chu, C.W., Jenkins, O.C., Mataric, M.J.: Markerless kinematic model and motion capture from volume sequences. In: CVPR 2003. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 475–482. IEEE Computer Society Press, Los Alamitos (2003)Google Scholar
  14. 14.
    Sundaresan, A., Chellappa, R.: Segmentation and probalistic registration of articulated body models. In: Proceedings of the 18th International Conference on Pattern Recognition, Hong Kong, vol. 2, pp. 92–96 (2006)Google Scholar
  15. 15.
    Jenkins, O., Mataric, M.: A spatio-temporal extension to isomap nonlinear dimension reduction. In: Proceedings of the 31th International Conference on Machine Learning, Alberta, Canada (2004)Google Scholar
  16. 16.
    Lin, R., Liu, C.B., Yang, M.H., Ahuja, N., Levinson, S.: Learning nonlinear manifolds from time series. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 245–256. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  17. 17.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) Advances in Neural Information Processing Systems 14, MIT Press, Cambridge (2002)Google Scholar
  18. 18.
    Tenenbaum, J.B., Silva, V.d., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)CrossRefGoogle Scholar
  19. 19.
    Roweis, S., Saul, L.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)CrossRefGoogle Scholar
  20. 20.
    MacQueen, J.B.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, pp. 281–297 (1967)Google Scholar
  21. 21.
    Heiser, W.J., Bennani, M.: Triadic distance models: Axiomatization and least squares representation. J. Math. Psy. 41, 189–206 (1997)zbMATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Hayashi, C.: Two dimensional quantification based on the measure of dissimilarity among three elements. Ann. I. Stat. Math. 24, 251–257 (1972)zbMATHCrossRefGoogle Scholar
  23. 23.
    Agarwal, S., Lim, J., Zelnik-Manor, L., Perona, P., Kriegman, D., Belongie, S.: Beyond pairwise clustering. In: Proceedings of CVPR, vol. 2, pp. 838–845 (2005)Google Scholar
  24. 24.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. PAMI 22(8), 888–905 (2000)Google Scholar
  25. 25.
    Ng, M.J.A., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: Advances in Neural Information Processing Systems 14: Proceedings of the 2001 (2001)Google Scholar
  26. 26.
    Bengio, Y., Paiement, J.F., Vincent, P.: Out-of-sample extensions for LLE, Isomap, MDS, eigenmaps, and spectral clustering. Technical report, Universite’ de Montreal (2003)Google Scholar
  27. 27.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. Royal Stat. Soc. 39, 1–38 (1977)zbMATHMathSciNetGoogle Scholar
  28. 28.
    Mateus, D., Cuzzolin, F., Boyer, E., Horaud, R.: Articulated shape matching by locally linear embedding and orthogonal alignment. In: Proceedings of the ICCV 2007-NTRL Workshop, Rio de Janeiro, Brasil (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Fabio Cuzzolin
    • 1
  • Diana Mateus
    • 1
  • Edmond Boyer
    • 1
  • Radu Horaud
    • 1
  1. 1.INRIA Rhone-Alpes, 655 avenue de l’Europe, 38334 MontbonnotFrance

Personalised recommendations