Advertisement

Motion Segmentation

  • René Vidal
  • Yi Ma
  • S. Shankar Sastry
Chapter
Part of the Interdisciplinary Applied Mathematics book series (IAM, volume 40)

Abstract

The previous two chapters have shown how to use a mixture of subspaces to represent and segment static images. In those cases, different subspaces were used to account for multiple characteristics of natural images, e.g., different textures. In this chapter, we will show how to use a mixture of subspaces to represent and segment time series, e.g., video and motion capture data. In particular, we will use different subspaces to account for multiple characteristics of the dynamics of a time series, such as multiple moving objects or multiple temporal events.

Keywords

Subspace Cluster Point Correspondence Motion Segmentation Affine Subspace Epipolar Constraint 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. Aggarwal, G., Roy-Chowdhury, A., & Chellappa, R. (2004). A system identification approach for video-based face recognition. In Proceedings of International Conference on Pattern Recognition (pp. 23–26).Google Scholar
  2. Ali, S., Basharat, A., & Shah, M. (2007). Chaotic invariants for human action recognition. In Proceedings of International Conference on Computer Vision.Google Scholar
  3. Avidan, S., & Shashua, A. (2000). Trajectory triangulation: 3D reconstruction of moving points from a monocular image sequence. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(4), 348–357.CrossRefGoogle Scholar
  4. Ayazoglu, M., Li, B., Dicle, C., Sznaier, M., & Camps, O. (2011). Dynamic subspace-based coordinated multicamera tracking. In IEEE International Conference on Computer Vision (pp. 2462–2469)Google Scholar
  5. Barbic, J., Safonova, A., Pan, J.-Y., Faloutsos, C., Hodgins, J. K., & Pollar, N. S. (2004). Segmenting motion capture data into distinct behaviors. In Graphics Interface.Google Scholar
  6. Béjar, B., Zappella, L., & Vidal, R. (2012). Surgical gesture classification from video data. In Medical Image Computing and Computer Assisted Intervention (pp. 34–41).Google Scholar
  7. Bissacco, A., Chiuso, A., Ma, Y., & Soatto, S. (2001). Recognition of human gaits. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Vol. 2, pp. 52–58).Google Scholar
  8. Boult, T., & Brown, L. (1991). Factorization-based segmentation of motions. In IEEE Workshop on Motion Understanding (pp. 179–186).Google Scholar
  9. Chan, A., & Vasconcelos, N. (2005a). Classification and retrieval of traffic video using auto-regressive stochastic processes. In Proceedings of 2005 IEEE Intelligent Vehicles Symposium (pp. 771–776).Google Scholar
  10. Chan, A., & Vasconcelos, N. (2005b). Mixtures of dynamic textures. In IEEE International Conference on Computer Vision (Vol. 1, pp. 641–647).Google Scholar
  11. Chaudhry, R., Ravichandran, A., Hager, G., & Vidal, R. (2009). Histograms of oriented optical flow and binet-cauchy kernels on nonlinear dynamical systems for the recognition of human actions. In IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
  12. CMU (2003). MOCAP database. http://mocap.cs.cmu.edu.
  13. Costeira, J., & Kanade, T. (1998). A multibody factorization method for independently moving objects. International Journal of Computer Vision, 29(3), 159–179.CrossRefGoogle Scholar
  14. Doretto, G., Chiuso, A., Wu, Y., & Soatto, S. (2003). Dynamic textures. International Journal of Computer Vision, 51(2), 91–109.CrossRefMATHGoogle Scholar
  15. Doretto, G., & Soatto, S. (2003). Editable dynamic textures. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. II, pp. 137–142).Google Scholar
  16. Doretto, G., & Soatto, S. (2006). Dynamic shape and appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12), 2006–2019.CrossRefGoogle Scholar
  17. Feng, X., & Perona, P. (1998). Scene segmentation from 3D motion. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 225–231).Google Scholar
  18. Fitzgibbon, A., & Zisserman, A. (2000). Multibody structure and motion: 3D reconstruction of independently moving objects. In European Conference on Computer Vision (pp. 891–906).Google Scholar
  19. Ghoreyshi, A., & Vidal, R. (2007). Epicardial segmentation in dynamic cardiac MR sequences using priors on shape, intensity, and dynamics, in a level set framework. In IEEE International Symposium on Biomedical Imaging (pp. 860–863).Google Scholar
  20. Han, M., & Kanade, T. (2000). Reconstruction of a scene with multiple linearly moving objects. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (Vol. 2, pp. 542–549).Google Scholar
  21. Han, M., & Kanade, T. (2001). Multiple motion scene reconstruction from uncalibrated views. In Proceedings of IEEE International Conference on Computer Vision (Vol. 1, pp. 163–170).Google Scholar
  22. Hartley, R., & Vidal, R. (2004). The multibody trifocal tensor: Motion segmentation from 3 perspective views. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. I, pp. 769–775).Google Scholar
  23. Hyndman, M., Jepson, A., & Fleet, D. J. (2007). Higher-order autoregressive models for dynamic textures. In British Machine Vision Conference (pp. 76.1–76.10). doi: 10.5244/C.21.76.
  24. Kanatani, K. (2001). Motion segmentation by subspace separation and model selection. In IEEE International Conference on Computer Vision (Vol. 2, pp. 586–591).Google Scholar
  25. Kanatani, K., & Matsunaga, C. (2002). Estimating the number of independent motions for multibody motion segmentation. In European Conference on Computer Vision (pp. 25–31).Google Scholar
  26. Kanatani, K., & Sugaya, Y. (2003). Multi-stage optimization for multi-body motion segmentation. In Australia-Japan Advanced Workshop on Computer Vision (pp. 335–349).Google Scholar
  27. Kim, S. J., Doretto, G., Rittscher, J., Tu, P., Krahnstoever, N., & Pollefeys, M. (2009). A model change detection approach to dynamic scene modeling. In Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance, 2009 (AVSS ’09) (pp. 490–495).Google Scholar
  28. Li, B., Ayazoglu, M., Mao, T., Camps, O. I., & Sznaier, M. (2011). Activity recognition using dynamic subspace angles. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (pp. 3193–3200). New York: IEEE.Google Scholar
  29. Ma, Y., Soatto, S., Kosecka, J., & Sastry, S. (2003). An Invitation to 3D Vision: From Images to Geometric Models. New York: Springer.MATHGoogle Scholar
  30. Nascimento, J. C., Figueiredo, M. A. T., & Marques, J. S. (2005). Recognition of human activities using space dependent switched dynamical models. In IEEE International Conference on Image Processing (pp. 852–855).Google Scholar
  31. Nunez, F., & Cipriano, A. (2009). Visual information model based predictor for froth speed control in flotation process. Minerals Engineering, 22(4), 366–371.CrossRefGoogle Scholar
  32. Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., & Bajcsy, R. (2013). Berkeley MHAD: A comprehensive multimodal human action database. In IEEE Workshop on Applications of Computer Vision.Google Scholar
  33. Overschee, P. V., & Moor, B. D. (1993). Subspace algorithms for the stochastic identification problem. Automatica, 29(3), 649–660.MathSciNetCrossRefMATHGoogle Scholar
  34. Rahimi, A., Darrell, T., & Recht, B. (2005). Learning appearance manifolds from video. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. 1, pp. 868–875).Google Scholar
  35. Ravichandran, A., Chaudhry, R., & Vidal, R. (2009). View-invariant dynamic texture recognition using a bag of dynamical systems. In IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
  36. Ravichandran, A., Chaudhry, R., & Vidal, R. (2013). Categorizing dynamic textures using a bag of dynamical systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(2), 342–353.CrossRefGoogle Scholar
  37. Ravichandran, A., & Vidal, R. (2008). Video registration using dynamic textures. In European Conference on Computer Vision.Google Scholar
  38. Ravichandran, A., & Vidal, R. (2011). Video registration using dynamic textures. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1), 158–171.CrossRefGoogle Scholar
  39. Ravichandran, A., Vidal, R., & Halperin, H. (2006). Segmenting a beating heart using polysegment and spatial GPCA. In IEEE International Symposium on Biomedical Imaging (pp. 634–637).Google Scholar
  40. Saisan, P., Bissacco, A., Chiuso, A., & Soatto, S. (2004). Modeling and synthesis of facial motion driven by speech. In European Conference on Computer Vision (Vol. 3, pp. 456–467).Google Scholar
  41. Shakernia, O., Vidal, R., & Sastry, S. (2003). Multi-body motion estimation and segmentation from multiple central panoramic views. In IEEE International Conference on Robotics and Automation (Vol. 1, pp. 571–576).Google Scholar
  42. Shashua, A., & Levin, A. (2001). Multi-frame infinitesimal motion model for the reconstruction of (dynamic) scenes with multiple linearly moving objects. In Proceedings of IEEE International Conference on Computer Vision (Vol. 2, pp. 592–599).Google Scholar
  43. Sturm, P. (2002). Structure and motion for dynamic scenes - the case of points moving in planes. In Proceedings of European Conference on Computer Vision (pp. 867–882).Google Scholar
  44. Szummer, M., & Picard, R. W. (1996). Temporal texture modeling. In IEEE International Conference on Image Processing (Vol. 3, pp. 823–826).Google Scholar
  45. Torr, P., Szeliski, R., & Anandan, P. (2001). An integrated Bayesian approach to layer extraction from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(3), 297–303.CrossRefGoogle Scholar
  46. Torr, P. H. S. (1998). Geometric motion segmentation and model selection. Philosophical Transactions of the Royal Society of London, 356(1740), 1321–1340.MathSciNetCrossRefMATHGoogle Scholar
  47. Tron, R., & Vidal, R. (2007). A benchmark for the comparison of 3-D motion segmentation algorithms. In IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
  48. Turaga, P., Veeraraghavan, A., Srivastava, A., & Chellappa, R. (2011). Statistical computations on special manifolds for image and video-based recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(11), 2273–2286.CrossRefGoogle Scholar
  49. Vidal, R. (2005). Multi-subspace methods for motion segmentation from affine, perspective and central panoramic cameras. In IEEE Conference on Robotics and Automation (pp. 1753–1758).Google Scholar
  50. Vidal, R., & Hartley, R. (2004). Motion segmentation with missing data by PowerFactorization and Generalized PCA. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. II, pp. 310–316).Google Scholar
  51. Vidal, R., & Ma, Y. (2004). A unified algebraic approach to 2-D and 3-D motion segmentation. In European Conference on Computer Vision (pp. 1–15).Google Scholar
  52. Vidal, R., Ma, Y., Soatto, S., & Sastry, S. (2006). Two-view multibody structure from motion. International Journal of Computer Vision, 68(1), 7–25.CrossRefGoogle Scholar
  53. Vidal, R., & Ravichandran, A. (2005). Optical flow estimation and segmentation of multiple moving dynamic textures. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. II, pp. 516–521).Google Scholar
  54. Vidal, R., & Sastry, S. (2003). Optimal segmentation of dynamic scenes from two perspective views. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. 2, pp. 281–286).Google Scholar
  55. Vidal, R., Soatto, S., Ma, Y., & Sastry, S. (2002b). Segmentation of dynamic scenes from the multibody fundamental matrix. In ECCV Workshop on Visual Modeling of Dynamic Scenes.Google Scholar
  56. Wang, J. M., Fleet, D. J., & Hertzmann, A. (2008b). Gaussian process dynamical models for human motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(2), 283–298.Google Scholar
  57. Wolf, L., & Shashua, A. (2001a). Affine 3-D reconstruction from two projective images of independently translating planes. In Proceedings of IEEE International Conference on Computer Vision (pp. 238–244).Google Scholar
  58. Wolf, L., & Shashua, A. (2001b). Two-body segmentation from two perspective views. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 263–270).Google Scholar
  59. Woolfe, F., & Fitzgibbon, A. (2006). Shift-invariant dynamic texture recognition. In Proceedings of European Conference on Computer Vision, pages II: 549–562.Google Scholar
  60. Wu, Y., Zhang, Z., Huang, T., & Lin, J. (2001). Multibody grouping via orthogonal subspace decomposition. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. 2, pp. 252–257).Google Scholar
  61. Xiong, F., Camps, O., & Sznaier, M. (2011). Low order dynamics embedding for high dimensional time series. In IEEE International Conference on Computer Vision (pp. 2368–2374).Google Scholar
  62. Xiong, F., Camps, O., & Sznaier, M. (2012). Dynamic context for tracking behind occlusions. In European Conference on Computer Vision. Lecture notes in computer science (Vol. 7576, pp. 580–593). Berlin/Heidelberg: Springer.Google Scholar
  63. Yuan, L., Wen, F., Liu, C., & Shum, H. (2004). Synthesizing dynamic texture with closed-loop linear dynamic system. In European Conference on Computer Vision (pp. 603–616).Google Scholar
  64. Zelnik-Manor, L., & Irani, M. (2003). Degeneracies, dependencies and their implications in multi-body and multi-sequence factorization. In IEEE Conference on Computer Vision and Pattern Recognition (Vol. 2, pp. 287–293).Google Scholar
  65. Zhang, T., Szlam, A., Wang, Y., & Lerman, G. (2010). Randomized hybrid linear modeling via local best-fit flats. In IEEE Conference on Computer Vision and Pattern Recognition (pp. 1927–1934).Google Scholar
  66. Zhou, F., la Torre, F. D., & Hodgins, J. K. (2008). Aligned cluster analysis for temporal segmentation of human motion. In International Conference on Automatic Face and Gesture Recognition.Google Scholar

Copyright information

© Springer-Verlag New York 2016

Authors and Affiliations

  • René Vidal
    • 1
  • Yi Ma
    • 2
  • S. Shankar Sastry
    • 3
  1. 1.Center for Imaging Science Department of Biomedical EngineeringJohns Hopkins UniversityBaltimoreUSA
  2. 2.School of Information Science and Technology ShanghaiTech UniversityShanghaiChina
  3. 3.Department of Electrical Engineering and Computer ScienceUniversity of California BerkeleyBerkeleyUSA

Personalised recommendations