Abstract
We address the problem of unsupervised online segmenting human motion sequences into different actions. Kernelized Temporal Cut (KTC), is proposed to sequentially cut the structured sequential data into different regimes. KTC extends previous works on online change-point detection by incorporating Hilbert space embedding of distributions to handle the nonparametric and high dimensionality issues. Based on KTC, a realtime online algorithm and a hierarchical extension are proposed for detecting both action transitions and cyclic motions at the same time. We evaluate and compare the approach to state-of-the-art methods on motion capture data, depth sensor data and videos. Experimental results demonstrate the effectiveness of our approach, which yields realtime segmentation, and produces higher action segmentation accuracy. Furthermore, by combining with sequence matching algorithms, we can online recognize actions of an arbitrary person from an arbitrary viewpoint, given realtime depth sensor input.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Barbic, J., Safonova, A., Pan, J.Y., Faloutsos, C., Hodgins, J.K., Pollard, N.S.: Segmenting motion capture data into distinct behaviors. In: Proc. Graphics Interface, pp. 185–194 (2004)
Lv, F., Nevatia, R.: Recognition and Segmentation of 3-D Human Action Using HMM and Multi-class AdaBoost. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part IV. LNCS, vol. 3954, pp. 359–372. Springer, Heidelberg (2006)
Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: Proc. CVPR (2008)
Weinland, D., Özuysal, M., Fua, P.: Making Action Recognition Robust to Occlusions and Viewpoint Changes. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 635–648. Springer, Heidelberg (2010)
Zhong, H., Shi, J., Visontai, M.: Detecting unusual activity in video. In: Proc. CVPR, pp. 816–8231 (2004)
Zhou, F., De la Torre, F., Hodgins, J.K.: Hierarchical aligned cluster analysis for temporal clustering of human motion. Under review at IEEE PAMI (2011)
Niebles, J.C., Chen, C.-W., Fei-Fei, L.: Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part II. LNCS, vol. 6312, pp. 392–405. Springer, Heidelberg (2010)
Chen, J., Gupta, A.: Parametric Statistical Change-point Analysis. Birkhäuser (2000)
Adams, R.P., MacKay, D.J.: Bayesian online changepoint detection. University of Cambridge Technical Report (2007)
De la Torre, F., Campoy, J., Ambadar, Z., Conn, J.F.: Temporal segmentation of facial behavior. In: Proc. ICCV (2007)
Hofmann, T., Scholkopf, B., Smola, A.: Kernel methods in machine learning. Annals of Statistics 36, 1171–1220 (2008)
Smola, A.J., Gretton, A., Song, L., Schölkopf, B.: A Hilbert Space Embedding for Distributions. In: Hutter, M., Servedio, R.A., Takimoto, E. (eds.) ALT 2007. LNCS (LNAI), vol. 4754, pp. 13–31. Springer, Heidelberg (2007)
Gretton, A., Fukumizu, K., Harchaoui, Z., Sriperumbudur, B.: A fast, consistent kernel two-sample test. In: NIPS, vol. 19, pp. 673–681 (2009)
Gretton, A., Borgwardt, K., Rasch, M., Scholkopf, B., Smola, A.: A kernel method for the two-sample-problem. In: NIPS, vol. 19, pp. 513–520 (2007)
Zhou, F., De la Torre, F.: Canonical time warping for alignment of human behavior. In: NIPS, vol. 22, pp. 2286–2294 (2009)
Xuan, X., Murphy, K.: Modeling changing dependency structure in multivariate time series. In: Proc. ICML (2007)
Saatci, Y., Turner, R., Rasmussen, C.: Gaussian process change point models. In: Proc. ICML (2010)
Desobry, F., Davy, M., Doncarli, C.: An online kernel change detection algorithm. IEEE Trans. on Signal Processing 53, 2961–2974 (2005)
Harchaoui, Z., Bach, F., Moulines, E.: Kernel change-point analysis. In: NIPS 21, pp. 609–616 (2009)
Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: NIPS, vol. 14, pp. 849–856 (2002)
von Luxburg, U.: A tutorial on spectral clustering. Statistics and Computing 17, 395–416 (2007)
Fox, E., Sudderth, E., Jordan, M., Willsky, A.: Nonparametric bayesian learning of switching linear dynamical systems. In: NIPS 21, pp. 457–464 (2009)
Zelnik-Manor, L., Irani, M.: Statistical analysis of dynamic actions. IEEE PAMI 28, 1530–1535 (2006)
Satkin, S., Hebert, M.: Modeling the Temporal Extent of Actions. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 536–548. Springer, Heidelberg (2010)
Gong, D., Medioni, G.: Dynamic manifold warping for view invariant action recognition. In: Proc. ICCV, pp. 571–578 (2011)
Hoai, M., Lan, Z., De la Torre, F.: Joint segmentation and classification of human actions in video. In: Proc. CVPR (2011)
Faivishevsky, L., Goldberger, J.: A nonparametric information theoretic clustering algorithm. In: Proc. ICML, pp. 351–358 (2010)
Bach, F.R., Jordan, M.I.: Kernel independent component analysis. JMLR 3, 1–48 (2003)
Laptev, I., Belongie, S., Perez, P., Wills, J.: Periodic motion detection and segmentation via approximate sequence alignment. In: Proc. ICCV, pp. 816–8231 (2005)
Sigal, L., Balan, A.O., Black, M.J.: Humaneva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. IJCV 87, 4–27 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gong, D., Medioni, G., Zhu, S., Zhao, X. (2012). Kernelized Temporal Cut for Online Temporal Segmentation and Recognition. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds) Computer Vision – ECCV 2012. ECCV 2012. Lecture Notes in Computer Science, vol 7574. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33712-3_17
Download citation
DOI: https://doi.org/10.1007/978-3-642-33712-3_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33711-6
Online ISBN: 978-3-642-33712-3
eBook Packages: Computer ScienceComputer Science (R0)