Advertisement

An Online Algorithm for Efficient and Temporally Consistent Subspace Clustering

  • Vasileios ZografosEmail author
  • Kai Krajsek
  • Bjoern Menze
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10111)

Abstract

We present an online algorithm for the efficient clustering of data drawn from a union of arbitrary dimensional, non-static subspaces. Our algorithm is based on an online min-Mahalanobis distance classifier, which simultaneously clusters and is updated from subspace data. In contrast to most existing methods, our algorithm can cope with large amounts of batch or sequential data and is temporally consistent when dealing with time varying data (i.e. time-series). Starting from an initial condition, the classifier provides a first estimate of the subspace clusters in the current time-window. From this estimate, we update the classifier using stochastic gradient descent. The updated classifier is applied back onto the data to refine the subspace clusters, while at the same time we recover the explicit rotations that align the subspaces between time- windows. The whole procedure is repeated until convergence, resulting in a fast, efficient and accurate algorithm. We have tested our algorithm on synthetic and three real datasets and compared with competing methods from literature. Our results show that our algorithm outperforms the competition with superior clustering accuracy and computation speed.

Keywords

Online Algorithm Subspace Cluster Batch Method Stochastic Gradient Descent Dynamic Texture 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

This research was supported by the TU Munchen - IAS (funded by the German Excellence Initiative and the EU 7th Framework Programme under grant agreement no. 291763, the Marie Curie COFUND program of the EU).

Supplementary material

416257_1_En_23_MOESM1_ESM.pdf (180 kb)
Supplementary material 1 (pdf 179 KB)

References

  1. 1.
    Vidal, R.: Subspace clustering. IEEE Signal Process. Mag. 28, 52–68 (2011)CrossRefGoogle Scholar
  2. 2.
    Jolliffe, I.: Principal Component Analysis. Springer, New York (1986)CrossRefzbMATHGoogle Scholar
  3. 3.
    Agrawal, R., Gehrke, J., Gunopulos, D., Raghavan, P.: Automatic subspace clustering of high dimensional data. Data Min. Knowl. Disc. 11, 5–33 (2005)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Elhamifar, E., Vidal, R.: Sparse subspace clustering. In: CVPR (2009)Google Scholar
  5. 5.
    Liu, G., Lin, Z., Yu, Y.: Robust subspace segmentation by low-rank representation. In: ICML (2010)Google Scholar
  6. 6.
    Zografos, V., Ellis, L., Mester, R.: Discriminative subspace clustering. In: CVPR (2013)Google Scholar
  7. 7.
    Vidal, R., Favaro, P.: Low rank subspace clustering (LRSC). Pattern Recogn. Lett. 43, 47–61 (2014)CrossRefGoogle Scholar
  8. 8.
    Patel, V.M., Nguyen, H.V., Vidal, R.: Latent space sparse and low-rank subspace clustering. IEEE J. Sel. Top. Sign. Proces. 9, 691–701 (2015)CrossRefGoogle Scholar
  9. 9.
    Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: analysis and an algorithm. In: NIPS (2001)Google Scholar
  10. 10.
    Wang, S., Fan, Y., Zhang, C., Xu, H., Hao, X., Hu, Y.: Subspace clustering of high dimensional data streams. In: 2008 Seventh IEEE/ACIS International Conference on Computer and Information Science, ICIS 2008, pp. 165–170 (2008)Google Scholar
  11. 11.
    Park, N.H., Lee, W.S.: Memory efficient subspace clustering for online data streams. In: Proceedings of the 2008 International Symposium on Database Engineering and Applications, IDEAS 2008, pp. 199–208 (2008)Google Scholar
  12. 12.
    Peng, X., Zhang, L., Yi, Z.: Scalable sparse subspace clustering. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 430–437 (2013)Google Scholar
  13. 13.
    Peng, X., Zhang, L., Yi, Z.: Inductive sparse subspace clustering. Electron. Lett. 49, 1222–1224 (2013)CrossRefGoogle Scholar
  14. 14.
    Vidal, R.: Online clustering of moving hyperplanes. In: Advances in Neural Information Processing Systems 19, pp. 1433–1440. MIT Press (2006)Google Scholar
  15. 15.
    Wang, J., Fu, Z.: Online motion segmentation based on sparse subspace clustering. J. Inf. Comput. Sci. 12, 1293–1300 (2015)CrossRefGoogle Scholar
  16. 16.
    Arandjelovic, O., Cipolla, R.: Incremental learning of temporally-coherent gaussian mixture models. In: BMVC (2005)Google Scholar
  17. 17.
    Engel, P.M., Heinen, M.R.: Incremental learning of multivariate gaussian mixture models. In: Rocha Costa, A.C., Vicari, R.M., Tonidandel, F. (eds.) SBIA 2010. LNCS (LNAI), vol. 6404, pp. 82–91. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-16138-4_9 CrossRefGoogle Scholar
  18. 18.
    Declercq, A., Piater, J.: Online learning of gaussian mixture models - a two-level approach. In: VISAPP (2008)Google Scholar
  19. 19.
    Bottou, L., Bengio, Y.: Convergence properties of the K-Means algorithm. In: Advances in Neural Information Processing Systems (1995)Google Scholar
  20. 20.
    Bradley, S., Mangasarian, O.: K-plane clustering. J. Global Optim. 16(1), 23–32 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Tseng, P.: Nearest q-flat to M points. J. Optim. Theory Appl. 105, 249–252 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Wang, D., Ding, C., Li, T.: K-subspace clustering. Mach. Learn. Knowl. Disc. Databases 5782, 506–521 (2009)CrossRefGoogle Scholar
  23. 23.
    Bottou, L.: Online algorithms and stochastic approximations. In: Online Learning and Neural Networks. Cambridge University Press (1998)Google Scholar
  24. 24.
    Sherman, J., Morrison, W.J.: Adjustment of an inverse matrix corresponding to a change in one element of a given matrix. Ann. Math. Stat. 21, 124–127 (1950)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Chen, G., Lerman, G.: Spectral curvature clustering (SCC). IJCV 81, 317–330 (2009)CrossRefGoogle Scholar
  26. 26.
    Tron, P., Vidal, R.: A benchmark for the comparison of 3-D motion segmentation algorithms. In: CVPR (2007)Google Scholar
  27. 27.
    Vidal, R., Ma, Y., Sastry, S.: Generalized principal component analysis (GPCA). IEEE PAMI 27, 1945–1959 (2005)CrossRefGoogle Scholar
  28. 28.
    Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE, vol. 86 (1998)Google Scholar
  29. 29.
    Péteri, R., Fazekas, S., Huiskes, M.J.: Dyntex: a comprehensive database of dynamic textures. Pattern Recogn. Lett. 31, 1627–1632 (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Department of Informatics, Institute for Advanced StudyTechnical University of MunichMunichGermany
  2. 2.IEK-8, Forschungszentrum JülichJülichGermany

Personalised recommendations