Supervised and Unsupervised Clustering with Probabilistic Shift

  • Sanketh Shetty
  • Narendra Ahuja
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6315)


We present a novel scale adaptive, nonparametric approach to clustering point patterns. Clusters are detected by moving all points to their cluster cores using shift vectors. First, we propose a novel scale selection criterion based on local density isotropy which determines the neighborhoods over which the shift vectors are computed. We then construct a directed graph induced by these shift vectors. Clustering is obtained by simulating random walks on this digraph. We also examine the spectral properties of a similarity matrix obtained from the directed graph to obtain a K-way partitioning of the data. Additionally, we use the eigenvector alignment algorithm of [1] to automatically determine the number of clusters in the dataset. We also compare our approach with supervised[2] and completely unsupervised spectral clustering[1], normalized cuts[3], K-Means, and adaptive bandwidth meanshift[4] on MNIST digits, USPS digits and UCI machine learning data.


Spectral Cluster Transition Probability Matrix Unsupervised Cluster Cluster Core Shift Vector 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Zelnik-Manor, L., Perona, P.: Self-tuning spectral clustering. In: Advances in Neural Information Processing Systems, vol. 17, pp. 1601–1608 (2004)Google Scholar
  2. 2.
    Ng, A., Jordan, M., Weiss, Y.: On spectral clustering: Analysis and an algorithm. In: Advances in NIPS, vol. 14 (2002)Google Scholar
  3. 3.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 888–905 (2000)CrossRefGoogle Scholar
  4. 4.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 603–619 (2002)CrossRefGoogle Scholar
  5. 5.
    Jain, A., Murty, M., Flynn, P.: Data clustering: A review. Surveys 31(3), 264–323 (1999)CrossRefGoogle Scholar
  6. 6.
    Xu, R., Wunsch, D.: Survey of clustering algorithms. IEEE Transactions on Neural Networks 16 (2005)Google Scholar
  7. 7.
    Fukunaga, K., Hostetler, L.: The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Transactions on Information Theory 21, 32–40 (1975)zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Koontz, W.L.G., Narendra, P.M., Fukunaga, K.: A graph-theoretic approach to nonparametric cluster analysis. IEEE Trans. Comput. 25, 936–944 (1976)zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Sheikh, Y.A., Khan, E., Kanade, T.: Mode-seeking by medoidshifts. In: Eleventh IEEE International Conference on Computer Vision (2007)Google Scholar
  10. 10.
    Shapira, L., Avidan, S., Shamir, A.: Mode-detection via median-shift. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1909–1916 (2009)Google Scholar
  11. 11.
    Comaniciu, D., Ramesh, V., Meer, P.: The variable bandwidth mean shift and data-driven scale selection. In: Proceedings of Eighth IEEE International Conference on Computer Vision, ICCV 2001, vol. 1, pp. 438–445 (2001)Google Scholar
  12. 12.
    Chung, F.R.K.: Spectral Graph Theory. CBMS Regional Conference Series in Mathematics, vol. 92. American Mathematical Society, Providence (1997)zbMATHGoogle Scholar
  13. 13.
    Maila, M., Shi, J.: A random walks view of spectral segmentation. In: AI and STATISTICS (AISTATS) (2001)Google Scholar
  14. 14.
    Nadler, B., Galun, M.: Fundamental limitations of spectral clustering. In: Advances in NIPS, pp. 1017–1024 (2007)Google Scholar
  15. 15.
    Ahuja, N.: A transform for multiscale image segmentation by integrated edge and region detection. PAMI 18, 1211–1235 (1996)Google Scholar
  16. 16.
    Gibbons, J.D.: Nonparametric Statistical Inference. Marcel Dekker, New York (1985)zbMATHGoogle Scholar
  17. 17.
    Shetty, S., Ahuja, N.: A uniformity criterion and algorithm for data clustering. In: Proceedings of the 19th ICPR (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Sanketh Shetty
    • 1
  • Narendra Ahuja
    • 1
  1. 1.Department of Electrical and Computer EngineeringUniversity of IllinoisUrbanaUSA

Personalised recommendations