Multimedia Tools and Applications

, Volume 77, Issue 3, pp 3189–3207 | Cite as

Discriminative unsupervised 2D dimensionality reduction with graph embedding

Article
  • 76 Downloads

Abstract

Dimensionality reduction is a great challenge in high dimensional unlabelled data processing. The existing dimensionality reduction methods are prone to employing similarity matrix and spectral clustering algorithm. However, the noises in original data always make the similarity matrix unreliable and degrade the clustering performance. Besides, existing spectral clustering methods just focus on the local structures and ignore the global discriminative information, which may lead to overfitting in some cases. To address these issues, a novel unsupervised 2-dimensional dimensionality reduction method is proposed in this paper, which incorporates the similarity matrix learning and global discriminant information into the procedure of dimensionality reduction. Particularly, the number of the connected components in the learned similarity matrix is equal to cluster number. We compare the proposed method with several 2-dimensional unsupervised dimensionality reduction methods and evaluate the clustering performance by K-means on several benchmark data sets. The experimental results show that the proposed method outperforms the state-of-the-art methods.

Keywords

Similarity matrix Spectral clustering Global discriminant information 

Notes

Acknowledgments

This work was jointly supported by Natural Science Basic Research Plan in Shannxi Province of China No. 2017JM6056, and Designing inter-core Datapath for voltage frequency island mpSoC No. 15JK1726.

References

  1. 1.
    Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15(6):1373–1396CrossRefMATHGoogle Scholar
  2. 2.
    Boyd S, Vandenberghe L (2013) Faybusovich: convex optimization. IEEE Trans Autom Control 51(11):1859–1859Google Scholar
  3. 3.
    Cai D, He X, Han J (2005) Document clustering using locality preserving indexing. IEEE Trans Knowl Data Eng 17(12):1624–1637CrossRefGoogle Scholar
  4. 4.
    Cai D, Zhang C, He X (2010) Unsupervised feature selection for multi-cluster data. In: ACM SIGKDD, p 333–342Google Scholar
  5. 5.
    Chang X, Yang Y (2016) Semisupervised feature analysis by mining correlations among multiple tasks. IEEE Trans Neural Netw Learn Syst. doi: 10.1109/TNNLS.2016.2582746
  6. 6.
    Chang X, Nie F, Ma Z, Yang Y, Zhou X (2015) A convex formulation for spectral shrunk clusteringGoogle Scholar
  7. 7.
    Chang X, Nie F, Wang S, Yang Y, Zhou X, Zhang C (2016) Compound rank-k projections for bilinear analysis. IEEE Trans Neural Netw Learn Syst 27(7):1502–1513. doi: 10.1109/TNNLS.2015.2441735 MathSciNetCrossRefGoogle Scholar
  8. 8.
    Chang X, Nie F, Yang Y, Huang H (2016) A convex sparse PCA for feature analysis. ACM Trans Knowl Discov Data 11(1):3:1–3:16CrossRefGoogle Scholar
  9. 9.
    Chang X, Ma Z, Lin M, Yang Y, Hauptmann A (2017) Feature interaction augmented sparse learning for fast kinect motion detection. IEEE Trans Image Process 26(8):3911–3920. doi: 10.1109/TIP.2017.2708506 MathSciNetCrossRefGoogle Scholar
  10. 10.
    Chang X, Ma Z, Yang Y, Zeng Z, Hauptmann AG (2017) Bi-level semantic representation analysis for multimedia event detection. IEEE Trans Cybern 47(5):1180–1197. doi: 10.1109/TCYB.2016.2539546 CrossRefGoogle Scholar
  11. 11.
    Chang X, Yu YL, Yang Y, Xing EP (2017) Semantic pooling for complex event analysis in untrimmed videos. IEEE Trans Pattern Anal Mach Intell 39(8):1617–1632. doi: 10.1109/TPAMI.2016.2608901 CrossRefGoogle Scholar
  12. 12.
    Chung FR (1997) Spectral graph theory. American Mathematical SocietyGoogle Scholar
  13. 13.
    Dong X, Huang H, Wen H (2010) A comparative study of several face recognition algorithms based on pca. In: ISCSCT, p 443Google Scholar
  14. 14.
    Du L, Shen YD (2015) Unsupervised feature selection with adaptive structure learning. In: ACM SIGKDD. ACM, p 209–218Google Scholar
  15. 15.
    Fan M, Chang X, Tao D (2017) Structure regularized unsupervised discriminant feature analysisGoogle Scholar
  16. 16.
    Gao L, Song J, Liu X, Shao J, Liu J, Shao J (2015) Learning in high-dimensional multimedia data: the state of the art. Multimedia Systems 21:1–11CrossRefGoogle Scholar
  17. 17.
    Gao L, Song J, Nie F, Yan Y, Sebe N, Heng TS (2015) Optimal graph learning with partial tags and multiple features for image and video annotation. In: The IEEE conference on computer vision and pattern recognition (CVPR)Google Scholar
  18. 18.
    He X, Yan S, Hu Y, Niyogi P, Zhang HJ (2005) Face recognition using laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3):328–340CrossRefGoogle Scholar
  19. 19.
    He X, Ji M, Zhang C, Bao H (2011) A variance minimization criterion to feature selection using laplacian regularization. IEEE Trans Pattern Anal Mach Intell 33(10):2013–2025CrossRefGoogle Scholar
  20. 20.
    Hu D, Feng G, Zhou Z (2007) Two-dimensional locality preserving projections (2dlpp) with its application to palmprint recognition. Pattern Recogn 40(1):339–342CrossRefMATHGoogle Scholar
  21. 21.
    Kadir SN, Goodman DF, Harris KD (2014) High-dimensional cluster analysis with the masked em algorithm. Neural Comput 26(11):2379–2394CrossRefGoogle Scholar
  22. 22.
    Kambhatla N, Leen TK (1997) Dimension reduction by local pca. Neural ComputationGoogle Scholar
  23. 23.
    Kodirov E, Xiang T, Fu Z, Gong S (2016) Learning robust graph regularisation for subspace clustering. British Machine Vision ConferenceGoogle Scholar
  24. 24.
    Kokiopoulou E, Saad Y (2007) Orthogonal neighborhood preserving projections: a projection-based dimensionality reduction technique. IEEE Trans Pattern Anal Mach Intell 29(12):2143–2156CrossRefGoogle Scholar
  25. 25.
    Fan K (1950) On a theorem of weyl concerning eigenvalues of linear transformations i. PNAS 35(11):652MathSciNetCrossRefGoogle Scholar
  26. 26.
    Lakshmanan KC, Sadtler PT, Tyler-Kabara EC, Batista AP, Yu BM (2015) Extracting low-dimensional latent structure from time series in the presence of delays. Neural Comput 27(9):1–32CrossRefGoogle Scholar
  27. 27.
    Luo M, Nie F, Chang X, Yang Y, Hauptmann A, Zheng Q (2016) Avoiding optimal mean robust pca/2dpca with non-greedy l1-norm maximization. IJCAIGoogle Scholar
  28. 28.
    Lyons M, Akamatsu S, Kamachi M, Gyoba J (1998) Coding facial expressions with gabor wavelets. In: Proceedings of the 3rd international conference on face & gesture recognition, FG ’98. IEEE Computer Society, Washington, p 200–205Google Scholar
  29. 29.
    Mohar B (1991) The laplacian spectrum of graphs. Graph Theory, Combinatorics, and Applications 12:871–898Google Scholar
  30. 30.
    Nene SA, Nayar SK, Murase H et al (1996) Columbia object image library (coil-20). Tech. rep., Technical report CUCS-005-96Google Scholar
  31. 31.
    Nie F, Wang X, Huang H (2014) Clustering and projected clustering with adaptive neighbors. In: ACM SIGKDD international conference on knowledge discovery and data mining. ACM, p 977–986Google Scholar
  32. 32.
    Nie F, Yuan J, Huang H (2014) Optimal mean robust principal component analysis. In: International conference on machine learning, p 1062–1070Google Scholar
  33. 33.
    Niyogi X (2004) Locality preserving projections. In: Advances in neural information processing systems, vol. 16. MIT PressGoogle Scholar
  34. 34.
    Phillips PJ, Wechsler H, Huang J, Rauss PJ (1998) The feret database and evaluation procedure for face-recognition algorithms. Image Vis Comput 16:295–306CrossRefGoogle Scholar
  35. 35.
    Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326CrossRefGoogle Scholar
  36. 36.
    Samaria FS, Harter AC (1994) Parameterisation of a stochastic model for human face identification. In: Proceedings of the 2nd IEEE workshop on applications of computer vision, 1994. IEEE Computer Society, p 138–142Google Scholar
  37. 37.
    Shortreed S, Meila M (2005) Unsupervised spectral learning. In: Proceedings of the 21st conference annual conference on uncertainty in artificial intelligence (UAI-05). AUAI Press, Arlington, p 534–541Google Scholar
  38. 38.
    Song J, Gao L, Puscas MM, Nie F, Shen F, Sebe N (2016) Joint graph learning and video segmentation via multiple cues and topology calibration. In: Proceedings of the 2016 ACM on multimedia conference. ACM, p 831–840Google Scholar
  39. 39.
    Turk MA, Pentland AP (1991) Face recognition using eigenfaces. In: CVPR, p 586–591Google Scholar
  40. 40.
    Woraratpanya K, Sornnoi M, Leelaburanapong S, Titijaroonroj T, Varakulsiripunt R, Kuroki Y, Kato Y (2015) An improved 2dpca for face recognition under illumination effects. In: International conference on information technology and electrical engineering. IEEE, p 448–452Google Scholar
  41. 41.
    Wu M, Schölkopf B (2006) A local learning approach for clustering. In: Schölkopf B, Platt J, Hoffman T (eds) Advances in neural information processing systems 19. MIT Press, p 1529–1536Google Scholar
  42. 42.
    Yang J, Zhang D, Frangi AF, yu Yang J (2004) Two-dimensional pca: a new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell 26(1): 131–137CrossRefGoogle Scholar
  43. 43.
    Yang Y, Xu D, Nie F, Yan S, Zhuang Y (2010) Image clustering using local discriminant models and global integration. IEEE Trans Image Process 19 (10):2761–2773MathSciNetCrossRefMATHGoogle Scholar
  44. 44.
    Yang Y, Shen H, Nie F, Ji R, Zhou X (2011) Nonnegative spectral clustering with discriminative regularization. In: AAAI conference on artificial intelligence, AAAI 2011. San Francisco, p 555–560Google Scholar
  45. 45.
    Zelnik-Manor L, Perona P (2005) Self-tuning spectral clustering. Adv Neural Inf Proces Syst 17:1601–1608Google Scholar
  46. 46.
    Zhang D, Zhou Z (2005) (2d) 2pca: two-directional two-dimensional pca for efficient face representation and recognition. Neurocomputing 69(1–3):224–231CrossRefGoogle Scholar
  47. 47.
    Zhao Z, Wang L, Liu H, Ye J (2013) On similarity preserving feature selection. IEEE Trans Knowl Data Eng 25(3):619–632CrossRefGoogle Scholar
  48. 48.
    Zhao X, Nie F, Wang S, Guo J, Xu P, Chen X (2017) Unsupervised 2d dimensionality reduction with adaptive structure learning. Neural ComputationGoogle Scholar
  49. 49.
    Zhu L, Shen J, Xie L, Cheng Z (2016) Unsupervised topic hypergraph hashing for efficient mobile image retrieval. IEEE Trans Cybern. doi: 10.1109/TCYB.2016.2591068
  50. 50.
    Zhu L, Shen J, Xie L, Cheng Z (2017) Unsupervised visual hashing with semantic assistant for content-based image retrieval. IEEE Trans Knowl Data Eng 29 (2):472–486. doi: 10.1109/TKDE.2016.2562624 CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.School of Information Science and TechnologyNorthwest UniversityXi’anChina
  2. 2.Faculty of Electrical Engineering and InformaticsBudapest University of Technology and EconomicsBudapestHungary

Personalised recommendations