Advertisement

Semi-supervised discriminant Isomap with application to visualization, image retrieval and classification

  • Rui Huang
  • Guopeng Zhang
  • Junli Chen
Original Article

Abstract

As one of the most promising nonlinear unsupervised dimensionality reduction (DR) technique, the Isomap reveals the intrinsic geometric structure of manifold by preserving geodesic distance of all data pairs. Recently, some supervised versions of Isomap have been presented to guide the manifold learning and increase the discriminating capability. However, the performance may deteriorate when there is no sufficient prior information available. Hence, a novel semi-supervised discriminant Isomap (SSD-Isomap) is proposed in the paper. First, two pairwise constraints including must-link and likely-link (LL) are used to depict the neighborhoods of data points. Then, two graphs are constructed based on the two constraints, and distances between points belonging to the LL constraint are modified by a scale parameter. Finally, the geodesic distance metric is obtained based on the graphs, and the corresponding optimal nonlinear subspace is sought. The performance of SSD-Isomap is evaluated by extensive experiments of data visualization, image retrieval and classification. Compared with other state-of-the-art DR methods, SSD-Isomap presents more accurate and robust results.

Keywords

Isomap Semi-supervised dimensionality reduction Visualization Image retrieval Classification 

References

  1. 1.
    Musa AB (2014) A comparison of ℓ1-regularizion, PCA, KPCA and ICA for dimensionality reduction in logistic regression. Int J Mach Learn Cybern 5(6):861–873CrossRefGoogle Scholar
  2. 2.
    Sharma A, Paliwal KK (2015) Linear discriminant analysis for the small sample size problem: an overview. Int J Mach Learn Cybern 6(3):443–454CrossRefGoogle Scholar
  3. 3.
    Cai D, He X, Han J (2008) Training linear discriminant analysis in linear time. In: IEEE 24th international conference on data engineering, Cancun, pp 209–217Google Scholar
  4. 4.
    Liu Y, Rong J (2006) Distance metric learning: a comprehensive survey. http://www.cs.cmu.edu/~liuy/frame_survey_v2.pdf. Accessed 6 May 2015
  5. 5.
    Belkin M, Niyogi P (2002) Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering. Adv Neural Inf Process Syst 14(6):585–591Google Scholar
  6. 6.
    Raducanu B, Dornaika F (2012) A supervised non-linear dimensionality reduction approach for manifold learning. Pattern Recognit 45(6):2432–2444CrossRefzbMATHGoogle Scholar
  7. 7.
    Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326CrossRefGoogle Scholar
  8. 8.
    Laurens VDM (2014) Accelerateing t-SNE using tree-based algorithms. J Mach Learn Res 15(1):3221–3245MathSciNetzbMATHGoogle Scholar
  9. 9.
    Tenenbaum JB, De SV, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323CrossRefGoogle Scholar
  10. 10.
    Silva VD, Tenenbaum JB (2003) Global versus local approaches to nonlinear dimensionality reduction. In: Advances in neural information processing systems, pp 705–712Google Scholar
  11. 11.
    Vlachos M, Domenicon C, Gunopulos D (2002) Non-linear dimensionality reduction techniques for classification and visualization. In: Proceeding of 8th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 645–651Google Scholar
  12. 12.
    Geng X, Zhan DC, Zhou ZH (2005) Supervised nonlinear dimensionality reduction for visualization and classification. IEEE Trans Syst Man Cybern Part B Cybern 35(6):1098–1107CrossRefGoogle Scholar
  13. 13.
    Zhang Z, Chow TW, Zhao M (2012) M-Isomap: orthogonal constrained marginal isomap for nonlinear dimensionality reduction. IEEE Trans Syst Man Cybern Part B Cybern 43(1):180–191Google Scholar
  14. 14.
    Yang B, Xiang M, Zhang Y (2016) Multi-manifold discriminant Isomap for visualization and classification. Pattern Recognit 55:215–230CrossRefGoogle Scholar
  15. 15.
    Meng M, Wei J, Wang J et al (2015) Adaptive semi-supervised dimensionality reduction based on pairwise constraints weighting and graph optimizing. Int J Mach Learn Cybern 8(3):793–805CrossRefGoogle Scholar
  16. 16.
    Chen WJ, Shao YH, Hong N (2013) Laplacian smooth twin support vector machine for semi-supervised classification. Int J Mach Learn Cybern 5(3):459–468CrossRefGoogle Scholar
  17. 17.
    Wang R, Wang XZ, Kwong S et al (2017) Incorporating diversity and informativeness in multiple-instance active learning. IEEEE Trans Fuzzy Syst 25(6):1460–1475CrossRefGoogle Scholar
  18. 18.
    Luo Y, Tao D, Xu C (2013) Vector-valued multi-view semi-supervised learning for multi-label image classification. In: Proceeding of 27th AAAI conference on artificial intelligence, pp 647–653Google Scholar
  19. 19.
    Zhu S, Sun X, Jin D (2016) Multi-view semi-supervised learning for image classification. Neurocomputing 208:136–142CrossRefGoogle Scholar
  20. 20.
    Zhu H, Wang X (2017) A cost-sensitive semi-supervised learning model based on uncertainty. Neurocomputing 251:106–114CrossRefGoogle Scholar
  21. 21.
    Ashfaq RAR, Wang XZ, Huang JZ et al (2017) Fuzziness based semi-supervised learning approach for intrusion detection system. Inf Sci 378:484–497CrossRefGoogle Scholar
  22. 22.
    Cai D, He X, Han J (2007) Semi-supervised discriminant analysis. In: IEEE 11th international conference on computer vision, pp 1–7Google Scholar
  23. 23.
    Yang X, Fu H, Zha H, Barlow J (2006) Semi-supervised nonlinear dimensionality reduction. In: Proceeding of 23th international conference on machine learning, pp 1065–1072Google Scholar
  24. 24.
    Hou V, Zhang C, Wu Y, Nie F (2010) Multiple view semi-supervised dimensionality reduction. Pattern Recognit 43(3):720–730CrossRefzbMATHGoogle Scholar
  25. 25.
    Zhang D, Zhou ZH, Chen S (2007) Semi-supervised dimensional reduction. In: Proceeding of the 7th SIAM international conference on data mining (SDM’07), pp 629–634Google Scholar
  26. 26.
    Xing EP, Ng AY, Jordan MI (2003) Distance metric learning with application to clustering with side-information. In: Advances in neural information processing systems, pp 505–512Google Scholar
  27. 27.
    Hoi SCH, Liu W, Lyu MR, Ma WY (2006) Learning distance metrics with contextual constraints for image retrieval. In: IEEE Computer Society conference on computer vision and pattern recognition (CVPR’06), pp 2072–2078Google Scholar
  28. 28.
    Xia H, Hoi SCH, Jin R, Zhao P (2014) Online multiple kernel similarity learning for visual search. IEEE Trans Pattern Anal Mach Intell 36(3):536–549CrossRefGoogle Scholar
  29. 29.
    Oliveira GL, Vieira AW, Vieira AW (2014) Sparse spatial coding: a novel approach to visual recognition. IEEE Trans Image Process 23(6):2719–2731MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Yang Y, Newsam S (2010) Bag-of-visual-words and spatial extensions for land-use classification. In: Sigspatial international conference on advances in geographic information systems. ACM, pp 270–279Google Scholar
  31. 31.
    Li F-F, Fergus R, Perona P (2004) Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. In: Conference on computer vision and pattern recognition workshop, pp 178–178Google Scholar
  32. 32.
    Georghiades AS, Belhumeur PN, Kriegman DJ (2001) From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660CrossRefGoogle Scholar
  33. 33.
    Peng X, Yu Z, Yi Z (2017) Constructing the L2-graph for robust subspace learning and subspace clustering. IEEE Trans Cybern 47(4):1053–1066CrossRefGoogle Scholar
  34. 34.
    Samaria FS, Harter AC (1994) Parameterization of a stochastic model for human face identification. In: Proceedings of IEEE workshop on applications of computer vision, pp 138–142Google Scholar
  35. 35.
    Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRefGoogle Scholar
  36. 36.
    Bache K, Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml. Accessed 26 June 2016
  37. 37.
    Yu J, Tao D, Li J (2014) Semantic preserving distance metric learning and applications. Inf Sci 281:674–686MathSciNetCrossRefGoogle Scholar
  38. 38.
    Wu P, Hoi SCH, Zhao P, Miao C, Liu Z. Y (2016) Online multi-modal distance metric learning with application to image retrieval. IEEE Trans Knowl Data Eng 28 (2):454–467CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Communication and Information EngineeringShanghai UniversityShanghaiChina

Personalised recommendations