Hidden Space Neighbourhood Component Analysis for Cancer Classification

  • Li ZhangEmail author
  • Xiaojuan Huang
  • Bangjun Wang
  • Fanzhang Li
  • Zhao Zhang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9950)


Neighbourhood component analysis (NCA) is a method for learning a distance metric which can maximize the classification performance of the K nearest neighbour (KNN) classifier. However, NCA suffers from the small size sample problem that the number of samples is much less than the number of features. To remedy this, this paper proposes a hidden space neighbourhood components analysis (HSNCA), which is a nonlinear extension of NCA. HSNCA first maps the data in the original space into a feature space by a set of nonlinear mapping functions, and then performs NCA in the feature space. Notably, the number of samples is equal to the number of features in the feature space. Thus, HSNCA can avoid the small size sample problem. Experimental results on DNA array datasets show that HSNCA is feasibility and efficiency.


Neighbourhood components analysis Nonlinear mapping Small size sample problem Feature space Nearest neighbour 



This work was supported in part by the National Natural Science Foundation of China under Grant Nos. 61373093, and 61402310, by the Natural Science Foundation of Jiangsu Province of China under Grant No. BK20140008, by the Natural Science Foundation of the Jiangsu Higher Education Institutions of China under Grant No. 13KJA520001, and by the Soochow Scholar Project.


  1. 1.
    Guyon, I., Weston, J., Barnhill, S., Vapink, V.: Gene selection for cancer classification using support vector machines. Mach. Learn. 46, 389–422 (2002)CrossRefzbMATHGoogle Scholar
  2. 2.
    Li, J.T., Jia, Y.M., Li, W.L.: Adaptive huberized support vector machine and its application to microarray classification. Neural Comput. Appl. 20, 123–132 (2011)CrossRefGoogle Scholar
  3. 3.
    Li, L., Weinberg, C.-R., Darden, T.-A., Pedersen, L.-G.: Gene selection for sample classification based on gene expression data: study of sensitivity to choice of parameters of the GA/KNN method. Bioinformatics 17, 1131–1142 (2001)CrossRefGoogle Scholar
  4. 4.
    Cover, T., Hart, P.: Nearest neighbor pattern classification. IEEE Trans. Inf. Theor. IT–13, 21–27 (1967)CrossRefzbMATHGoogle Scholar
  5. 5.
    Goldberger, J., Roweis, S., Hinton, G., Salakhutdinov, R.: Neighbourhood components analysis. In: Advances in Neural Information Processing Systems, vol. 17, pp. 513–520. MIT Press (2004)Google Scholar
  6. 6.
    Shental, N., Hertz, T., Weinshall, D., Pavel, M.: Adjustment learning and relevant component analysis. In: Proceedings of 7th European Conference on Computer Vision, London, UK, pp. 776–792 (2002)Google Scholar
  7. 7.
    Globerson, A., Roweis, S.T.: Metric learning by collapsing classes. In: Advances in Neural Information Processing Systems, vol. 18 (2005)Google Scholar
  8. 8.
    Weinberger, K.Q., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. J. Mach. Learn. Res. 10, 207–244 (2009)zbMATHGoogle Scholar
  9. 9.
    Xing, E.P., Ng, A.Y., Jordan, M.I., Russell, S.: Distance metric learning, with application to clustering with side-information. In: Advances in Neural Information Processing Systems, vol. 14, pp. 521–528. MIT Press, Cambridge (2002)Google Scholar
  10. 10.
    Chopra, S., Hadsell, R., LeCunGoldberger, Y.: Learning a similiarty metric discriminatively, with application to face verification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, San Diego, CA, pp. 349C–356 (2005)Google Scholar
  11. 11.
    Davis, J.V., Kulis, B., Jain, P., Sra, S., Dhillon, I.S.: Information-theoretic metric learning. In: Proceedings of 24th International Conference on Machine Learning, pp. 209–216. ACM, New York (2007)Google Scholar
  12. 12.
    Qin, C., Song, S., Huang, G., Zhu, L.: Unsupervised neighborhood component analysis for clustering. Neurocomputing 168, 609–617 (2015)CrossRefGoogle Scholar
  13. 13.
    Yang, W., Wang, K., Zuo, W.: Neighborhood component feature selection for high-dimensional data. J. Comput. 7(1), 161–168 (2012)Google Scholar
  14. 14.
    Yang, Z., Laaksonen, J.: Regularized neighborhood component analysis. In: Ersbøll, B.K., Pedersen, K.S. (eds.) SCIA 2007. LNCS, vol. 4522, pp. 253–262. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  15. 15.
    Qin, C., Song, S., Huang, G.: Non-linear neighborhood component analysis based on constructive neural networks. In: Proceedings of 2014 IEEE International Conference on Systems, Man and Cybernetics, pp. 1997–2002. IEEE (2014)Google Scholar
  16. 16.
    Yang, W., Wang, K., Zuo, W.: Fast neighborhood component analysis. Neurocomputing 83(6), 31–37 (2012)CrossRefGoogle Scholar
  17. 17.
    Zhang, L., Zhou, W.D., Jiao, L.C.: Hidden space support vector machines. IEEE Trans. Neural Netw. 15(6), 1424–1434 (2004)CrossRefGoogle Scholar
  18. 18.
    Zhou, W., Zhang, L., Jiao, L.: Hidden space principal component analysis. In: Ng, W.-K., Kitsuregawa, M., Li, J., Chang, K. (eds.) PAKDD 2006. LNCS (LNAI), vol. 3918, pp. 801–805. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  19. 19.
    Zhang, L., Zhou, W.D., Chang, P.-C.: Generalized nonlinear discriminant analysis and its small sample size problems. Neurocomputing 74, 568–574 (2011)CrossRefGoogle Scholar
  20. 20.
    Ding, C., Zhang, L., Wang, B.J.: Hidden space discriminant neighborhood embedding. In: Proceedings of 2014 International Joint Conference on Neural Networks, pp. 271–277. IEEE (2014)Google Scholar
  21. 21.
    Zhang, L., Zhou, W.-D., Chang, P.-C., Liu, J., Yan, Z., Wang, T., Li, F.-Z.: Kernel sparse representation-based classifier. IEEE Trans. Sig. Process. 60, 1684–1695 (2012)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Xu, Z., Dai, M., Meng, D.: Fast and efficient strategies for model selection of gaussian support vector machine. IEEE Trans. Syst. Man Cybern. - Part B: Cybern. 39(5), 1292–1307 (2009)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Li Zhang
    • 1
    Email author
  • Xiaojuan Huang
    • 1
  • Bangjun Wang
    • 1
  • Fanzhang Li
    • 1
  • Zhao Zhang
    • 1
  1. 1.School of Computer Science and Technology & Joint International Research Laboratory of Machine Learning and Neuromorphic ComputingSoochow UniversitySuzhouChina

Personalised recommendations