Abstract
The complexity of optimizations in semi-supervised dimensionality reduction methods has limited their usage. In this paper, an unsupervised and semi-supervised nonlinear dimensionality reduction method that aims at lower space complexity is proposed. First, a positive and negative competitive learning strategy is introduced to the single layered Self-Organizing Incremental Neural Network (SOINN) to process partially labeled datasets. Then, we formulate the dimensionality reduction of SOINN weight vectors as a quadratic programming problem with graph similarities calculated from previous step as constraints. Finally, an approximation of distances between newly arrived samples and the SOINN weight vectors is proposed to complete the dimensionality reduction task. Experiments are carried out on two artificial datasets and the NSL-KDD dataset comparing with Isomap, Transductive Support Vector Machine etc. The results show that the proposed method is effective in dimensionality reduction and an efficient alternate transductive learner.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Beyer, O., Cimiano, P.: Online semi-supervised growing neural gas. Int. J. Neural Syst. 22(05), 1250023 (2012)
Cai, X., Wei, J., Wen, G., Yu, Z.: Local and global preserving semisupervised dimensionality reduction based on random subspace for cancer classification. IEEE J. Biomed. Health Inform. 18(2), 500–507 (2014)
Cai, X., Wen, G., Wei, J., Yu, Z.: Relative manifold based semi-supervised dimensionality reduction. Frontiers Comput. Sci. 8(6), 923–932 (2014)
Edmonds, J., Karp, R.M.: Theoretical improvements in algorithmic efficiency for network flow problems. J. ACM (JACM) 19(2), 248–264 (1972)
Fritzke, B., et al.: A growing neural gas network learns topologies. Adv. Neural Inf. Process. Syst. 7, 625–632 (1995)
Gomory, R.E., Hu, T.C.: Multi-terminal network flows. J. Soc. Ind. Appl. Math. 9(4), 551–570 (1961)
Graepel, T., Herbrich, R., Bollmann-Sdorra, P., Obermayer, K.: Classification on pairwise proximity data. Adv. Neural Inf. Process. Syst. 438–444 (1999)
Kohonen, T.: The self-organizing map. Neurocomputing 21(1), 1–6 (1998)
Maximo, V.R., Quiles, M.G., Nascimento, M.C.: A consensus-based semi-supervised growing neural gas. In: 2014 International Joint Conference on Neural Networks (IJCNN), pp. 2019–2026. IEEE (2014)
Pan, S.J., Kwok, J.T., Yang, Q.: Transfer learning via dimensionality reduction. In: AAAI, vol. 8, pp. 677–682 (2008)
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)
Raina, R., Battle, A., Lee, H., Packer, B., Ng, A.Y.: Self-taught learning: transfer learning from unlabeled data. In: Proceedings of the 24th International Conference on Machine Learning, pp. 759–766. ACM (2007)
Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323–2326 (2000)
Shaw, B., Jebara, T.: Structure preserving embedding. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 937–944. ACM (2009)
Shen, F., Yu, H., Sakurai, K., Hasegawa, O.: An incremental online semi-supervised active learning algorithm based on self-organizing incremental neural network. Neural Comput. Appl. 20(7), 1061–1074 (2011)
Tavallaee, M., Bagheri, E., Lu, W., Ghorbani, A.A.: A detailed analysis of the KDD CUP 99 data set. In: 2009 Proceedings of the Second IEEE Symposium on Computational Intelligence for Security and Defence Applications (2009)
Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)
Torgerson, W.S.: Theory and methods of scaling (1958)
Wang, J.: Maximum variance unfolding. In: Wang, J. (ed.) Geometric Structure of High-Dimensional Data and Dimensionality Reduction, pp. 181–202. Springer, Heidelberg (2011)
Weinberger, K.Q., Packer, B.D., Saul, L.K.: Nonlinear dimensionality reduction by semidefinite programming and kernel matrix factorization. In: Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics, pp. 381–388. Citeseer (2005)
Yang, X., Fu, H., Zha, H., Barlow, J.: Semi-supervised nonlinear dimensionality reduction. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 1065–1072. ACM(2006)
Acknowledgment
This work was partly supported by National Natural Science Foundations of China (No. 61301148, No. 61272061 and No. 71403299), the fundamental research funds for the central universities of China (No. 531107040263, 531107040276), the Research Funds for the Doctoral Program of Higher Education of China (No. 20120161120019 and No. 20130161110002), Hunan Natural Science Foundation of China (No. 14JJ7023).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Xiang, Z., Xiao, Z., Huang, Y., Wang, D., Fu, B., Chen, W. (2016). Unsupervised and Semi-supervised Dimensionality Reduction with Self-Organizing Incremental Neural Network and Graph Similarity Constraints. In: Bailey, J., Khan, L., Washio, T., Dobbie, G., Huang, J., Wang, R. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2016. Lecture Notes in Computer Science(), vol 9651. Springer, Cham. https://doi.org/10.1007/978-3-319-31753-3_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-31753-3_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-31752-6
Online ISBN: 978-3-319-31753-3
eBook Packages: Computer ScienceComputer Science (R0)