Abstract
The massive high-dimensional data brings about great time complexity, high storage burden and poor generalization ability of learning models. Feature selection can alleviate curse of dimensionality by selecting a subset of features. Unsupervised feature selection is much challenging due to lack of label information. Most methods rely on spectral clustering to generate pseudo labels to guide feature selection in unsupervised setting. Graphs for spectral clustering can be constructed in different ways, e.g., kernel similarity, or self-representation. The construction of adjacency graphs could be affected by the parameters of kernel functions, the number of nearest neighbors or the size of the neighborhood. However, it is difficult to evaluate the effectiveness of different graphs in unsupervised feature selection. Most existing algorithms only select one graph by experience. In this paper, we propose a novel adaptive multi-graph fusion based unsupervised feature selection model (GFFS). The proposed model is free of graph selection and can combine the complementary information of different graphs. Experiments on benchmark datasets show that GFFS outperforms the state-of-the-art unsupervised feature selection algorithms.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Benabdeslem, K., Hindawi, M.: Efficient semi-supervised feature selection: constraint, relevance, and redundancy. IEEE Trans. Knowl. Data Eng. 26(5), 1131–1143 (2014). https://doi.org/10.1109/TKDE.2013.86
Cai, D., He, X., Han, J.: Document clustering using locality preserving indexing. IEEE Trans. Knowl. Data Eng. 17(12), 1624–1637 (2005). https://doi.org/10.1109/TKDE.2005.198
Cai, D., Zhang, C., He, X.: Unsupervised feature selection for multi-cluster data. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 333–342. ACM (2010). https://doi.org/10.1145/1835804.1835848
Dy, J.G., Brodley, C.E., Kak, A., Broderick, L.S., Aisen, A.M.: Unsupervised feature selection applied to content-based retrieval of lung images. IEEE Trans. Pattern Anal. Mach. Intell. 25(3), 373–378 (2003). https://doi.org/10.1109/TPAMI.2003.1182100
Elhamifar, E., Vidal, R.: Sparse subspace clustering. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 2790–2797. IEEE (2009). https://doi.org/10.1109/CVPRW.2009.5206547
Fang, Y., Wang, R., Dai, B.: Graph-oriented learning via automatic group sparsity for data analysis. In: 2012 IEEE 12th International Conference on Data Mining (ICDM), pp. 251–259. IEEE (2012). https://doi.org/10.1109/ICDM.2012.82
Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)
He, X., Cai, D., Niyogi, P.: Laplacian score for feature selection. In: Advances in Neural Information Processing Systems, pp. 507–514 (2005)
He, X., Yan, S., Hu, Y., Niyogi, P., Zhang, H.J.: Face recognition using laplacianfaces. IEEE Trans. Pattern Anal. Mach. Intell. 27(3), 328–340 (2005). https://doi.org/10.1109/TPAMI.2005.55
Hou, C., Nie, F., Yi, D., Wu, Y.: Feature selection via joint embedding learning and sparse regression. In: IJCAI Proceedings-International Joint Conference on Artificial Intelligence, vol. 22, pp. 1324–1329 (2011). https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-224
Kuhn, H.W., Tucker, A.W.: Nonlinear programming. In: Proceedings of the 2nd Berkeley Symposium on Mathematical Statistics and Probability, pp. 481–492 (1951)
Li, Z., Yang, Y., Liu, J., Zhou, X., Lu, H., et al.: Unsupervised feature selection using nonnegative spectral analysis. In: AAAI, vol. 2, pp. 1026–1032 (2012)
Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., Ma, Y.: Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 171–184 (2013). https://doi.org/10.1109/TPAMI.2012.88
Liu, Y., Jin, R., Yang, L.: Semi-supervised multi-label learning by constrained non-negative matrix factorization. In: AAAI, vol. 6, pp. 421–426 (2006)
Liu, Y., Zhang, C., Zhu, P., Hu, Q.: Generalized multi-view unsupervised feature selection. In: Kůrková, V., Manolopoulos, Y., Hammer, B., Iliadis, L., Maglogiannis, I. (eds.) ICANN 2018. LNCS, vol. 11140, pp. 469–478. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01421-6_45
Lu, C.-Y., Min, H., Zhao, Z.-Q., Zhu, L., Huang, D.-S., Yan, S.: Robust and efficient subspace segmentation via least squares regression. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7578, pp. 347–360. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33786-4_26
Lu, C., Tang, J., Lin, M., Lin, L., Yan, S., Lin, Z.: Correntropy induced L2 graph for robust subspace clustering. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1801–1808 (2013). https://doi.org/10.1109/ICCV.2013.226.
Luo, D., Nie, F., Ding, C., Huang, H.: Multi-subspace representation and discovery. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011. LNCS (LNAI), vol. 6912, pp. 405–420. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23783-6_26
Nie, F., Cai, G., Li, X.: Multi-view clustering and semi-supervised classification with adaptive neighbours. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, 4–9 February 2017, San Francisco, California, USA, pp. 2408–2414 (2017). http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14833
Nie, F., Huang, H., Cai, X., Ding, C.H.: Efficient and robust feature selection via joint l2, 1-norms minimization. In: Advances in Neural Information Processing Systems, pp. 1813–1821 (2010)
Nie, F., Li, J., Li, X., et al.: Parameter-free auto-weighted multiple graph learning: a framework for multiview clustering and semi-supervised classification. In: IJCAI, pp. 1881–1887 (2016)
Qian, M., Zhai, C.: Robust unsupervised feature selection. In: IJCAI, pp. 1621–1627 (2013)
Tang, J., Liu, H.: Unsupervised feature selection for linked social media data. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 904–912. ACM (2012). https://doi.org/10.1145/2339530.2339673
Wang, S., Tang, J., Liu, H.: Embedded unsupervised feature selection. In: AAAI, pp. 470–476 (2015)
Xu, W., Liu, X., Gong, Y.: Document clustering based on non-negative matrix factorization. In: Proceedings of the 26th annual international ACM SIGIR Conference on Research and Development in Informaion Retrieval, pp. 267–273. ACM (2003). https://doi.org/10.1145/860435.860485
Yang, Y., Shen, H.T., Ma, Z., Huang, Z., Zhou, X.: L2, 1-norm regularized discriminative feature selection for unsupervised learning. In: IJCAI Proceedings-International Joint Conference on Artificial Intelligence, vol. 22, p. 1589 (2011). https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-267.
Zhao, Z., Liu, H.: Spectral feature selection for supervised and unsupervised learning. In: Proceedings of the 24th International Conference on Machine Learning, pp. 1151–1157. ACM (2007). https://doi.org/10.1145/1273496.1273641
Zhao, Z., Wang, L., Liu, H., et al.: Efficient spectral feature selection with minimum redundancy. In: AAAI, pp. 673–678 (2010)
Zhu, P., Hu, Q., Zhang, C., Zuo, W.: Coupled dictionary learning for unsupervised feature selection. In: AAAI, pp. 2422–2428 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Niu, S., Zhu, P., Hu, Q., Shi, H. (2019). Adaptive Graph Fusion for Unsupervised Feature Selection. In: Tetko, I., Kůrková, V., Karpov, P., Theis, F. (eds) Artificial Neural Networks and Machine Learning – ICANN 2019: Deep Learning. ICANN 2019. Lecture Notes in Computer Science(), vol 11728. Springer, Cham. https://doi.org/10.1007/978-3-030-30484-3_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-30484-3_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30483-6
Online ISBN: 978-3-030-30484-3
eBook Packages: Computer ScienceComputer Science (R0)