As a semi-supervised learning method, Laplacian support vector machine (LapSVM) is popular. Unfortunately, the model generated by LapSVM has a poor sparsity. A sparse decision model has always been fascinating because it could implement data reduction and improve performance. To generate a sparse model of LapSVM, we propose an \(\ell _1\)-norm Laplacian support vector machine (\(\ell _1\)-norm LapSVM), which replaces the \(\ell _2\)-norm with the \(\ell _1\)-norm in LapSVM. The \(\ell _1\)-norm LapSVM has two techniques that can induce sparsity: the \(\ell _1\)-norm regularization and the hinge loss function. We discuss two situations for the \(\ell _1\)-norm LapSVM, linear and nonlinear ones. In the linear \(\ell _1\)-norm LapSVM, the sparse decision model implies that features with nonzero coefficients are contributive. In other words, the linear \(\ell _1\)-norm LapSVM can perform feature selection to achieve the goal of data reduction. Moreover, the nonlinear (kernel) \(\ell _1\)-norm LapSVM can also implement data reduction in terms of sample selection. In addition, the optimization problem of the \(\ell _1\)-norm LapSVM is a convex quadratic programming one. That is, the \(\ell _1\)-norm LapSVM has a unique and global solution. Experimental results on semi-supervised classification tasks have shown a comparable performance of our \(\ell _1\)-norm LapSVM.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Adankon MM, Cheriet M (2009) Model selection for LS-SVM: application to handwriting recognition. Pattern Recogn 42(12):3264–3270
Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7(1):2399–2434
Bennett KP, Demiriz A (1999) Semi-supervised support vector machines. In: Proceedings of international conference on neural information processing systems, pp 368–374
Bruzzone L, Chi M, Marconcini M (2006) A novel transductive SVM for semisupervised classification of remote-sensing images. IEEE Trans Geosci Remote Sens 44(11):3363–3373
Bühler T, Hein M (2009) Spectral clustering based on the graph p-laplacian. In: Proceedings of international conference on machine learning, pp 81–88
Chen L, Yang M (2017) Semi-supervised dictionary learning with label propagation for image classification. Comput Vis Media 3(1):83–94
Chen Y, Wang G, Dong S (2003) Learning with progressive transductive support vector machine. Pattern Recogn Lett 24(12):1845–1855
Cheng S, Huang Q, Liu J, Tang X (2013) A novel inductive semi-supervised SVM with graph-based self-training. In: Proceedings of international conference on intelligent science and intelligent data engineering. Lecture notes in computer science, vol 7751, pp 82–89. Springer, Berlin
Dheeru D, Karra Taniskidou E (2017) UCI machine learning repository. http://archive.ics.uci.edu/ml
Dong A, Fl Chung, Deng Z, Wang S (2016) Semi-supervised SVM with extented hidden features. IEEE Trans Cybern 46(12):2924–2937
Fan J, Tian Z, Zhao M, Chow TW (2018) Accelerated low-rank representation for subspace clustering and semi-supervised classification on large-scale data. Nerual Networks 100:39–48
Fan J, Zhang Y, Udell M (2020) Polynomial matrix completion for missing data imputation and transductive learning. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20), pp. 3842–3849
Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: Proceedings of international conference on knowledge discovery and data mining, pp 77–86
Gammerman A, Vovk V, Vapnik V (2013) Learning by transduction. In: Proceedings of the fourteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann, San Francisco, CA, pp 148–155
Gao Y, Ma J, Yuille AL (2017) Semi-supervised sparse representation based classification for face recognition with insufficient labeled samples. IEEE Trans Image Process 26(5):2545–2560
Gasso G, Zapien K, Canu S (2007) Sparsity regularization path for semi-supervised SVM. In: Proceedings of international conference on machine learning and applications, pp 25–30
Gu Z, Zhang Z, Sun J, Li B (2017) Robust image recognition by l1-norm twin-projection support vector machine. Neurocomputing 223:1–11
Han M, Yin J (2008) The hidden neurons selection of the wavelet networks using support vector machines and ridge regression. Neurocomputing 72(1):471–479
Huimin P, Qiang L, Liran Y, Ping Z (2020) A novel semi-supervised support vector machine with asymmetric squared loss. In: Advances in data analysis and classification, vol 9
Jiang J, Ma J, Chen C, Jiang X, Wang Z (2017) Noise robust face image super-resolution through smooth sparse representation. IEEE Trans Cybern 47(11)
Le HM, Thi HAL, Nguyen MC (2015) Sparse semi-supervised support vector machines by DC programming and DCA. Neurocomputing 153:62–76
Li Z, Zhang Z, Qin J, Zhang Z, Shao L (2020) Discriminative fisher embedding dictionary learning algorithm for object recognition. IEEE Trans Neural Netw Learn Syst 3(3):789–800
Li Y, Kwok JT, Zhou Z (2009) Semi-supervised learning using label mean. In: Proceedings of international conference on machine learning, pp 633–640
Liu RJ, Wang YH, Baba T, Uehara Y, Masumoto D, Nagata S (2008) SVM-based active feedback in image retrieval using clustering and unlabeled data. Pattern Recogn 41(8):2645–2655
Liu Z, Liu H, Zhao Z (2018) Weighted least squares support vector machine for semi-supervised classification. Wireless Pers Commun 103(1):797–808
Ma J, Tian J, Bai X, Tu Z (2013) Regularized vector field learning with sparse approximation for mismatch removal. Pattern Recogn 46:3519–3532
Peng X, Wang Y (2010) A bi-fuzzy progressive transductive support vector machine (bfptsvm) algorithm. Expert Syst Appl 37(1):527–533
Poggio T, Girosi F (1998) A sparse representation for function approximation. Neural Comput 10(6):1445–1454
Refaeilzadeh P, Tang L, Liu H (2016) Encyclopedia of database systems. In: Cross-validation. Springer, Berlin, pp 532–538
Schölkopf B (2008) Sparseness of support vector machines. Mach Learn 4(6):1071–1105
Sun Y, Zhang Z, Jiang W, Zhang Z, Zhang L, Wang M (2020) Discriminative local sparse representation by robust adaptive dictionary pair learning. IIEEE Trans Neural Netw Learn Syst 31:1–15
Tan J, Zhen L, Deng N, Zhang Z (2014) Laplacian p-norm proximal support vector machine for semi-supervised classification. Neurocomputing 144(1):151–158
Vapnik VN (1999) An overview of statistical learning theory. IEEE Trans Neural Netw 10(5):988–999
Wang XY, Wang T, Bu J (2011) Color image segmentation using pixel wise support vector machine classification. Pattern Recogn 44(4):777–787
Yang N, Sang Y, He R, Wang X (2010) Label propagation algorithm based on non-negative sparse representation. In: International conference on life system modeling and intelligent computing, pp 348–357
Zhang Z, Chow TW (2012) Maximum margin multisurface support tensor machines with application to image classification and segmentation. Expert Syst Appl 49:849–860
Zhang L, Zhou W (2010) On the sparseness of 1-norm support vector machines. Neural Netw 23(3):373–385
Zhang L, Zhou W, Jiao L (2004) Hidden space support vector machines. IEEE Trans Neural Netw 15(6):1424–1434
Zhang L, Zhou W, Chang P, Liu J, Yan Z, Wang T, Li F (2012) Kernel sparse representation-based classifier. IEEE Trans Signal Process 60:1684–1695
Zhang L, Zhou W, Li F (2015) Kernel sparse representation-based classifier ensemble for face recognition. Multimed Tools Appl 74(1):123–137
Zhang Z, Jiang W, Qin J, Zhang L, Li F, Zhang M, Yan S (2018) Jointly learning structured analysis discriminative dictionary and analysis multiclass classifier. IEEE Trans Neural Netw Learn Syst 29(8):3798–3814
Zhang Z, Jiang W, Zhang Z, Li S, Liu G, Qin J (2019) Scalable block-diagonal locality-constrained projective dictionary learning. In: Twenty-eighth international joint conference on artificial intelligence IJCAI-19
Zhao M, Liu J, Zhang Z, Fan J (2020) A scalable sub-graph regularization for efficient content based image retrieval with long-term relevance feedback enhancement. Knowledge-based systems, p 106505. https://doi.org/10.1016/j.knosys.2020.106505
Zhou W, Zhang L, Jiao L (2006) Hidden space principal component analysis. In: Proceedings of Pacific-Asia conference on advances in knowledge discovery and data mining, pp 801–805
Zhu X, Goldberg A (2009) Introduction to semi-supervised learning. Morgan and Claypool, Vermont
Zhu J, Rosset S, Hastie T, Tibshirani R (2003) 1-norm support vector machines. In: Proceedings of the 16th international conference on neural information processing ,Systems vol 16(1), pp 49–56
We would like to thank three anonymous reviewers and Editor Zhao for their valuable comments and suggestions, which have significantly improved this paper.
This work was supported in part by the Natural Science Foundation of the Jiangsu Higher Education Institutions of China under Grant No. 19KJA550002, by the Six Talent Peak Project of Jiangsu Province of China under Grant No. XYDXX-054, by the Priority Academic Program Development of Jiangsu Higher Education Institutions, and by the Collaborative Innovation Center of Novel Software Technology and Industrialization.
Conflict of interest
The authors declare that they have no conflict of interest.
Human and animal rights statement
This article does not contain any studies with human participants or animals performed by any of the authors.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Zheng, X., Zhang, L. & Xu, Z. L1-norm Laplacian support vector machine for data reduction in semi-supervised learning. Neural Comput & Applic (2021). https://doi.org/10.1007/s00521-020-05609-9
- Semi-supervised learning
- Support vector machine
- \(\ell _1\)-norm regularization
- Laplacian regularization