Abstract
Traditional ridge regression (RR) utilizing \({L_{2}}\)-norm as basic measurement is sensitive to outliers and it has the potential risk of overfitting in the computing procedure while dealing with recognition task. Also, the projection number learned by RR is no more than the number of class. LDA is also a well-known method for discriminative feature selection, but the learned projections are limited by the rank of the so-called between-class scatter matrix. In all, both ridge regression and LDA have small-class problem. To solve these problems in both RR and LDA, we propose a method called robust jointly sparse regression (RJSR). RJSR uses \({L_{2,1}}\)-norm instead of \({L_2}\)-norm on both loss function and regularization term to guarantee the robustness to outliers and joint sparsity for effective feature selection. In addition, differ from existing \({L_{2,1}}\)-norm based methods, RJSR incorporates the flexible factor and the robust measurement to guarantee robustness. An alternatively iterative algorithm is designed to compute the optimal solution and the convergence of this algorithm is proved. Experimental evaluation on several well-known data sets shows the merits of the proposed method on feature selection and classification, especially in the case when the face images are corrupted by block noise.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Argyriou A, Evgeniou T, Pontil M (2006) Multi-task feature learning. Adv Neural Inf Process Syst. doi:https://doi.org/10.1007/s10994-007-5040-8
Argyriou A, Evgeniou T, Pontil M (2008) Convex multi-task feature learning. Mach Learn 73(3):243–272. doi:https://doi.org/10.1007/s10994-007-5040-8
Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720. doi:https://doi.org/10.1109/34.598228
Bradley PS, Mangasarjan OL (1998) Feature selection via concave minimization and support vector machines. In: Proceedings of the fifteenth international conference on machine learning (ICML’98)
Bruzzone L (2017) Feature selection based on high dimensional model representation for hyperspectral images. IEEE Trans Image Process. doi:https://doi.org/10.1109/TIP.2017.2687128
Ding C, Zhou D, He X, Zha H (2006) R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization. In: Proceedings of the 23rd international conference on machine learning—ICML’06. doi:https://doi.org/10.1145/1143844.1143880
Fang X-Z, Xu Y (2012) Combine the clustering algorithm and representation-based algorithm for concurrent classification of test samples. 2012 IEEE symposium on computational intelligence for security and defence applications, pp 1–5
He R, Tan T, Wang L, Zheng W-S. 2012. l2,1 regularized correntropy for robust feature selection. In: CVPR, pp 2504–2511. doi:https://doi.org/10.1109/CVPR.2012.6247966
He J, Ding L, Jiang L (2014) Kernel ridge regression classification. In: Proceedings of international joint conference on neural networks, Beijing, China, pp 2263–2267
Huang H, Ding C (2008) Robust tensor factorization using R1 norm. 26th IEEE conference on computer vision and pattern recognition, CVPR. doi:https://doi.org/10.1109/CVPR.2008.4587392
Huang J, Li G, Huang Q, Wu X (2017) Joint feature selection and classification for multilabel learning. IEEE Trans Cybern:1–14
Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding:675–678. doi:https://doi.org/10.1145/2647868.2654889. arXiv:https://arxiv.org/abs/1408.5093 (arXiv Preprint)
Kong D, Ding C, Huang H (2011) Robust nonnegative matrix factorization using L21-norm. In: Proceedings of the 20th ACM international conference on information and knowledge management (CIKM), pp 673–82. doi:https://doi.org/10.1145/2063576.2063676
Kuo CH, Lee JD (2012) Face recognition based on a two-view projective transformation using one sample per subject. IET Comput Vis 6(5):489–498. doi:https://doi.org/10.1049/iet-cvi.2011.0248
Lai Z (2012) Sparse local discriminant projections for discriminant knowledge extraction and classification. IET Comput Vis 6(6):551–559. doi:https://doi.org/10.1049/iet-cvi.2011.0196
Ma Z, Yang Y, Sebe N, Zheng K, Hauptmann AG (2013) Multimedia event detection using a classifier-specific intermediate representation. Multimed IEEE Trans 15(7):1628–1637. doi:https://doi.org/10.1109/TMM.2013.2264928
Martinez AA, Benavente R (1998) The AR face database. CVC Technical Report 24
Mo D, Lai Z (2017) Robust jointly sparse regression for image feature selection. The 4th Asian Conference on Pattern Recognition (ACPR 2017)
Nie F, Huang H, Cai X, Ding C. 2010. Efficient and robust feature selection via joint L2,1-norms minimization. Adv Neural Inf Process Syst 23:1813–1821. https://papers.nips.cc/paper/3988-efficient-and-robust-feature-selection-via-joint-l21-norms-minimization.pdf
Obozinski G, Taskar B, Jordan M (2006) Multi-task feature selection. Technical report, Department of Statistics Department, University of California, Berkeley
Phillips PJ, Moon H, Rauss P, Rizvi SA, Staten Island, New York, and Staten Island (1997) The FERET evaluation methodology for face-recognition algorithms State University of New York at Buffalo, Amherst, NY 14260, pp 137–143
Roweis S (n.d) USPS handwritten digit dataset [Online]
Shang R, Wang W, Stolkin R, Jiao L (2016a) Knowledge-based systems subspace learning-based graph regularized feature selection. Knowl Based Syst 112:152–165. doi:https://doi.org/10.1016/j.knosys.2016.09.006
Shang R, Zhang Z, Jiao L, Liu C, Li Y (2016b) Self-representation based dual-graph regularized feature selection clustering. Neurocomputing 171:1242–1253. doi:https://doi.org/10.1016/j.neucom.2015.07.068
Shang R, Zhang Z, Jiao L, Wang W, Yang S (2016c) Global discriminative-based nonnegative spectral clustering. Pattern Recogn 55:172–182. doi:https://doi.org/10.1016/j.patcog.2016.01.035
Shang R, Wang W, Stolkin R (2017) Non-negative spectral learning and sparse regularized feature selection, 1–14
Shekhar S, Patel VM, Nasrabadi NM (2013) Joint sparse representation for robust multimodal biometrics recognition. IEEE Trans Pattern Anal Mach Intell X(X):1–14
Shi X, Yang Y, Guo Z, Lai Z (2014) Face recognition by sparse discriminant analysis via joint L2,1-norm minimization. Pattern Recogn 47(7):2447–2453. doi:https://doi.org/10.1016/j.patcog.2014.01.007
Tibshirani R (2007) Regression shrinkage and selection via the Lasso Robert Tibshirani. J R Stat Soc Ser B Stat Methodol 58(1):267–288. doi:https://doi.org/10.1111/j.1467-9868.2011.00771.x
Wan M, Lou Z, Jin Z (2009) Margin maximum embedding discriminant (MMED) for feature extraction and classification. Pattern Recognition, 2009. CCPR 2009. Chinese Conference on, no. 1, pp 1–5
Wang L, Ji Z, Hui Z (2007) Hybrid huberized support vector machines for microarray classi cation. In: International conference on machine learning
Xiang S, Nie F, Meng G, Pan C, Zhang C (2012) Discriminative least squares regression for multiclass classification and feature selection. IEEE Trans Neural Netw Learn Syst 23 (11):1738–1754
Xu D, Yan S, Tao D, Zhang L, Li X, Zhang H-J (2006) Human gait recognition with matrix representation. IEEE Trans Circ Syst Video Technol 16(7):896–903. doi:https://doi.org/10.1109/TCSVT.2006.877418
Xu Y, Zhang B, Zhong Z (2015) Multiple representations and sparse representation for image classification. Pattern Recogn Lett 68(P1):9–14. doi:https://doi.org/10.1016/j.patrec.2015.07.032
Yang J, Zhang D, Frangi AF, Yang J-Y (2004) Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Trans Pattern Anal Mach Intell 26(1):131–137. doi:https://doi.org/10.1109/TPAMI.2004.10004
Yang Y, Shen HT, Ma Z, Huang Z, Zhou X (2011) L21-norm regularized discriminative feature selection for unsupervised learning. Int J Conf Artif Intell. doi:https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-267
Yang W, Sun C, Zheng W (2016) A regularized least square based discriminative projections for feature extraction. Neurocomputing 175:198–205. doi:https://doi.org/10.1016/j.neucom.2015.10.049
Zhang J, Yu J, Wan J, Zeng Z (2015) L21 norm regularized fisher criterion for optimal feature selection. Neurocomputing 166:455–463. doi:https://doi.org/10.1016/j.neucom.2015.03.033
Zou H, Hastie T, Tibshirani R (2004) Sparse principal component analysis. J Comput Graph Stat 15(2):1–30. doi:https://doi.org/10.1198/106186006X113430
Zuo W, Zhang D, Wang K (2006) Bidirectional PCA with assembled matrix distance metric for image recognition. IEEE Trans Syst Man Cybern Part B Cybern 36(4):863–872
Acknowledgements
This work was supported in part by the Natural Science Foundation of China (Grant 61573248, Grant 61773328, Grant 61375012 and Grant 61703283), China Postdoctoral Science Foundation (Project 2016M590812 and Project 2017T100645), the Guangdong Natural Science Foundation (Project 2017A030313367 and Project 2017A030310067), and Shenzhen Municipal Science and Technology Innovation Council (No. JCYJ20170302153434048).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Mo, D., Lai, Z. & Meng, Y. Robust jointly sparse regression and its applications. J Ambient Intell Human Comput 9, 1797–1807 (2018). https://doi.org/10.1007/s12652-017-0607-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-017-0607-0