Abstract
In this work we investigate the effect of convex and non-convex regularization on the Generalized Matrix Learning Vector Quantization (GMLVQ) classifier, in order to obtain sparse models that guarantee a better generalization ability. Three experiments are used for evaluating six different sparse models in terms of classification accuracy and qualitative sparseness. The results show that non-convex models outperform traditional convex sparse models and non-regularized GMLVQ.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Antoniadis, A., Fan, J.: Regularization of wavelet approximations. J. Am. Stat. Assoc. 939–967 (2011)
Bertsekas, D.P.: Nonlinear programming. Athena Sci. (1999)
Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press (2004)
Candes, E., Tao, T.: The dantzig selector: statistical estimation when p is much larger than n. Ann. Stat. 2313–2351 (2007)
Fan, J., Li, R.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96(456), 1348–1360 (2001)
Figueiredo, M.A.: Adaptive sparseness for supervised learning. IEEE Trans. Pattern Anal. Mach. Intell. 25(9), 1150–1159 (2003)
Hammer, B., Villmann, T.: Generalized relevance learning vector quantization. Neural Netw. 15(8), 1059–1068 (2002)
Kaden, M., Lange, M., Nebel, D., Riedel, M., Geweniger, T., Villmann, T.: Aspects in classification learning-review of recent developments in learning vector quantization. Found. Comput. Decis. Sci. 39(2), 79–105 (2014)
Kohonen, T.: Self-organizing maps, vol. 30. Springer (2001)
Leahy, R.M., Jeffs, B.D.: On the design of maximally sparse beamforming arrays. IEEE Trans. Antennas Propag. 39(8), 1178–1187 (1991)
Lee, H., Battle, A., Raina, R., Ng, A.Y.: Efficient sparse coding algorithms. In: Advances in Neural Information Processing Systems, pp. 801–808 (2006)
Ng, A.Y.: Feature selection, l 1 vs. l 2 regularization, and rotational invariance. In: Proceedings of the Twenty-First International Conference on Machine Learning, p. 78. ACM (2004)
Nikolova, M.: Local strong homogeneity of a regularized estimator. SIAM J. Appl. Math. 61(2), 633–658 (2000)
Nova, D., Estévez, P.A.: A review of learning vector quantization classifiers. Neural Comput. Appl. 25(3–4), 511–524 (2014)
Osborne, M.R., Presnell, B., Turlach, B.A.: A new approach to variable selection in least squares problems. IMA J. Numer. Anal.-Inst. Math. Appl. 20(3), 389–404 (2000)
Sato, A., Yamada, K.: Generalized learning vector quantization. Adv. Neural Inf. Process. Syst. 423–429 (1996)
Schneider, P., Bunte, K., Stiekema, H., Hammer, B., Villmann, T., Biehl, M.: Regularization in matrix relevance learning. IEEE Trans. Neural Netw. 21(5), 831–840 (2010)
Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Royal Stat. Soc. Ser. B (Methodological) 267–288 (1996)
Tropp, J., et al.: Just relax: convex programming methods for identifying sparse signals in noise. IEEE Trans. Inf. Theor. 52(3), 1030–1051 (2006)
Weston, J., Elisseeff, A., Schölkopf, B., Tipping, M.: Use of the zero norm with linear models and kernel methods. J. Mach. Learn. Res. 3, 1439–1461 (2003)
Zhang, T.: Some sharp performance bounds for least squares regression with l1 regularization. Ann. Stat. 37(5A), 2109–2144 (2009)
Acknowledgments
This contribution was funded by CONICYT-CHILE under grants Fondecyt 1140816 and DPI20140090.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Nova, D., Estévez, P.A. (2016). A Study on GMLVQ Convex and Non-convex Regularization. In: Merényi, E., Mendenhall, M., O'Driscoll, P. (eds) Advances in Self-Organizing Maps and Learning Vector Quantization. Advances in Intelligent Systems and Computing, vol 428. Springer, Cham. https://doi.org/10.1007/978-3-319-28518-4_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-28518-4_27
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-28517-7
Online ISBN: 978-3-319-28518-4
eBook Packages: EngineeringEngineering (R0)