Abstract
The paper suggests an extended version of principle of empirical risk minimization and principle of smoothly winsorized sums minimization for robust neural networks learning. It’s based on using of M-averaging functions instead of the arithmetic mean for empirical risk estimation (M-risk). Theese approaches generalize robust algorithms based on using median and quantiles for estimation of mean losses. An iteratively reweighted schema for minimization of M-risk is proposed. This schema allows to use weighted version of traditional back propagation algorithms for neural networks learning in presence of outliers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Vapnik, V.: The Nature of Statistical Learning Theory. Information Science and Statistics. Springer, New York (2000). https://doi.org/10.1007/978-1-4757-3264-1
Huber, P.J.: Robust Statistics. John Wiley and Sons, Hoboken (1981)
Mesiar, R., Komornikova, M., Kolesarova, A., Calvo, T.: Aggregation functions: a revision. In: Bustince, H., Herrera, F., Montero, J. (eds.) Fuzzy Sets and Their Extensions: Representation Aggregation and Models. Springer, Heidelberg (2008). https://doi.org/10.1016/j.fss.2009.05.012
Grabich, M., Marichal, J.-L., Pap, E.: Aggregation Functions. Encyclopedia of Mathematics and its Applications. Series (Book 127). Cambridge University Press, Cambridge (2009)
Beliakov, G., Bustince Sola, H., Calvo Sánchez, T.: A Practical Guide to Averaging Functions. SFSC, vol. 329. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-24753-3
Calvo, T., Beliakov, G.: Aggregation functions based on penalties. Fuzzy Sets Syst. 161(10), 1420–1436 (2010). https://doi.org/10.1016/j.fss.2009.05.012
Yohai, V.J.: High breakdown-point and high efficiency robust estimates for regression. Ann. Stat. 15, 642–656 (1987). https://doi.org/10.1214/aos/1176350366
Rousseeuw, P.J.: Least median of squares regression. J. Am. Stat. Assoc. 79, 871–880 (1987)
Rousseeuw, P.J., Leroy, A.M.: Robust Regression and Outlier Detection. John Wiley and Sons, Hoboken (1987)
Newey, W., Powell, J.: Asymmetric least squares estimation and testing. Econometrica. 55(4), 819–847 (1987)
Ma, Y., Li, L., Huang, X., Wang, S.: Robust support vector machine using least median loss penalty. In: IFAC Proceedings Volumes (18th IFAC World Congress), vol. 44, no. 1, pp. 11208–11213 (2011). https://doi.org/10.3182/20110828-6-IT-1002.03467
Shibzukhov, Z.M.: Correct aggregate operations with algorithms. Pattern Recogn. Image Anal. 24(3), 377–382 (2014). https://doi.org/10.1134/S1054661814030171
Shibzukhov, Z.M.: Aggregation correct operations on algorithms. Doklady Math. 91(3), 391–393 (2015). https://doi.org/10.1134/S1064562415030357
Shibzukhov, Z.M.: On the principle of empirical risk minimization based on averaging aggregation functions. Doklady Math. 96(2), 494–497 (2017). https://doi.org/10.1134/S106456241705026X
Beliakov, G., Kelarev, A., Yearwood, J.: Robust artificial neural networks and outlier detection. Technical report. arxiv:1110.0169v1 [math.OC] 20 Oct 2011. https://doi.org/10.1080/02331934.2012.674946
Acknowledgments
This work is supported by the RFBR grant 18-01-00050.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Shibzukhov, Z.M. (2018). Robust Neural Networks Learning: New Approaches. In: Huang, T., Lv, J., Sun, C., Tuzikov, A. (eds) Advances in Neural Networks – ISNN 2018. ISNN 2018. Lecture Notes in Computer Science(), vol 10878. Springer, Cham. https://doi.org/10.1007/978-3-319-92537-0_29
Download citation
DOI: https://doi.org/10.1007/978-3-319-92537-0_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-92536-3
Online ISBN: 978-3-319-92537-0
eBook Packages: Computer ScienceComputer Science (R0)