Abstract
We explore four local learning versions of regularization networks. While global learning algorithms create a global model for all testing points, the local learning algorithms use neighborhoods to learn local parameters and create on the fly a local model specifically designed for any particular testing point. This approach delivers breakthrough performance in many application domains. Usually however the computational overhead is substantial, and in some cases prohibited. For speeding up the online predictions we exploit both multithreaded parallel implementations as well as interplay between locally optimized parameters and globally optimized parameters. The multithreaded local learning regularization neural networks are implemented with OpenMP. The accuracy of the algorithms is tested against several benchmark datasets. The parallel efficiency and speedup is evaluated on a multi-core system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Poggio, T., Girosi, F.: Regularization algorithms for learning that are equivalent to multilayer networks. Science 247, 978–982 (1990)
Girosi, F., Jones, M., Poggio, T.: Regularization theory and neural networks architectures. Neural Computation 7, 219–269 (1995)
Evgeniou, T., Pontil, M., Poggio, T.: Regularization Networks and Support Vector Machines. Advances in Computational Mathematics 13, 1–50 (2000)
Poggio, T., Smale, S.: The mathematics of learning: Dealing with data. Notices of the American Mathematical Society 50(5), 537–544 (2003)
Bottou, L., Vapnik, V.: Local learning algorithms. Neural Computation 4(6), 888–900 (1992)
Vapnik, V., Bottou, L.: Local Algorithms for Pattern Recognition and Dependencies Estimation. Neural Computation 5(6), 893–909 (1993)
Robins, A., Frean, M.: Local Learning Algorithms for Sequential Tasks in Neural Networks. Advanced Computational Intelligence 2, 221–227 (1998)
Vijayakumar, S., Schaal, S.: Local Adaptive Subspace Regression. Neural Processing Letters 7(3), 139–149 (1998)
Vijayakumar, S., Schaal, S.: Locally weighted projection regression: an O(n) algorithm for incremental real time learning in high dimensional space. In: Proceedings of Seventeenth International conference on Machine Learning (ICML 2000), pp. 1079–1086 (2000)
Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. Advances in Neural Information Processing Systems 16 (2004)
Wu, M., Schölkopf, B.: Transductive classification via local learning regularization. In: Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics (2007)
Wu, M., Yu, K., Yu, S., Schölkopf, B.: Local learning projections. In: Proceedings of the Twenty-Fourth International Conference on Machine Learning (2007)
Blanzieri, E., Melgani, F.: An adaptive SVM nearest neighbor classifier for remotely sensed imagery. In: Proceedings of IEEE International Conference on Geoscience and Remote Sensing Symposium (IGARSS 2006), pp. 3931–3934 (2006)
Zhang, H., Berg, A.C., Maire, M., Malik, J.: SVM-KNN: discriminative nearest neighbor classification for visual category recognition. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 2126–2136 (2006)
Yang, T., Kecman, V.: Adaptive local hyperplane classification. Neurocomputing 71, 3001–3004 (2008)
Wu, M., Schölkopf, B.: A local learning approach for clustering. In: Advances in Neural Information Processing Systems, vol. 19 (2006)
Wang, F., Zhang, C., Li, T.: Clustering with local and global regularization. In: Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence (2007)
Kokkinos, Y., Margaritis, K.: Parallel and local learning for fast probabilistic neural networks in scalable data mining. In: ACM Proceedings of 6th Balkan Conference in Informatics, (BCI 2013), pp. 47–52 (2013)
Rifkin, R.M., Lippert, R.A.: Notes on regularized least squares. Technical report, MIT Computer Science and Artificial Intelligence Laboratory (2007)
Buyya, R. (ed.): High Performance Cluster Computing: Programming and Applications, vol. 2. Prentice Hall, New Jersey (1999)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Kokkinos, Y., Margaritis, K.G. (2015). Multithreaded Local Learning Regularization Neural Networks for Regression Tasks. In: Iliadis, L., Jayne, C. (eds) Engineering Applications of Neural Networks. EANN 2015. Communications in Computer and Information Science, vol 517. Springer, Cham. https://doi.org/10.1007/978-3-319-23983-5_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-23983-5_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-23981-1
Online ISBN: 978-3-319-23983-5
eBook Packages: Computer ScienceComputer Science (R0)