Abstract
Many real life problems require the classification of items into naturally ordered classes. These problems are traditionally handled by conventional methods intended for the classification of nominal classes, where the order relation is ignored. This paper proposes a hybrid neural network model applied to ordinal classification using a possible combination of projection functions (product unit, PU) and kernel functions (radial basis function, RBF) in the hidden layer of a feed-forward neural network. A combination of an evolutionary and a gradient-descent algorithms is adapted to this model and applied to obtain an optimal architecture, weights and node typology of the model. This combined basis function model is compared to the corresponding pure models: PU neural network, and the RBF neural network. Combined functions using projection and kernel functions are found to be better than pure basis functions for the task of ordinal classification in several datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Asuncion, A., Newman, D.: UCI machine learning repository (2007), http://www.ics.uci.edu/~mlearn/MLRepository.html
Baccianella, S., Esuli, A., Sebastiani, F.: Evaluation measures for ordinal regression. In: Proceedings of the Ninth International Conference on Intelligent Systems Design and Applications (ISDA 2009), Pisa, Italy (December 2009)
Bishop, C.M.: Pattern Recognition and Machine Learning. Information Science and Statistics, 1st edn. Springer, Heidelberg (2006)
Bishop, C.: Improving the generalization properties of radial basis function neural networks. Neural Computation 8, 579–581 (1991)
Castro, L.N., Hruschka, E.R., Campello, R.J.G.B.: An evolutionary clustering technique with local search to design rbf neural network classifiers. In: Proceedings of the IEEE International Joint Conference on Neural Networks, pp. 2083–2088 (2004)
Chauvin, Y., Rumelhart, D.E.: Backpropagation: Theory, Architectures, and Applications. Lawrence Erlbaum Associates, Inc., Mahwah (1995)
Chu, W., Ghahramani, Z.: Gaussian processes for ordinal regression. Journal of Machine Learning Research 6, 1019–1041 (2005)
Cohen, S., Intrator, N.: A hybrid projection-based and radial basis function architecture: initial values and global optimisation. Pattern Analysis & Applications 5, 113–120 (2002)
Demšar, J.: Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, 1–30 (2006)
Donoho, D.: Projection-based approximation and a duality with kernel methods. The Annals of Statistics 5, 58–106 (1989)
Fleiss, J.L., Cohen, J., Everitt, B.S.: Large sample standard errors of kappa and weighted kappa. Psychological Bulletin 72(5), 323–327 (1969)
Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Computer Journal 7, 149–154 (1964)
Friedman, M.: A comparison of alternative tests of significance for the problem of m rankings. Annals of Mathematical Statistics 11(1), 86–92 (1940)
Gutiérrez, P.A., Hervás-Martínez, C., Carbonero-Ruz, M., Fernandez, J.C.: Combined projection and kernel basis functions for classification in evolutionary neural networks. Neurocomputing 27(13-15), 2731–2742 (2009)
Gutiérrez, P.A., Lopez-Granados, F., Peña-Barragán, J.M., Jurado-Expósito, M., Gómez-Casero, M.T., Hervás-Martínez, C.: Mapping sunflower yield as affected by Ridolfia segetum patches and elevation by applying evolutionary product unit neural networks to remote sensed data. Computers and Electronics in Agriculture 60(2), 122–132 (2008)
Hecht-Nielsen, R.: Neurocomputing. Addison-Wesley (1990)
Hervás-Martínez, C., Garcia-Gimeno, R.M., Martínez-Estudillo, A.C., Martínez-Estudillo, F.J., Zurera-Cosano, G.: Improving microbial growth prediction by product unit neural networks. Journal of Food Science 71(2), M31–M38 (2006)
Igel, C., Hüsken, M.: Empirical evaluation of the improved rprop learning algorithms. Neurocomputing 50(6), 105–123 (2003)
Ishibuchi, H., Yoshida, T., Murata, T.: Balance between genetic search and local search in hybrid evolutionary multi-criterion optimization algorithms. IEEE Transactions on Evolutionary Computation 7(2), 204–223 (2003)
Koza, J.R., Rice, J.P.: Genetic generation of both the weights and architecture for a neural network. In: Proceedings of International Joint Conference on Neural Networks, vol. 2, pp. 397–404. IEEE Press, Seattle (1991)
Lee, S.H., Hou, C.L.: An art-based construction of RBF networks. IEEE Transactions on Neural Networks 13(6), 1308–1321 (2002)
Li, L., Lin, H.T.: Ordinal Regression by Extended Binary Classification. In: Advances in Neural Information Processing Systems, vol. 19, pp. 865–872 (2007)
Lievens, S., Baets, B.D.: Supervised ranking in the weka environment. Information Sciences 180(24), 4763–4771 (2010), http://www.sciencedirect.com/science/article/pii/S0020025510002756
Lippmann, R.P.: Pattern classification using neural networks. IEEE Transactions on Neural Networks 27, 47–64 (1989)
Martínez-Estudillo, A.C., Martínez-Estudillo, F.J., Hervás-Martínez, C., García, N.: Evolutionary product unit based neural networks for regression. Neural Networks 19(4), 477–486 (2006)
McCullagh, P.: Regression models for ordinal data (with discussion). Journal of the Royal Statistical Society 42(2), 109–142 (1980)
PASCAL: Pascal (pattern analysis, statistical modelling and computational learning) machine learning benchmarks repository (2011), http://mldata.org/
Schmitt, M.: On the complexity of computing and learning with multiplicative neural networks. Neural Computation 14, 241–301 (2001)
Yao, X.: Evolving artificial neural networks. Proceedings of the IEEE 87(9) (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dorado-Moreno, M., Gutiérrez, P.A., Hervás-Martínez, C. (2012). Ordinal Classification Using Hybrid Artificial Neural Networks with Projection and Kernel Basis Functions. In: Corchado, E., Snášel, V., Abraham, A., Woźniak, M., Graña, M., Cho, SB. (eds) Hybrid Artificial Intelligent Systems. HAIS 2012. Lecture Notes in Computer Science(), vol 7209. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28931-6_31
Download citation
DOI: https://doi.org/10.1007/978-3-642-28931-6_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28930-9
Online ISBN: 978-3-642-28931-6
eBook Packages: Computer ScienceComputer Science (R0)