Abstract
Radial Basis Function (RBF) Kernel Machines have become commonly used in Machine Learning tasks, but they contain certain flaws (e.g., some suffer from fast growth in the number of learning parameters while predicting data with large number of variations). Besides, Kernel Machines with single hidden layers lack mechanisms for feature selection in multidimensional data space, and machine learning tasks become intractable. This paper investigates “deep learning” architecture composed of multilayered adaptive non-linear components – Multilayer RBF Kernel Machine – to address RBF limitations. Three different approaches of features selection and dimensionality reduction to train RBF based on Multilayer Kernel Learning are explored, and comparisons made between them in terms of accuracy, performance and computational complexity. Results show that the multilayered system produces better results than single-layer architecture. In particular, developing decision support system in term of data mining.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Orr, M.J.L.: Introduction to radial basis function network (1996). http://www.cc.gatech.edu/~isbell/tutorials/rbf-intro.pdf
Orr, M.J.L.: Recent advances in radial basis function networks (1999). http://www.anc.ed.ac.uk/rbf/papers/recad.ps
Kubat, M.: Decision trees can initialize radial basis function networks. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.43.6674&rep=rep1&type=pdf
Schwenker, F., Kestler, H.A., Palm, G.: Three learning phases for radial-basis-function networks (2001). http://sci2s.ugr.es/keel/pdf/specific/articulo/skg01.pdf
Kohonen, T.: Learning vector quantization. In: Arbib, M.A. (ed.) The Handbook of Brain Theory and Neural Networks, pp. 537–540. MIT Press, Cambridge (1995)
Schwenker F., Kestler H.A., Palm, G.: 3-D Visual Object Classification with Hierarchical Radial Basis Function Networks. http://www.uni-ulm.de/fileadmin/website_uni_ulm/iui/Ulmer_Informatik_Berichte/2001/UIB_2001-02.pdf
Cortes, C., Vapnik, V.: Support-vector networks (1995). http://image.diku.dk/imagecanon/material/cortes_vapnik95.pdf
Wettschereck, D., Dietterich, D.: Improving the performance of the radial basis function networks by learning center locations. http://papers.nips.cc/paper/544-improving-the-performance-of-radial-basis-function-networks-by-learning-center-locations.pdf
Chen, S., Grant, P.M., Cowan, C.F.N.: Orthogonal least-squares algorithm for training multi-output radial basis function networks (1992). https://cours.etsmtl.ca/sys828/REFS/B3/Chen_IEEE1992.pdf
Gomm, B.J., Yu, D.L.: Selecting radial basis function network centers with recursive orthogonal least-squares training (2000). http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=839002
De Castro, L.N., Von Zuben, F.J.: An immunological approach to initialize centers of radial basis function neural networks (2001). http://www.dca.fee.unicamp.br/~vonzuben/research/lnunes_dout/artigos/cbrn01.pdf
Yousef, R., el Hindi, K.: Training radial basis function networks using reduced sets as center points. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.131.4275&rep=rep1&type=pdf
Vachkov, G., Stoyanov, V., Christova, N.: Growing RBF network models for solving nonlinear approximation and classification problems (2015). http://www.scs-europe.net/dlib/2015/ecms2015acceptedpapers/0481-is_ECMS2015_0053.pdf
Billing, A.S., Zheng, G.L.: Radial basis function network configuration using genetic algorithms. http://www.sciencedirect.com/science/article/pii/089360809500029Y
Fasshauer, G.E., Zhang, J.G.: On choosing “optimal” shape parameters for RBF approximation. http://www.math.iit.edu/~fass/Dolomites.pdf
Hoffmann, G.A.: Adaptive transfer functions in radial basis function (RBF) networks. In: Bubak, M., Albada, G.D., Sloot, Peter M.A., Dongarra, J. (eds.) ICCS 2004. LNCS, vol. 3037, pp. 682–686. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24687-9_102. http://www.citemaster.net/get/a719201c-fb59-11e3-8a7f-00163e009cc7/hoffmann04adaptive.pdf
Duch, W., Jankowski, N.: Transfer functions: hidden probabilities for better neural networks. https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2001-400.pdf
Dorffner, G.: A unified framework for MLPs and RBFNs: introducing conic section functions networks. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.49.7264&rep=rep1&type=pdf
Mongillo, M.: Choosing basis functions and shape parameters for radial basis function methods. https://www.siam.org/students/siuro/vol4/S01084.pdf
Webb, R.A., Shennon, S.: Shape-adaptive radial basis functions. http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=728359
Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. http://www.cl.uni-heidelberg.de/courses/ws14/deepl/BengioETAL12.pdf
Wang, X.: The application of deep kernel machines to various types of data. https://uwaterloo.ca/computational-mathematics/sites/ca.computational-mathematics/files/uploads/files/shirly_project.pdf
Bengio, Y., LeCun, Y.: Scaling learning algorithms towards AI. http://cseweb.ucsd.edu/~gary/cs200/s12/bengio-lecun-07.pdf
Bengio, Y.: Learning deep architecture for AI. http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf
Cho, Y.: Kernel methods for deep learning. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.387.4491&rep=rep1&type=pdf
LeCun, Y., Bengio, Y.: Convolutional networks for images, speech and time-series. http://yann.lecun.com/exdb/publis/pdf/lecun-bengio-95a.pdf
Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked denoising auto-encoders: learning useful representations in a deep network using local denoising criteria. http://www.jmlr.org/papers/volume11/vincent10a/vincent10a.pdf
Hinton, G.E., Osindero, S., Teh, Y.-W.: A fast learning algorithm for deep-belief nets. https://www.cs.toronto.edu/~hinton/nipstutorial/nipstut3.pdf
Cho, Y., Saul, K.: Large margin classification in infinite neural networks. http://cseweb.ucsd.edu/~yoc002/paper/neco_arccos.pdf
Bouvrie, J., Rosasco, L., Poggio, T.: On invariance in hierarchical models. http://papers.nips.cc/paper/3732-on-invariance-in-hierarchical-models.pdf
Bo, L., Ren, X., Fox, D.: Kernel descriptors for visual recognition. http://www.cs.washington.edu/robotics/postscripts/kdes-nips-10.pdf
Bo, L., Lai, K., Ren, X., Fox, D.: Object recognition with hierarchical kernel descriptors. http://www.cs.washington.edu/robotics/postscripts/hkdes-cvpr-11.pdf
Mairal, J., Koniusz, P., Harchaoui, Z., Schmid, C.: Convolutional kernel networks. http://arxiv.org/pdf/1406.3332v2.pdf
Zhuang, J., Tsang, I.W., Hoi, S.C.H.: Two-layer multiple kernel learning. http://jmlr.csail.mit.edu/proceedings/papers/v15/zhuang11a/zhuang11a.pdf, http://www.di.ens.fr/~fbach/skm_icml.pdf
Huang, P.-S., Avron, H., Sainath, T.N., Sindhvani, V., Ramabhadran, B.: Kernel methods match deep neural networks on TIMIT. http://www.ifp.illinois.edu/~huang146/papers/Kernel_DNN_ICASSP2014.pdf
Jose, C., Goyal, P., Aggrwal, P., Varma, M.: Local deep kernel learning for efficient non-linear SVM prediction. http://research.microsoft.com/en-us/um/people/manik/pubs%5Cjose13.pdf
Yger, F., Berar, M., Gasso, G., Rakotomamonjy, A.: A supervised strategy for deep kernel machine. https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2011-21.pdf
Schölkopf, B., Smola, A., Müller, K.-R.: Kernel principal component analysis. In: Gerstner, W., Germond, A., Hasler, M., Nicoud, J.-D. (eds.) ICANN 1997. LNCS, vol. 1327, pp. 583–588. Springer, Heidelberg (1997). doi:10.1007/BFb0020217. http://www.ics.uci.edu/~welling/classnotes/papers_class/Kernel-PCA.pdf
Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering analysis and an algorithm. http://ai.stanford.edu/~ang/papers/nips01-spectral.pdf
Welling, M.: Kernel canonical correlation analysis. http://www.ics.uci.edu/~welling/classnotes/papers_class/kCCA.pdf
Wiering, M.A., Schutten, M., Millea, A., Meijster, A., Schomaker, L.R.B.: Deep support vector machines for regression problems. http://www.ai.rug.nl/~mwiering/GROUP/ARTICLES/DSVM_extended_abstract.pdf
Takeda, H., Farsiu, S., Milanfar, P.: Kernel regression for image processing and reconstruction. http://people.duke.edu/~sf59/KernelRegression_Final.pdf
Unsupervised kernel regression for non-linear dimensionality reduction. http://www.iro.umontreal.ca/~memisevr/pubs/ukr.pdf
Memisevic, R.: Unsupervised kernel dimension reduction. http://www.cs.berkeley.edu/~jordan/papers/wang-sha-jordan-nips11.pdf
https://archive.ics.uci.edu/ml/datasets.html, https://physionet.org/physiobank/database/#multi
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Alsalamah, M., Amin, S. (2017). Multilayer Radial Basis Function Kernel Machine. In: Perego, P., Andreoni, G., Rizzo, G. (eds) Wireless Mobile Communication and Healthcare. MobiHealth 2016. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 192. Springer, Cham. https://doi.org/10.1007/978-3-319-58877-3_34
Download citation
DOI: https://doi.org/10.1007/978-3-319-58877-3_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-58876-6
Online ISBN: 978-3-319-58877-3
eBook Packages: Computer ScienceComputer Science (R0)