Abstract
Unsupervised learning techniques, such as clustering and sparse coding, have been adapted for use with data sets exhibiting nonlinear relationships through the use of kernel machines. These techniques often require an explicit computation of the kernel matrix, which becomes expensive as the number of inputs grows, making it unsuitable for efficient online learning. This paper proposes an algorithm and a neural architecture for online approximated nonlinear kernel clustering using any shift-invariant kernel. The novel model outperforms traditional low-rank kernel approximation based clustering methods, it also requires significantly lower memory requirements than those of popular kernel k-means while showing competitive performance on large data sets.
Keywords
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Oja, E.: Neural networks, principal components, and subspaces. Int. J. Neural Syst. 1(01), 61–68 (1989)
Barlow, H.B.: Unsupervised learning. Neural Comput. 1(3), 295–311 (1989)
Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: a strategy employed by V1? Vis. Res. 37(23), 3311–3325 (1997)
Sanger, T.D.: Optimal unsupervised learning in a single-layer linear feedforward neural network. Neural Netw. 2(6), 459–473 (1989)
Plumbley, M.D.: A Hebbian/anti-Hebbian network which optimizes information capacity by orthonormalizing the principal subspace. In: Third International Conference on Artificial Neural Networks, 1993, pp. 86–90. IET (1993)
Pehlevan, C., Chklovskii, D.B.: A Hebbian/anti-Hebbian network derived from online non-negative matrix factorization can cluster and discover sparse features. In: 2014 48th Asilomar Conference on Signals, Systems and Computers, pp. 769–775. IEEE (2014)
Cox, T.F., Cox, M.A.: Multidimensional Scaling. CRC Press, Boca Raton (2000)
Williams, C.K.: On a connection between kernel PCA and metric multidimensional scaling. In: Advances in Neural Information Processing Systems, pp. 675–681 (2001)
Williams, C.K., Seeger, M.: Using the Nyström method to speed up kernel machines. In: Proceedings of the 13th International Conference on Neural Information Processing Systems, pp. 661–667. MIT press (2000)
Rahimi, A., Recht, B.: Random features for large-scale kernel machines. In: Advances in Neural Information Processing Systems, pp. 1177–1184 (2008)
Aronszajn, N.: Theory of reproducing kernels. Trans. Am. Math. Soc. 68(3), 337–404 (1950)
Rudin, W.: Fourier Analysis on Groups. Courier Dover Publications, New York (2017)
Chitta, R., Jin, R., Jain, A.K.: Efficient kernel clustering using random Fourier features. In: IEEE 12th International Conference on Data Mining (ICDM), pp. 161–170. IEEE (2012)
Ding, C., He, X., Simon, H.D.: On the equivalence of nonnegative matrix factorization and spectral clustering. In: Proceedings of the 2005 SIAM International Conference on Data Mining, pp. 606–610. SIAM (2005)
Bahroun, Y., Soltoggio, A.: Online representation learning with single and multi-layer Hebbian networks for image classification tasks. In: Proceedings of the 26th International Conference on Artificial Neural Networks, ICANN 2017. Springer International Publishing (2017, to appear)
Pennington, J., Felix, X.Y., Kumar, S.: Spherical random features for polynomial kernels. In: Advances in Neural Information Processing Systems, pp. 1846–1854 (2015)
Schölkopf, B., Smola, A., Müller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10(5), 1299–1319 (1998)
Chitta, R., Jin, R., Havens, T.C., Jain, A.K.: Approximate kernel k-means: solution to large scale kernel clustering. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 895–903. ACM (2011)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Blackard, J.A., Dean, D.J.: Comparative accuracies of artificial neural networks and discriminant analysis in predicting forest cover types from cartographic variables. Comput. Electron. Agric. 24(3), 131–151 (1999)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Bahroun, Y., Hunsicker, E., Soltoggio, A. (2017). Neural Networks for Efficient Nonlinear Online Clustering. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10634. Springer, Cham. https://doi.org/10.1007/978-3-319-70087-8_34
Download citation
DOI: https://doi.org/10.1007/978-3-319-70087-8_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70086-1
Online ISBN: 978-3-319-70087-8
eBook Packages: Computer ScienceComputer Science (R0)