Kernel Robust Soft Learning Vector Quantization

  • Daniela Hofmann
  • Barbara Hammer
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7477)


Prototype-based classification schemes offer very intuitive and flexible classifiers with the benefit of easy interpretability of the results and scalability of the model complexity. Recent prototype-based models such as robust soft learning vector quantization (RSLVQ) have the benefit of a solid mathematical foundation of the learning rule and decision boundaries in terms of probabilistic models and corresponding likelihood optimization. In its original form, they can be used for standard Euclidean vectors only. In this contribution, we extend RSLVQ towards a kernelized version which can be used for any positive semidefinite data matrix. We demonstrate the superior performance of the technique, kernel RSLVQ, in a variety of benchmarks where results competitive or even superior to state-of-the-art support vector machines are obtained.


Support Vector Machine Learning Rule Vector Quantization Learn Vector Quantization Learn Vector Quantization Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Biehl, M., Ghosh, A., Hammer, B.: Dynamics and generalization ability of LVQ algorithms. Journal of Machine Learning Research 8, 323–360 (2007)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Biehl, M., Hammer, B., Verleysen, M., Villmann, T. (eds.): Similarity-Based Clustering. LNCS (LNAI), vol. 5400. Springer, Heidelberg (2009)Google Scholar
  3. 3.
    Boulet, R., Jouve, B., Rossi, F., Villa, N.: Batch kernel SOM and related Laplacian methods for social network analysis. Neurocomputing 71(7-9), 1257–1273 (2008)CrossRefGoogle Scholar
  4. 4.
    Chen, Y., Garcia, E.K., Gupta, M.R., Rahimi, A., Cazzanti, L.: Similarity-based classification: Concepts and algorithms. JMLR 10, 747–776 (2009)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Cottrell, M., Hammer, B., Hasenfuss, A., Villmann, T.: Batch and median neural gas. Neural Networks 19, 762–771 (2006)zbMATHCrossRefGoogle Scholar
  6. 6.
    Frasconi, P., Gori, M., Sperduti, A.: A general framework for adaptive processing of data structures. IEEE TNN 9(5), 768–786 (1998)Google Scholar
  7. 7.
    Frey, B.J., Dueck, D.: Clustering by passing messages between data points. Science 315, 972–976 (2007)MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Gärtner, T.: Kernels for Structured Data. PhD thesis, Univ. Bonn (2005)Google Scholar
  9. 9.
    Hammer, B., Hasenfuss, A.: Topographic mapping of large dissimilarity datasets. Neural Computation 22(9), 2229–2284 (2010)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Hammer, B., Micheli, A., Sperduti, A.: Universal approximation capability of cascade correlation for structures. Neural Computation 17, 1109–1159 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Hammer, B., Villmann, T.: Generalized relevance learning vector quantization. Neural Networks 15(8-9), 1059–1068 (2002)CrossRefGoogle Scholar
  12. 12.
    Kohonen, T.: Self-Oganizing Maps, 3rd edn. Springer (2000)Google Scholar
  13. 13.
    Kohonen, T., Somervuo, P.: How to make large self-organizing maps for nonvectorial data. Neural Networks 15(8-9), 945–952 (2002)CrossRefGoogle Scholar
  14. 14.
    Pekalska, E., Duin, R.P.: The Dissimilarity Representation for Pattern Recognition. Foundations and Applications. World Scientific (2005)Google Scholar
  15. 15.
    Qin, A.K., Suganthan, P.N.: Kernel neural gas algorithms with application to cluster analysis. In: Proceedings of the 17th International Conference on Pattern Recognition (ICPR 2004), vol. 4, pp. 617–620. IEEE Computer Society, Washington, DC (2004)CrossRefGoogle Scholar
  16. 16.
    Qin, A.K., Suganthan, P.N.: A novel kernel prototype-based learning algorithm. In: Proc. of the 17th International Conference on Pattern Recognition (ICPR 2004), Cambridge, UK (August 2004)Google Scholar
  17. 17.
    Rossi, F., Villa-Vialaneix, N.: Consistency of functional learning methods based on derivatives. Pat. Rec. Letters 32(8), 1197–1209 (2011)CrossRefGoogle Scholar
  18. 18.
    Sato, A., Yamada, K.: Generalized Learning Vector Quantization. In: NIPS (1995)Google Scholar
  19. 19.
    Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: Computational capabilities of graph neural networks. IEEE TNN 20(1), 81–102 (2009)Google Scholar
  20. 20.
    Schneider, P., Biehl, M., Hammer, B.: Distance learning in discriminative vector quantization. Neural Computation 21, 2942–2969 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  21. 21.
    Seo, S., Obermayer, K.: Soft learning vector quantization. Neural Comput. 15, 1589–1604 (2003)zbMATHCrossRefGoogle Scholar
  22. 22.
    van der Maaten, L., Hinton, G.: Visualizing high-dimensional data using t-sne. JMLR 9, 2579–2605 (2008)zbMATHGoogle Scholar
  23. 23.
    Vellido, A., Martin-Guerroro, J.D., Lisboa, P.: Making machine learning models interpretable. In: ESANN 2012 (2012)Google Scholar
  24. 24.
    Williams, C., Seeger, M.: Using the nyström method to speed up kernel machines. In: Advances in Neural Information Processing Systems 13, pp. 682–688. MIT Press (2001)Google Scholar
  25. 25.
    Zhu, X., Gisbrecht, A., Schleif, F.-M., Hammer, B.: Approximation techniques for clustering dissimilarity data. Neurocomputing 90, 72–84 (2012)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Daniela Hofmann
    • 1
  • Barbara Hammer
    • 1
  1. 1.CITEC Center of ExcellenceBielefeld UniversityGermany

Personalised recommendations