Advertisement

Clustering and Neural Networks

  • Hans-Hermann Bock
Part of the Studies in Classification, Data Analysis, and Knowledge Organization book series (STUDIES CLASS)

Abstract

This paper considers the usage of neural networks for the construction of clusters and classifications from given data and discusses, conversely, the use of clustering methods in neural network algorithms. We survey related work in the fields of k-means clustering, stochastic approximation, Kohonen maps, Hopfield networks and multi-layer perceptrons. We propose various new approaches, reveal the asymptotic behaviour of Kohonen maps, and point to possible extensions.

Key words

Clustering Methods Neural Networks Asymptotics for Kohonen Maps Hopfield networks Multi-Layer-Perceptrons Gravitational Clustering 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adorf, H.-M., Murtagh, F. (1988): Clustering based on neural network processing. In: D. Edwards, N.E. Raun (eds.): COMPSTAT 1988. Physica Verlag, Heidelberg, 1988, 239–244.Google Scholar
  2. Anouar, F., Badran, F., Thiria, S. (1996): Topological maps for mixture density. Intern. Conf. on Artificial Neural Networks, Bochum, Germany, ICANN’96.Google Scholar
  3. Anouar, F., Badran, F., Thiria, S. (1997): Cartes topologiques et nuées dynamiques. In: Thiria, S., et al. (eds.) (1997), 190–206.Google Scholar
  4. Ambroise, Ch., Govaert, G. (1996): Constrained clustering and Kohonen self-organizing maps. J. of Classification 13, 299–313.CrossRefGoogle Scholar
  5. Badran, F., Daigremont, Ph., Thiria, S. (1997): Régression par carte topologique. In: Thiria, S., et al. (eds.) (1997), 207–222.Google Scholar
  6. Bezdek, J.C. (1981): Pattern recognition with fuzzy objective function algorithms. Plenum Press, New York.Google Scholar
  7. Bock, H.H (1974): Automatische Klassifikation. Vandenhoeck & Ruprecht, Göttingen, 1974, 480 pp.Google Scholar
  8. Bock, H.H (1979): Fuzzy clustering procedures. In: R. Tommassonne (ed.): Analyse de données et informatique. INRIA, Le Chesnay, France, 1979, 205–218.Google Scholar
  9. Bock, H.H. (1985): On some significance tests in cluster analysis. J. of Classification 2, 77–108.CrossRefGoogle Scholar
  10. Bock, H.H. (1996a): Probability models and hypotheses testing in partitioning cluster analysis. In: Ph. Arabie, L. Hubert, G. De Soete (eds.): Clustering and classification. World Scientific, River Edge, NJ, 1996, 377–453.Google Scholar
  11. Bock, H.H. (1996b): Probabilistic models in partitional cluster analysis. In: A. Ferligoj, A. Kramberger (eds.): Developments in data analysis. FDV, Metodoloski zvezki, 12, Ljubljana, 1996, 3–25.Google Scholar
  12. Bock, H.H. (1996c): Probabilistic models in cluster analysis. Computational Statistics and Data Analysis 23, 5–28.CrossRefGoogle Scholar
  13. Bock, H.H. (1997): Simultaneous visualization and clustering methods as an alternative to Kohonen maps. In: G. Della Riccia et al. (eds.), 1997, 67–85.Google Scholar
  14. Bock, H.H. (1998a): Probabilistic aspects in classification. In: Hayashi, Ch., et al., Proc. IFCS-96, 1998, 3–21.Google Scholar
  15. Bock, H.H. (1998b): Clustering and neural network approaches. In: H. Locarek-Junge et al. (eds.): Proc. 22th Annual Conference of the Gesellschaft für Klassifikation, University of Dresden, 4-6 March 1998. Springer-Verlag, Heidelberg (submitted).Google Scholar
  16. Bouton, C., and G. Pages (1993): Selforganization and convergence of the one-dimensional Kohonen algorithm with non-uniformly distributed stimuli. Stochastic Processes and Applications 47 (1993) 249–274.CrossRefGoogle Scholar
  17. Braverman, E.M. (1966): The method of potential functions in the problem of training machines to recognize patterns without a teacher. Automation and Remote Control 27, 1748–1771.Google Scholar
  18. Butler, G.A. (1969): A vector field approach to cluster analysis. Pattern Recognition 1, 291–299.CrossRefGoogle Scholar
  19. Carroll, J.D., Chaturvedi, A. (1998): Fitting the CANDCLUS/MUMCLUS models with partitioning and other constraints. In: Hayashi, Ch., et al., Proc. IFCS-96, 1998, 496–505.Google Scholar
  20. Coleman, J.S. (1971): Clustering in n dimensions by use of a system of forces. Journal of Mathematical Sociology 1, 1–47.CrossRefGoogle Scholar
  21. Cottrell, M., Fort, J.-C. (1989): Etude d’un processus d’auto-organisation. Ann. Inst. Henri Poincaré Probab. Statist. 23, 1–20.Google Scholar
  22. Della Riccia, G., et al. (eds.) (1997):: Learning, networks and statistics. CISM Courses and Lectures no. 382. Springer, Wien - New York, 1997.Google Scholar
  23. Firmin, Ch., and D. Hamad (1995): Gaussian neural networks applied to cluster analysis problem. In: W. Gaul and D. Pfeifer (eds.): From information to knowledge. Springer-Verlag, Heidelberg, 159–166.Google Scholar
  24. Fort, J.-C., Pages, G. (1996): About the Kohonen algorithm: strong or weak self-or- ganisation? Neural Networks 9, 773–785.CrossRefGoogle Scholar
  25. Hayashi, Ch., Ohsumi, N., Yajima, K., Tanaka, Y., Bock, H.-H., Baba, Y. (eds.): Data science, classification and related methods. Proc. IFCS-96. Springer-Verlag, Tokyo, 1998.Google Scholar
  26. Grossberg, S. (ed.) (1987): The adaptive brain II: Vision, speech, language, and motor control. Elsevier, Amsterdam.Google Scholar
  27. Hopfield, J.J. (1982): Neural networks and physical systems with emergent collective computational capabilities. Proc. Natl. Acad. Sci. USA 79, 2554–2558.CrossRefGoogle Scholar
  28. Hopfield, J.J., Tank, D.W. (1985): Neural computation of decisions in optimization problems. Biological Cybernetics 52, 141–152.Google Scholar
  29. Kamgar-Parsi, B., J.A. Gualtieri, J.E. Devaney and B. Kamgar-Parsi (1990): Clustering with neural networks. Biological Cybernetics 63, 201–208.CrossRefGoogle Scholar
  30. Kohonen, T. (1982): Self-organized formation of topologically correct feature maps. Biological Cybernetics 43, 59–69.CrossRefGoogle Scholar
  31. Kohonen, T. (1995): Self-organizing maps. Springer, New York.Google Scholar
  32. Kohonen, T., S. Kaski, H. Lappalainen, J. Salojärvi (1997): The adaptive-subspace self-organizing map (ASSOM). Workshop on Neural Networks, Helsinki, Website.Google Scholar
  33. Kovalenko, A. (1996):: A multi-layer neural network algorithm for high-density cluster analysis. Paper presented at IFCS-96, Kobe, Japan.Google Scholar
  34. MacQueen, J. (1967): Some methods for classification and analysis of multivariate observations. In: L.M. LeCam et al. (eds): Proc. 5th Berkely Symp. on Math. Stat. Probab. Univ. of California Press, Los Angeles, vol. 1, 281–297.Google Scholar
  35. Moshou, D., H. Ramon (1997): Extended self-organizing maps with local linear mappings for function approximation and system identification. Workshop on Neural Networks, Helsinki, Website.Google Scholar
  36. Murtagh, F. (1995): The Kohonen self-organizing map method: an assessment. J. of Classification 12, 165–190.CrossRefGoogle Scholar
  37. Murtagh, F. (1996): Neural networks for clustering. In: Ph. Arabie, L. Hubert, G. De Soete (eds.): Clustering and classification. World Scientific, River Edge, NJ, 1996, 235–269.Google Scholar
  38. Pollard, D. (1981): Strong consistency of k-means clustering. Annals of Probab. 10, 919–926.CrossRefGoogle Scholar
  39. Pollard, D. (1982): Quantization and the method of k-means. IEEE Trans. Information Theory IT-28, 199–205.Google Scholar
  40. Postma, E.O., Hudson, P.T.W. (1995): Adaptive resonance theory. In: P.J. Braspenning, F. Thuijsman, A.J.M.M. Weijters (eds.): Artificial neural networks. Springer, Berlin, 1995.Google Scholar
  41. Ritter, H. (1997): Neural networks for rapid learning in computer vision and robotics. In: G. Delia Riccia et al. (eds.), 1997, 25–39.Google Scholar
  42. Sato, M., Sato, Y. (1995): Neural clustering: Implementation of clustering model using neural networks. Proc. IEEE Conference, 1995, 3609–3614.Google Scholar
  43. Sato, M., Sato, Y. (1998): Additive clustering model and its generalizations. In: Ch. Hayashi et al., Proc. IFCS-96, 1998, 312–319.Google Scholar
  44. Shepard, R.N., Arabie, Ph. (1979): Additive clustering: Representation of similarities as combinations of discrete overlapping properties. Psychological Reviews 86, 87–123.CrossRefGoogle Scholar
  45. Thiria, S., Lechevallier, Y., Gascuel, O., Canu, S. (1997): Statistique et méthodes neuronales. Dunod, Paris, 311 pp.Google Scholar
  46. Tolat, V.V. (1990): An analysis of Kohonen’s self-organizing maps using a system of energy functions. Biological Cybernetics 64, 155–164.CrossRefGoogle Scholar
  47. Tsypkin, Y.Z., Kelmans, G.K. (1967): Recursive self-training algorithms. Engineering Cybernetics USSR, 1967, V, 70–79.Google Scholar
  48. Wright, W.E. (1977): Gravitational clustering. Pattern Recognition 9, 151–166.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin · Heidelberg 1998

Authors and Affiliations

  • Hans-Hermann Bock
    • 1
  1. 1.Institut für StatistikTechnische Hochschule AachenAachenGermany

Personalised recommendations