Advertisement

Measures for the Organization of Self-Organizing Maps

  • Daniel Polani
Part of the Studies in Fuzziness and Soft Computing book series (STUDFUZZ, volume 78)

Abstract

The “self-organizing” dynamics of Self-Organizing Maps (SOMs) is a prominent property of the model that is intuitively very accessible. Nevertheless, a rigorous definition of a measure for the state of organization of a SOM that is also natural, captures the intuitive properties of organization and proves to be useful in practice, is quite difficult to formulate. The goal of this chapter is to give an overview over the relevant problems in and different approaches towards the development of organization measures for SOMs.

Keywords

Energy Function Receptive Field Input Space Learning Rule Delaunay Triangulation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Bibliography on Chapter 2

  1. Abraham, R., Marsden, J. E., and Ratiu, T., (1983). Manifolds, Tensor Analysis, and Applications. Global Analysis, Pure and Applied. Addison-Wesley.MATHGoogle Scholar
  2. Alder, M., Togneri, R., and Attikiouzel, Y., (1991). Dimension of the speech space. Proceedings of the IEE-1, 138 (3): 207–214.CrossRefGoogle Scholar
  3. Ashby, W. R., (1947). Principles of the self-organizing dynamic system. J. Gen. Psychol., 37: 125–128.CrossRefGoogle Scholar
  4. Bauer, H.-U., Herrmann, M., and Villmann, T., (1999). Neural maps and topographic vector quantization. Neural Networks, 12 (4–5): 659–676.CrossRefGoogle Scholar
  5. Bauer, H.-U., and Pawelzik, K. R., (1992). Quantifying the Neighbourhood Preservation of Self-Organizing Feature Maps. IEEE Transactions on Neural Networks, 3 (4): 570–579.CrossRefGoogle Scholar
  6. Bezdek, J. C., and Pal, N. R., (1995). Index of Topological Preservation for Feature Extraction. Pattern Recognition, 28 (3): 381–91.CrossRefGoogle Scholar
  7. Böhme, D., (1994). Entwicklung eines Stetigkeitsmaßes für die Einbettung von Kohonen-Netzen. Diplomarbeit, Universität Mainz, Institut für Informatik.Google Scholar
  8. Cottrell, M., Fort, J., and Pagès, G., (1994). Two or three things that we know about the Kohonen algorithm In Verleysen, M., editor, Proceedings of the European Symposium on Artificial Neural Networks (ESANN),235–244. Brussels.Google Scholar
  9. Cottrell, M., and Fort, J.-C., (1987). Étude d’un processus d’auto-organisation. Ann. Inst. Henri Poincaré, 23 (1): 1–20.MathSciNetMATHGoogle Scholar
  10. Demartines, P., and Blayo, F., (1992). Kohonen self-organizing maps: Is the normalization necessary? Complex Systems, 6 (2): 105–123.MATHGoogle Scholar
  11. Demartines, P., and Hérault, J., (1995). Curvilinear Component Analysis. In Quinzième Coloque Gretsi. Juan-les-Pins.Google Scholar
  12. Der, R., and Herrmann, M., (1993). Phase Transitions in Self-Organized Maps. In Gielen, S., and Kappen, B., editors, Proc. ICANN’93, International Conference on Artificial Neural Networks, 597–600. London, UK: Springer.Google Scholar
  13. Erwin, E., Obermayer, K., and Schulten, K., (1992a). Self-Organizing Maps: order- ing, convergence properties and energy functions. Biol. Cybern., 67: 47–55.CrossRefMATHGoogle Scholar
  14. Erwin, E., Obermayer, K., and Schulten, K., (1992b). Self-organizing maps: sta- tionary states, metastability and convergence rate. Biol. Cybern., 67: 35–45.CrossRefMATHGoogle Scholar
  15. Forster, O., (1984). Analysis, vol. 3. Braunschweig/Wiesbaden: Vieweg.CrossRefGoogle Scholar
  16. Goodhill, G., Finch, S., and Sejnowski, T., (1995). Quantifying neighborhood preservation in topographic mappings. Technical Report Series INC-9509, Institute for Neural Computation. http://www.giccs.georgetown.edu/geoff/pubs.html,January 2001
  17. Goodhill, G., and Sejnowski, T., (1997a). Objective functions for topography: a comparison of optimal maps. In Bullinaria, J. A., Glasspool, D. G., and Houghton, G., editors, Proceedings of the Fourth Neural Computation and Psychology Workshop: Connectionist Representations.Google Scholar
  18. Goodhill, G. J., and Sejnowski, T. J., (1997b). A unifying objective function for topographic mappings. Neural Computation, 9: 1291–1304.CrossRefGoogle Scholar
  19. Graepel, T., Burger, M., and Obermayer, K., (1997). Deterministic annealing for topographic vector quantization and self-organizing maps. In Proceedings of WSOM’97, Workshop on Self-Organizing Maps, Espoo, Finland, June 4–6, 345–350. Espoo, Finland: Helsinki University of Technology, Neural Networks Research Centre.Google Scholar
  20. Graepel, T., Burger, M., and Obermayer, K., (1998). Self-organizing maps: generalizations and new optimization techniques. Neurocomputing, 21 (1–3): 173–90.CrossRefMATHGoogle Scholar
  21. Haken, H., (1983). Advanced synergetics. Berlin: Springer-Verlag.CrossRefMATHGoogle Scholar
  22. Haykin, S., (1999). Neural networks: a comprehensive foundation. Prentice Hall.Google Scholar
  23. Henle, M., (1979). A Combinatorial Introduction to Topology. W. H. Freeman. Heskes, T., ( 1999 ). Energy functions for self-organizing maps. In Oja, E., andGoogle Scholar
  24. Kaski, S., editors, Kohonen Maps,303–316. Amsterdam: Elsevier. Keywords: self-organising maps, energy functions, soft assignments.Google Scholar
  25. Hulle, M. M. V., (2000). Fathful Representations and Topographic Maps. Wiley.Google Scholar
  26. Hulle, M. M. V., and Martinez, D., (1993). On an unsupervised learning rule for scalar quantization following the maximum entropy principle. Neural Computation, 5: 939–953.CrossRefGoogle Scholar
  27. Huile, V., (1997). Topology-preserving map formation achieved with a purely local unsupervised competitive learning rule. Neural Networks, 10 (3): 431–446.CrossRefGoogle Scholar
  28. Jetschke, G., (1989). Mathematik der Selbstorganisation. Braunschweig: Vieweg.CrossRefMATHGoogle Scholar
  29. Kaski, S., and Lagus, K., (1996). Comparing Self-Organizing Maps. In von der Malsburg, C., von Seelen, W., Vorbrüggen, J. C., and Sendhoff, B., editors, Proceedings of ICANN96, vol. 1112 of Lecture Notes in Computer Science, 809–814.Google Scholar
  30. Kohonen, T., (1989). Self-Organization and Associative Memory, vol. 8 of Springer Series in Information Sciences. Berlin, Heidelberg, New York: Springer-Verlag. Third edition.CrossRefGoogle Scholar
  31. Linsker, R., (1988). Self-Organization in a Perceptual Network. Computer, 21 (3): 105–117.CrossRefGoogle Scholar
  32. Luttrell, S., (1989). Self-Organization: a derivation from first principles of a class of learning algorithms. In Proceedings 3rd IEEE Int. Joint Conf. on Neural Networks, vol. 2, 495–498. IEEE Neural Networks Council, Washington.Google Scholar
  33. Martinetz, T., and Schulten, K., (1993). Competitive Hebbian Rule Forms Delaunay Triangulations, Perfect Topology Preserving Maps and Discrete Models of Manifolds. Technical Report UIUC-BI-TB-93–04, Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign.Google Scholar
  34. Martinetz, T., and Schulten, K., (1994). Topology Representing Networks. Neural Networks, 7 (2).Google Scholar
  35. McInerney, M., and Dhawan, A., (1994). Training the Self-Organizing Feature Map using Hybrids of Genetic and Kohonen Methods. In Proc. ICNN’94, Int. Conf. on Neural Networks, 641–644. Piscataway, NJ: IEEE Service Center.Google Scholar
  36. Mees, A. I., (1981). Dynamics of feedback systems. John Wiley liu sons, Ltd.MATHGoogle Scholar
  37. Mehler, F., (1994). Selbstorganisierende Karten in Spracherkennungssystemen. Dissertation, Institut für Informatik, Johannes Gutenberg-Universität Mainz.Google Scholar
  38. Minamimoto, K., Ikeda, K., and Nakayama, K., (1993). Topology Analysis of Data Space Using Self-Organizing Feature Maps. In Proc. ICNN’95, IEEE Intl. Conf. on Neural Networks, vol. II, 789–794.Google Scholar
  39. Okabe, A., Boots, B., and Sugihara, K., (1992). Spatial Tesselations, Concepts and Applications of Voronoi Diagrams. Wiley.Google Scholar
  40. O’Neill, B., (1983). Semi-Riemannian Geometry. Pure and applied mathematics. San Diego: Academic Press.Google Scholar
  41. Polani, D., (1995). On the Choice of Organization Measures for Self-Organizing Feature Maps. Technical Report 1/95, Institut für Informatik, Universität Mainz.Google Scholar
  42. Polani, D., (1996). Adaption der Topologie von Kohonen-Karten durch Genetische Algorithmen, vol. 143 of Dissertationen zur Künstlichen Intelligenz. Infix. (In German).Google Scholar
  43. Polani, D., (1997a). Fitness Functions for the Optimization of Self-Organizing Maps. In Bäck, T., editor, Proceedings of the Seventh International Conference on Genetic Algorithms, 776–783. Morgan Kaufmann.Google Scholar
  44. Polani, D., (1997b). Organization Measures for Self-Organizing Maps. In Kohonen, T., editor, Proceedings of the Workshop on Self-Organizing Maps (WSOM 87), 280–285. Helsinki University of Technology.Google Scholar
  45. Polani, D., (1999). On the Optimization of Self-Organizing Maps by Genetic Algorithms. In Oja, E., and Kaski, S., editors, Kohonen Maps. Elsevier.Google Scholar
  46. Polani, D., and Uthmann, T., (1992). Adaptation of Kohonen Feature Map Topologies by Genetic Algorithms. In R. Männer, and Manderick, B., editors, Parallel Problem Solving from Nature, 2, 421–429. Elsevier Science Publishers B. V.Google Scholar
  47. Polani, D., and Uthmann, T., (1993). Training Kohonen Feature Maps in different Topologies: an Analysis using Genetic Algorithms. In Forrest, S., editor, Proceedings of the Fifth International Conference on Genetic Algorithms, 326333. San Mateo, CA: Morgan Kaufmann.Google Scholar
  48. Reichl, L., (1980). A Modern Course in Statistical Physics. Austin: University of Texas Press.Google Scholar
  49. Ritter, H., Martinetz, T., and Schulten, K., (1994). Neuronale Netze. Addison-Wesley.Google Scholar
  50. Ritter, H., and Schulten, K., (1989). Convergence Properties of Kohonen’s Topology Conserving Maps: Fluctuations, Stability and Dimension Selection. Biological Cybernetics 60, 59–71.MathSciNetGoogle Scholar
  51. Sammon, Jr., J. W., (1969). A nonlinear mapping for data structure analysis. IEEE Trans. Comput., C-18: 401–409.Google Scholar
  52. Shalizi, C. R., (1996). Is the Primordial Soup Done Yet?. http://www.santafe.edu/-shalizi/Self-organization/soup-done/, Jan 23, 2001
  53. Speckmann, H., Raddatz, G., and Rosenstiel, W., (1994). Improvement of learning results of the self-organizing map by calculating fractal dimensions. In Verleysen, M., editor, ESANN 84 — Proceedings of the European Symposium on Artificial Neural Networks, 251–255. Brussels, Belgium: D facto.Google Scholar
  54. Spitzner, A., and Polani, D., (1998). Order Parameters for Self-Organizing Maps. In Niklasson, L., Bodén, M., and Ziemke, T., editors, Proc. of the 8th Int. Conf. on Artificial Neural Networks (ICANN 98), Skövde, Sweden,vol. 2, 517–522. Springer.Google Scholar
  55. Torgerson, W. S., (1952). Multidimensional scaling I. Theory and method. Psychometrika, 17: 401–419.MathSciNetCrossRefMATHGoogle Scholar
  56. Van Hulle, M. M., (1997). Nonparametric Density Estimation and Regression Achieved with Topographic Maps Maximizing the Information-Theoretic Entropy of Their Outputs. Biological Cybernetics, 77: 49–61.CrossRefMATHGoogle Scholar
  57. Villmann, T., (1996). Topologieerhaltung in selbstorganisierenden neuronalen Merkmalskarten. PhD thesis, Universität Leipzig.Google Scholar
  58. Villmann, T., (1999). Topology Preservation in Self-Organizing Maps. In Oja, E., and Kaski, S., editors, Kohonen Maps,279–292. Amsterdam: Elsevier. Keywords: self-organising map, topology preservation, growing self-organizing map.Google Scholar
  59. Villmann, T., Der, R., Herrmann, M., and Martinetz, T., (1997). Topology Preservation in Self-Organizing Feature Maps: Exact Definition and Measurement. IEEE Trans. Neural Networks, 8 (2): 256–266.CrossRefGoogle Scholar
  60. Villmann, T., Der, R., and Martinetz, T., (1994a). A New Quantitative Measure of Topology Preservation in Kohonen’s Feature Maps. In Proc. ICNN’94, Int. Conf. on Neural Networks, 645–648. Piscataway, NJ: IEEE Service Center.Google Scholar
  61. Villmann, T., Der, R., and Martinetz, T., (1994b). A Novel Approach to Measure the Topology Preservation of Feature Maps. In Marinaro, M., and Morasso, P. G., editors, Proc. ICANN’94, Int. Conf. on Artificial Neural Networks, vol. I, 298–301. London, UK: Springer.Google Scholar
  62. Wiskott, L., and Sejnowski, T. J., (1997). Objective Functions for Neural Map Formation. Lecture Notes in Computer Science, 1327.Google Scholar
  63. Zrehen, S., and Blayo, F., (1992). A geometric organization measure for Kohonen’s map. In Proc. of Neuro-Nîmes, 603–610.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Daniel Polani

There are no affiliations available

Personalised recommendations