Advertisement

Parallel Implementations of Self-Organizing Maps

  • Timo D. Hämäläinen
Part of the Studies in Fuzziness and Soft Computing book series (STUDFUZZ, volume 78)

Abstract

This chapter focuses on parallel implementations of the Self-Organizing Map (SOM) featuring different levels of parallelism. The basic arithmetic-logical operations of SOM are first reviewed for a consideration of implementation issues such as number precision, memory consumption and time complexity. Mapping involves network, training set, neuron and weight parallelism. Examples of the weight and neuron parallel mappings are given for abstract platforms to conduct general principles. Neuron parallel mapping is considered in great detail as it is the most commonly used approach. A review of implementations is given from supercomputers to VLSI (Very Large Scale Integration) chips with criteria for performance comparison.

Keywords

Input Vector Processing Unit Parallel Implementation Digital Signal Processor Neural Network Computation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Bibliography on Chapter 11

  1. 1.
    T. Kohonen, Self-Organization and Associative Memory, Springer-Verlag, Berlin, 1980.Google Scholar
  2. 2.
    T. Kohonen, Self-Organizing Maps, Springer-Verlag, Berlin, 1995.CrossRefGoogle Scholar
  3. 3.
    T. Kohonen, S. Kaski (Eds.), Kohonen Maps, Elsevier, Amsterdam, 1999.Google Scholar
  4. 4.
    T. Kohonen, The ‘Neural’ Phonetic Typewriter, Computer, Vol. 21, 1988, 11–22.CrossRefGoogle Scholar
  5. 5.
    X. Ling, D. Soergel, G. Marchionini, “A self-organizing semantic map for information retrieval”, in: Proceedings of 14th annual international conference on RD in information retrieval, 1991, 262–269.Google Scholar
  6. 6.
    T. Kohonen, S. Kaski, K. Lagus, T. Honkela, “Very large two-level SOM for the browsing of newsgroups”, in: Proceedings of International Conference on Artificial Neural Networks, 1996, 269–274.Google Scholar
  7. 7.
    T. Kohonen, S. Kaski, K. Lagus, J. Salojärvi, J. Honkela, V. Paatero, and H. Saarela, “Self organization of a massive document collection”, IEEE Transactions on Neural Networks, Vol. 11, No. 3, 2000, 574–585.CrossRefGoogle Scholar
  8. 8.
    P. Ienne, P. Thiran, T. Vassilas, “Modified self-organizing feature map algorithms for efficient digital hardware implementation”, IEEE Transactions on Neural Networks, Vol. 8, No. 2, 2000, 315–330.CrossRefGoogle Scholar
  9. 9.
    A. Peleg, U. Weiser, “MMX technology extension to the Intel architecture,” IEEE Micro, Vol. 16, No. 4, 1996, 42–50.CrossRefGoogle Scholar
  10. 10.
    T. Thakkar and T. Huff, “The interne streaming SIMD extensions,” IEEE Computer, Dec. 1999, 26–34.Google Scholar
  11. 11.
    P. Thiran, V. Peiris, P. Heim, B. Hochet, “Quantization effects in digitally behaving circuit implementations of Kohonen networks”, IEEE Transactions on Neural Networks, Vol. 5, No. 3, 1994, 450–458.CrossRefGoogle Scholar
  12. 12.
    A. Rauber, P. Tomisch,D. Merkl, “parSOM: a parallel Implementation of the self organizing map exploiting cache effects–making the SOM fit for interactive high-performance data analysis”, In: Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks, Vol. 6, 2000, 177–182.Google Scholar
  13. 13.
    P.W. Diodato, “Embedded DRAM: more than just a memory”, IEEE Communications Magazine, Vol. 38, No. 7, 2000, 118–126.CrossRefGoogle Scholar
  14. 14.
    T. Hämäläinen, H. Klapuri, J. Saarinen and K. Kaski, “Mapping of SOM and LVQ Algorithms on a Tree Shape Parallel Computer System”, Parallel Computing, Vol. 23, 1997, 271–289.CrossRefMATHGoogle Scholar
  15. 15.
    D. Bertsekas, J. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods, Prentice-Hall, USA, 1989.MATHGoogle Scholar
  16. 16.
    T. Nordström and B. Svensson, “Using and designing massively parallel computers for artificial neural networks”, Journal of Parallel and Distributed Computing, Vol. 14, No. 3, 1992, 260–285.CrossRefGoogle Scholar
  17. 17.
    C.H. Wu, R.E. Hodges, “Parallelizing the self-organizing feature map on multiprocessor systems”, Parallel Computing, No. 17, 1991, 821–832.CrossRefMATHGoogle Scholar
  18. 18.
    V. Demian, J-C. Mignot, “Implementation of the self-organizing feature map on parallel computers”, Computers and Artificial Intelligence, No. 1, 1996, 63–80.Google Scholar
  19. 19.
    E. Schikuta, C. Weidmann, Data parallel simulation of self-organizing maps on hyper-cube architectures, in: Proceedings of Workshop on Self-Organizing Maps, WSOM 87, Helsinki, Finland, 1997, 142–147.Google Scholar
  20. 20.
    K. Obermayer, H. Ritter and K. Schulten, “Large-scale simulations of self-organizing neural networks on parallel computers: application to biological modelling”, Parallel Computing, Vol. 13, No. 3, 1990, 381–404.CrossRefGoogle Scholar
  21. 21.
    R. Mann and S. Haykin, “A parallel implementation of Kohonen feature maps on the Warp systolic computer”, in: Proceedings of International Joint Conference on Neural Networks, Vol. 2, 1990, 84–87.Google Scholar
  22. 22.
    M. Yasunaga, K. Tominaga, Jung Hwan Kim, “Parallel self-organization map using multiple stimuli”, in: Proceedings International Joint Conference on Neural Networks, Vol. 2, 1999, 1127–1130.Google Scholar
  23. 23.
    N. Bandeira, V.J. Lobo, F. Moura-Pires, “Training a self-organizing map distributed on a PVM network”, in: Proceedings of IEEE Joint Conference on Neural Networks, Vol. 1, 1998, 457–461.Google Scholar
  24. 24.
    J.S. Lange, P. Schonmeier, H. Freiesleben,“Parallelization of analyses using self-organizing maps with PVM”, Nuclear Instruments and Methods in Physics Research A, No. 389, 1997, 274–76.CrossRefGoogle Scholar
  25. 25.
    H. Guan, Chi-kwong Li, To-yat Cheung, Songnian Yu, “Parallel design and implementation of SOM neural computing model in PVM environment of a distributed system”, in: Proceedings of Advances in Parallel and Distributed Computing, 1997, 26–31.Google Scholar
  26. 26.
    T. Bollinger, “Linux in practice: an overview of applications”, IEEE Software, Vol. 16 No. 1, 1999, 72–79.CrossRefGoogle Scholar
  27. 27.
    N. Boden, D. Cohen, R. Felderman, A. Kulawik, C. Sietz, J. Seizovic, W. Su, “Myrinet–A Gigabit-per-Second Local Area Network”, IEEE Micro, Feb 1995, 29–36.Google Scholar
  28. 28.
    H. Simeon and A. Ultsch, Kohonen Networks on Transputers: Implementation and Animation, in: Proceedings of International Neural Network Conference, Vol. 2, 1990, 643–646.Google Scholar
  29. 29.
    J.S. Lange, C. Fukunaga, M. Tanaka, A. Bozek, “Transputer Self-Organizing Map Algorithm for Beam Background Rejection at the Belle Silicon Vertex Detector”, Nuclear Instruments Methods In Physics Research Section A No. 420, 1999, 288–309.CrossRefGoogle Scholar
  30. 30.
    J.M. Auger, “Parallel implementation on transputer of Kohonen’s algorithm”, in: Proceedings of Computing with Parallel Architectures: T. Node,1991, 215–226.Google Scholar
  31. 31.
    M.E. Azema-Barac, “A generic strategy for mapping neural network models on transputer-based machines”, in: G. L. Reijns, J. Luo (Eds.), Transputing in numerical and neural network applications, IOS Press, 1992, 244–249.Google Scholar
  32. 32.
    H. Kihl, J.P. Urban, J. Gresser, S. Hagmann, “Neural network based hand-eye positioning with a Transputer-based system”, in: Proceedings of High-Performance Computing and Networking–International Conference and Exhibition, 1995, 281–286.Google Scholar
  33. 33.
    R. Togneri and Y. Attikiouzel, “Parallel Implementation of the Kohonen Algorithm on Transputer”, in: Proceedings of International Joint Conference on Neural Networks, Vol. II, 1991, 1717–1722.Google Scholar
  34. 34.
    S.A. Wilde, K.M. Curtis, “A transputer based self-organizing neural network for speech synthesis parameter arbitration”, in: Proceedings of Transputer Applications and Systems–1993 World Transputer Congress, 1993, 1242–1253.Google Scholar
  35. 35.
    H.C. Card, G.K. Rosendahl, D.K. McNeill, R.D. McLeod, “Competitive Learning Algorithms and Neurocomputer Architecture”, IEEE Transactions on Computers, Vol. 47, No. 8, 1998, 847–858.CrossRefGoogle Scholar
  36. 36.
    T. Cornu, P. Ienne, D. Niebur, P. Thiran and M. Viredaz, “Design, Implementation and Test of a Multi-Model Systolic Neural Network Accelerator”, Scientific Programming, Vol. 5, No. 1, 1996, 47–61.Google Scholar
  37. 37.
    U. Müller, A. Gunzinger, W. Guggenbühl, “Fast Neural Net Simulation with a DSP Processor Array”, IEEE Transactions on Neural Networks, Vol. 6, No. 1, 1995, 203–213.CrossRefGoogle Scholar
  38. 38.
    P. Kolinummi, P. Pulkkinen, T. Hämäläinen, J. Saarinen, “Parallel implementation of Self-Organizing map on the partial tree shape neurocomputer”, Neural Processing Letters, Vol. 12, No. 2, 2000, 171–182.CrossRefMATHGoogle Scholar
  39. 39.
    G. Myklebust and J.G. Solheim, “Parallel self-organizing Maps for actual applications”, in: Proceedings IEEE International Conference on Neural Networks, Vol. II, 1995, 1054–1059.CrossRefGoogle Scholar
  40. 40.
    T. Hämäläinen, J. Saarinen and K. Kaski, “TUTNC: A general purpose parallel computer for neural network computations”, Microprocessors and Microsystems, Vol. 9, No. 8, 1995, 447–465.CrossRefGoogle Scholar
  41. 41.
    P. Kolinummi, P. Hämäläinen, T. Hämäläinen, J. Saarinen, “PARNEU: General-purpose partial tree computer”, Microprocessors and Microsystems, Vol. 24, No. 1, 2000, 23–42.CrossRefGoogle Scholar
  42. 42.
    P. Kolinummi, T. Hämäläinen, J. Saarinen, “Chained Backplane communication architecture for scalable multiprocessor systems”, Journal of Systems Architecture Vol. 46, No. 11, 955–972.Google Scholar
  43. 43.
    D. Hammerström and N. Nguyen, “An Implementation of Kohonen’s self-organizing map on the Adaptive Solutions neurocomputer”, in: T. Kohonen, K. Mäkisara, O. Simula and J. Kangas (Eds.), Artificial Neural Networks, North-Holland, Amsterdam, Vol. 1, 1991, 715–720.Google Scholar
  44. 44.
    T. Fischer, W. Eppler, H. Gemmeke, G. Kock, T. Becher, “The SAND neurochip and its embedding in the MiND system”, in: Proceedings of Artificial Neural Networks, 1997, 1235–1240.Google Scholar
  45. 45.
    W. Eppler, T. Fischer, H. Gemmeke, T. Koder, R. Stotzka, “Neural chip SAND/I for real time pattern recognition ”, IEEE Transactions on Nuclear Science, Vol. 45, No. 4, 1819–1823.Google Scholar
  46. 46.
    U. Ramacher, “SYNAPSE–A Neurocomputer That Synthesizes Neural Algorithms on a Parallel Systolic Engine”, Journal of Parallel and Distributed Computing, Vol. 14, No. 3, 1992, 306–318.CrossRefGoogle Scholar
  47. 47.
    B. Hochet, V. Peiris, S. Abdo, M. Declercq, “Implementation of a Learning Kohonen Neuron Based on a New Multilevel Storage Technique”, IEEE Journal of Solid-State Circuits, Vol. 26, No. 3, 1991, 262–267.CrossRefGoogle Scholar
  48. 48.
    J. Choi and B. J. Sheu, “A high precision VLSI winner-take-all circuit for self-organizing neural networks” IEEE Journal of Solid-State Circuits, Vol. 28, May 1993, 579–584.CrossRefGoogle Scholar
  49. 49.
    S.Rüping, M. Porrmann, U. Rueckert, “SOM Accelerator System”, Neurocomputing, Vol. 21, No. 1–3, 1998, 31–50.CrossRefGoogle Scholar
  50. 50.
    X. Fang, P. Thole, J. Göppert and W Rosenstiel, “A Hardware Supported System for a Special Online Application of Self-Organizing Map”, In: Proceedings of the International Conference on Neural Networks, 1996, 956–961.Google Scholar
  51. 51.
    J. Lubkin, G. Cauwenberghs, “VLSI implementation of fuzzy adaptive resonance and learning vector quantization”, in: Proceedings of the Seventh International Conference on Microelectronics for Neural, Fuzzy and Bio-Inspired Systems, 1999, 147–54.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Timo D. Hämäläinen

There are no affiliations available

Personalised recommendations