Advertisement

From Integrated Circuits Technology to Silicon Grey Matter: Hardware Implementation of Artificial Neural Networks

  • Kurosh Madani
Conference paper

Abstract

A very large number of works concerning the area of Artificial Neural Networks deal with implementation of these models as software but also hardware solutions. However, hardware implementations of these models and issued solutions have essentially concerned the execution speed aspects. Today, a new question becomes unavoidable: taking into account the actual computers operation speeds (exceeding several Giga-operations per second), the specific hardware implementation of Artificial Neural Networks is it still an pertinent subject? This paper deals with two main goals. The first one is related to ANN's hardware implementation showing how theoretical bases of ANNs could lead to electronic implementation of these intelligent techniques. The second aim of the paper is to discuss the above formulated question through learning plasticity and robustness of ANN hardware implementations.

Keywords

artificial neural networks hardware implementation global perturbation immunity structural robustness learning plasticity 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

7 References

  1. [1]
    C.A. Mead, "Analogue VLSI and neural systems", Addison Wesley 1989.Google Scholar
  2. [2]
    R.F. Lyon and C.A. Mead, "An Analog electronic cochlea". IEEE transactions on Acoustic, Speech and signal Processing, Vol. 36, No7, pp. 1119–1134, 1988.CrossRefGoogle Scholar
  3. [3]
    L. Jackel, "Electronic neural networks". In NATO ARW, Neuro-algorithms, architecture and applications, Les Arcs, 1989.Google Scholar
  4. [4]
    M. Chiaberge, L. M. Reyneri, "Cintia: A Neuro-Fuzzy Real-Time Controller for Law-Power Embedded Systems", IEEE Micro Vol. 15, pp. 40–47, June 1995.Google Scholar
  5. [5]
    Bazoon M., Stacey D. A., and Cui C., ‘A hierarchical artificial neural network system for the classification of cervical cells', IEEE Int. Conf. On Neural Networks, Orlando, July 1994.Google Scholar
  6. [6]
    G. Mercier, K. Madani, "CMAC Real-Time Adaptive Control Implementation on a DSP Based Card",, From Natural to Artificial Neural Computation, LNCS Vol. 930, Springer Verlag, pp. 1114–1120, 1995.Google Scholar
  7. [7]
    K. Madani, P. Garda, E. Belhaire, F. Devos, Two Analog Counters for Neural Network Implementation, IEEE Journal of Solid-State Circuits, VOL. 26, No 7, JULY 1991, pp. 966–974.CrossRefGoogle Scholar
  8. [8]
    J.L. Wyatt and D.L. Standley, "Circuit design criteria for stable lateral inhibition neural networks" In IEEE International Symposium Circuits and systems, IEEE pp 997–1000, June 1988.Google Scholar
  9. [9]
    M.A. Sivilotti, M.R. Emerling, and C.A. Mead, "VLSI Architectures for implementation of Neural Network". In AIP conference Proceedings on Neural Network for computing, J.S. DENKER, American Institute of physic, Snowbird, UTAH pp408–413, 1986.Google Scholar
  10. [10]
    M. Verleysen and P. Jespers, "precision of sum-of-product in Analog Neural Network". In Proceedings of the first International workshop on Microelectronics for Neural Networks, Dortmund, RFA, June 1990.Google Scholar
  11. [11]
    P. Garda E. Belhaire, An Analog chip set with digital I/O for synchronous Boltzmann Machine, VLSI for Artificial Intelligence and Neural Network, Kluwer Academic, J.G.Delgado-frias and W.R. Moore, BOSTON, 1990.Google Scholar
  12. [12]
    P. Lalanne, J.C. Rodier, H. Richard, P. Chavel, E. Belhaire, K. Madani, P. Garda, 2-D optical generator of updating probabilities for VLSI implementation of Boltzmann Machines, International Journal of Optical Computing, Vol. 1, pp. 25–30, 1990.Google Scholar
  13. [13]
    R David, E. Williams, G. De Trémiolles, P. Tannhof, Description and Practical Uses of IBM ZISC-036, VI-DYNN'98-Virtual Intelligence-Dynamic Neural Networks Stockholm-Sweden-June 22–26, 1998.Google Scholar
  14. [14]
    J. Alspector, B. Gupta, R.B. Allen, performance of a stochastic learning microchip, Neural Information Processing Systems, Ed. David Touretzky, Morgan-Kaufmann, pp; 748–760, 1989.Google Scholar
  15. [15]
    M.A. Arbib (ed.), “Handbook of Brain Theory and Neural Networks” 2ed. M.I.T. Press. 2003.Google Scholar
  16. [16]
    S. Hebb, The Organization of Behaviour, Wiley and Sons, New-York, U.S.A., 1949.Google Scholar
  17. [17]
    T. Kohonen, Self-Organization and Associative Memory, Springer-Verlag, Germany, 1984.Google Scholar
  18. [18]
    D. Rumelhart, G. Hinton, R. Williams, Learning Internal Representations by Error Propagation", Rumelhart D., McClelland J., "Parallel Distributed Processing: Explorations in the Microstructure of Cognition", I & II, MIT Press, Cambridge MA, 1986.Google Scholar
  19. [19]
    H.P. Graf, L.D. Jackel, Analog electronic Neural Network Circuits, IEEE Circuit & Devices Magazine July 1989, pp.44–49.Google Scholar
  20. [20]
    H.P. Graf, L.D. Jackel, W.E. Hubbard, VLSI implementation of a Neural Network model, Computer, IEEE 1988, pp.34–41.Google Scholar
  21. [21]
    M. Bogdan, H. Speakman, W. Rosenstiel, Kobold: A neural Coprocessor for Back-propagation with on-line learning, Proc. NeuroMicro 94, Torino, Italy, pp. 110–117.Google Scholar
  22. [22]
    Reyneri L.M., 1995. Weighted Radial Basis Functions for Improved Pattern Recognition and Signal Processing. Neural Processing Letters, Vol. 2, No. 3, pp 2–6, May 1995.CrossRefGoogle Scholar
  23. [23]
    Trémiolles G., Madani K., Tannhof P., 1996. A New Approach to Radial Basis Function's like Artificial Neural Networks. In NeuroFuzzy'96, IEEE European Workshop, Vol. 6 No 2, pp 735–745, April 16 to 18, Prague, Czech Republic, 1996.Google Scholar
  24. [24]
    De Tremiolles G. I., "Contribution to the theoretical study of neuromimetic models and to their experimental validation: use in industrial applications" (Contribution à l'étude théorique des modèles neuromimétiques et à leur validation expérimentale: mise en œuvre d'applications industrielles), Ph.D. thesis report, University Paris XII, 05 March 1998.Google Scholar
  25. [25]
    Madani K., Tremiolles G., Tanhoff P., 2003-a. Image processing using RBF like neural networks: A ZISC-036 based fully parallel implementation solving real world and real complexity industrial problems. In Journal of Applied Intelligence No 18, 2003, Kluwer Academic Publishers, pp. 195–231.CrossRefGoogle Scholar
  26. [26]
    R. Azencott, "Synchronous Boltzmann Machines and their learning algorithms". In NATO ARW, Springer-Verlag, les arcs, February 1989.Google Scholar
  27. [27]
    G.E. Hinton and T.J. Sejnowski, "learning in Boltzmann machines". In Cognitive 85, PARIS, PP 283–290, 1985.Google Scholar
  28. [28]
    V. Lafargue, "Contribution à la réalisation électronique de Réseaux de Neurones formels: Intégration mixte de l'apprentissage des machines de Boltzmann"; Ph. D. Report, thèse de doctorat en science de l'université PARIS XI, Orsay, January 1993.Google Scholar
  29. [29]
    E. Belhaire, "Contribution à la réalisation électronique de réseaux de Neurones Formels: Intégration Analogique d'une machine de BOLTZMANN"; Ph.D. report, thèse de doctorat en science de l'université Paris XI, Orsay February 1992.Google Scholar
  30. [30]
    J.J. Hopfield, "Neurons with graded response have collective computational properties like those of two state neurones". Proceedings of the national Academy of science of U.S.A., vol 81 pp 3088–3092, 1984.CrossRefGoogle Scholar
  31. [31]
    K. Madani, I. Berechet, G. De Tremiolles, Analysis of limitations in Analog Implementation of stochastic Artificial Neural Network V, Orlando, Floride, U.S.A., 4–8 April 1994.Google Scholar
  32. [32]
    K. Madani, G. De Tremiolles, Global Perturbation Effects Analysis in a CMOS Analogue Implementation of Synchronous Boltzmann Machine, 3-rd. International Workshop on Thermal Investigations of Integrated Circuits and Microstructures, IEEE-CNRS, Cannes-Côte d'Azur, September 21–23, 1997.Google Scholar
  33. [33]
    K. Madani, G. De Tremiolles, Effects of Global Perturbations on Learning Capability in a CMOS Analogue Implementation of Synchronous Boltzmann Machine, Lecture Notes in Computer Science — Biological and Artificial Computation: From Neuroscience to Technology, Edited by: Jose Mira, Roberto M. Diaz and Joan Cabestany-Springer Verlag Berlin Heidelberg 1999, NoISBN: 3-540-66069-0, pp. 107–116.Google Scholar

Copyright information

© Springer Science+Business Media, Inc. 2005

Authors and Affiliations

  • Kurosh Madani
    • 1
  1. 1.Intelligence in Instrumentation and Systems Laboratory (I2S / JE 2353), Sénart Institute of TechnologyUniversity Paris-XIILieusaintFrance

Personalised recommendations