• Ke-Lin DuEmail author
  • M. N. S. Swamy


This chapter gives a brief introduction to the history of neural networks and machine learning. The concepts related to neurons, neural networks, and neural network processors are also described. This chapter concludes with an outline of the book.


  1. 1.
    Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985). A learning algorithm for Boltzmann machines. Cognitive Science, 9, 147–169.CrossRefGoogle Scholar
  2. 2.
    Adhikari, S. P., Yang, C., Kim, H., & Chua, L. O. (2012). Memristor bridge synapse-based neural network and its learning. IEEE Transactions on Neural Networks and Learning Systems, 23(9), 1426–1435.CrossRefGoogle Scholar
  3. 3.
    Back, A. D., & Trappenberg, T. P. (2001). Selecting inputs for modeling using normalized higher order statistics and independent component analysis. IEEE Transactions on Neural Networks, 12(3), 612–617.CrossRefGoogle Scholar
  4. 4.
    Battiti, R. (1994). Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on Neural Networks, 5(4), 537–550.CrossRefGoogle Scholar
  5. 5.
    Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007). Greedy layer-wise training of deep networks. In B. Schlkopf, J. Platt, & T. Hofmann (Eds.), Advances in neural information processing systems (Vol. 19, pp. 153–160). Cambridge, MA: MIT Press.Google Scholar
  6. 6.
    Bishop, C. M. (1995). Neural networks for pattern recognition. New York: Oxford Press.zbMATHGoogle Scholar
  7. 7.
    Broomhead, D. S., & Lowe, D. (1988). Multivariable functional interpolation and adaptive networks. Complex Systems, 2, 321–355.MathSciNetzbMATHGoogle Scholar
  8. 8.
    Candes, E. J., & Recht, B. (2009). Exact matrix completion via convex optimization. Foundations of Computational Mathematics, 9(6), 717–772.MathSciNetCrossRefGoogle Scholar
  9. 9.
    Candes, E. J., Romberg, J. K., & Tao, T. (2006). Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics, 59(8), 1207–1223.MathSciNetCrossRefGoogle Scholar
  10. 10.
    Carpenter, G. A., & Grossberg, S. (1987). A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer Vision, Graphics, and Image Processing, 37, 54–115.CrossRefGoogle Scholar
  11. 11.
    Chua, L. O., & Yang, L. (1988). Cellular neural network: I. Theory; II. Applications. IEEE Transactions on Circuits and Systems, 35, 1257–1290.MathSciNetCrossRefGoogle Scholar
  12. 12.
    Comon, P. (1994). Independent component analysis—A new concept? Signal Processing, 36(3), 287–314.CrossRefGoogle Scholar
  13. 13.
    Donoho, D. L. (2006). Compressed sensing. IEEE Transactions on Information Theory, 52(4), 1289–1306.MathSciNetCrossRefGoogle Scholar
  14. 14.
    Du, K.-L., & Swamy, M. N. S. (2006). Neural networks in a softcomputing framework. London: Springer.zbMATHGoogle Scholar
  15. 15.
    Eccles, J. (1976). From electrical to chemical transmission in the central nervous system. Notes and Records of the Royal Society of London, 30(2), 219–230.CrossRefGoogle Scholar
  16. 16.
    Estevez, P. A., Tesmer, M., Perez, C. A., & Zurada, J. M. (2009). Normalized mutual information feature selection. IEEE Transactions on Neural Networks, 20(2), 189–201.CrossRefGoogle Scholar
  17. 17.
    FitzHugh, R. (1961). Impulses and physiological states in theoretical models of nerve membrane. Biophysical Journal, 1, 445–466.CrossRefGoogle Scholar
  18. 18.
    Friedman, J. H., & Tukey, J. W. (1974). A projection pursuit algorithm for exploratory data analysis. IEEE Transactions on Computers, 23(9), 881–889.CrossRefGoogle Scholar
  19. 19.
    Fukushima, K. (1975). Cognition: A self-organizing multulayered neural network. Biological Cybernetics, 20, 121–136.CrossRefGoogle Scholar
  20. 20.
    Fukushima, K. (1980). Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36, 193–202.CrossRefGoogle Scholar
  21. 21.
    Fukushima, K. (2011). Increasing robustness against background noise: Visual pattern recognition by a neocognitron. Neural Networks, 24(7), 767–778.CrossRefGoogle Scholar
  22. 22.
    Grossberg, S. (1972). Neural expectation: Cerebellar and retinal analogues of cells fired by unlearnable and learnable pattern classes. Kybernetik, 10, 49–57.CrossRefGoogle Scholar
  23. 23.
    Grossberg, S. (1976). Adaptive pattern classification and universal recording: I. Parallel development and coding of neural feature detectors; II. Feedback, expectation, olfaction, and illusions. Biological Cybernetics, 23, 121–134 & 187–202.Google Scholar
  24. 24.
    Hebb, D. O. (1949). The organization of behavior. New York: Wiley.Google Scholar
  25. 25.
    Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science, 313(5786), 504–507.MathSciNetCrossRefGoogle Scholar
  26. 26.
    Hodgkin, A. L., & Huxley, A. F. (1952). A quantitative description of ion currents and its applications to conductance and excitation in nerve membranes. Journal of Physiology, 117, 500–544.CrossRefGoogle Scholar
  27. 27.
    Holland, J. (1975). Adaptation in natural and artificial systems. Ann Arbor, MI: University of Michigan Press.Google Scholar
  28. 28.
    Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the United States of America, 79, 2554–2558.MathSciNetCrossRefGoogle Scholar
  29. 29.
    Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Transactions on Neural Networks, 14(6), 1569–1572.MathSciNetCrossRefGoogle Scholar
  30. 30.
    Kasabov, N. (2009). Integrative connectionist learning systems inspired by nature: Current models, future trends and challenges. Natural Computing, 8, 199–218.MathSciNetCrossRefGoogle Scholar
  31. 31.
    Kirkpatrick, S., Gelatt, C. D, Jr., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220, 671–680.MathSciNetCrossRefGoogle Scholar
  32. 32.
    Kohavi, R., & John, G. H. (1997). Wrappers for feature subset selection. Artificial Intelligence, 97, 273–324.CrossRefGoogle Scholar
  33. 33.
    Kohonen, T. (1972). Correlation matrix memories. IEEE Transactions on Computers, 21, 353–359.CrossRefGoogle Scholar
  34. 34.
    Kohonen, T. (1982). Self-organized formation of topologically correct feature maps. Biological Cybernetics, 43, 59–69.MathSciNetCrossRefGoogle Scholar
  35. 35.
    Kosko, B. (1987). Adaptive bidirectional associative memories. Applied Optics, 26, 4947–4960.CrossRefGoogle Scholar
  36. 36.
    Leiva-Murillo, J. M., & Artes-Rodriguez, A. (2007). Maximization of mutual information for supervised linear feature extraction. IEEE Transactions on Neural Networks, 18(5), 1433–1441.CrossRefGoogle Scholar
  37. 37.
    Lippman, R. P. (1987). An introduction to computing with neural nets. IEEE ASSP Magazine, 4(2), 4–22.MathSciNetCrossRefGoogle Scholar
  38. 38.
    Markram, H., Lubke, J., Frotscher, M., & Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science, 275(5297), 213–215.CrossRefGoogle Scholar
  39. 39.
    McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biology, 5, 115–133.MathSciNetzbMATHGoogle Scholar
  40. 40.
    Minsky, M. L., & Papert, S. (1969). Perceptrons. Cambridge, MA: MIT Press.zbMATHGoogle Scholar
  41. 41.
    Oja, E. (1982). A simplified neuron model as a principal component analyzer. Journal of Mathematical Biology, 15, 267–273.MathSciNetCrossRefGoogle Scholar
  42. 42.
    Parisien, C., Anderson, C. H., & Eliasmith, C. (2008). Solving the problem of negative synaptic weights in cortical models. Neural Computation, 20(6), 1473–1494.MathSciNetCrossRefGoogle Scholar
  43. 43.
    Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. San Mateo, CA: Morgan Kaufmann.zbMATHGoogle Scholar
  44. 44.
    Picone, J. (1993). Signal modeling techniques in speech recognition. Proceedings of the IEEE, 81(9), 1215–1247.CrossRefGoogle Scholar
  45. 45.
    Rosenblatt, R. (1962). Principles of neurodynamics. New York: Spartan Books.zbMATHGoogle Scholar
  46. 46.
    Rubner, J., & Tavan, P. (1989). A self-organizing network for principal-component analysis. Europhysics Letters, 10, 693–698.CrossRefGoogle Scholar
  47. 47.
    Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In D. E. Rumelhart & J. L. McClelland (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition, 1: Foundation (pp. 318–362). Cambridge, MA: MIT Press.Google Scholar
  48. 48.
    Schwefel, H. P. (1981). Numerical optimization of computer models. Chichester: Wiley.zbMATHGoogle Scholar
  49. 49.
    Tripp, B., & Eliasmith, C. (2016). Function approximation in inhibitory networks. Neural Networks, 77, 95–106.CrossRefGoogle Scholar
  50. 50.
    Tuckwell, H. C. (1988). Introduction to theoretical neurobiology. Cambridge, UK: Cambridge University Press.CrossRefGoogle Scholar
  51. 51.
    Vapnik, V. N. (1998). Statistical learning theory. New York: Wiley.zbMATHGoogle Scholar
  52. 52.
    von der Malsburg, C. (1973). Self-organizing of orientation sensitive cells in the striata cortex. Kybernetik, 14, 85–100.CrossRefGoogle Scholar
  53. 53.
    Werbos, P. J. (1974). Beyond regressions: New tools for prediction and analysis in the behavioral sciences. Ph.D. thesis, Harvard University, Cambridge, MA.Google Scholar
  54. 54.
    Widrow, B., & Hoff, M. E. (1960). Adaptive switching circuits. Convention Record of IRE Eastern Electronic Show & Convention (WESCON1960), 4, 96–104.Google Scholar
  55. 55.
    Zadeh, L. A. (1965). Fuzzy sets. Information and Control, 8, 338–353.MathSciNetCrossRefGoogle Scholar
  56. 56.
    Zilles, K. (1990). Cortex. In G. Pixinos (Ed.), The human nervous system. New York: Academic Press.Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringConcordia UniversityMontrealCanada
  2. 2.Xonlink Inc.HangzhouChina

Personalised recommendations