Skip to main content
SpringerLink
Log in
SpringerLink
Log in

Deep Neural Networks—A Brief History

  • Chapter
  • First Online:
Advances in Data Analysis with Computational Intelligence Methods

Part of the book series: Studies in Computational Intelligence ((SCI,volume 738))

Abstract

In this chapter we describe Deep Neural Networks (DNN), their history, and some related work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Cao, Y., Chen, Y., Khosla, D.: Spiking deep convolutional neural networks for energy-efficient object recognition. Int. J. Comput. Vis. (2014)

    Google Scholar 

  2. Cios, K.J., Shin, I.: Image recognition neural network: IRNN. Neurocomputing 7(2), 159–185 (1995)

    Article  MATH  Google Scholar 

  3. Fukushima, K.: Neocognitron: a self organizing neural network model for a mechanism for pattern recognition unaffected by shift in position. Biol. Cybern. 36, 193–202 (1980)

    Article  MATH  Google Scholar 

  4. Hebb DO. 1949. The Organization of Behavior. Wiley

    Google Scholar 

  5. Hinton, G.E., Osindero, S., Teh, Y.: A fast learning algorithm for deep belief nets. Neural Comput. 18, 1527–1554 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  6. Hinton, G.E., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  7. Hodgkin, A.L., Huxley, A.F.: A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol. 177, 500–544 (1952)

    Article  Google Scholar 

  8. Hubel, D.H., Wiesel, T.N.: Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160, 106–154 (1962)

    Article  Google Scholar 

  9. Izhikevich, E.M.: Simple model of spiking neurons. IEEE Trans. Neural Networks 14, 1569–1572 (2003)

    Article  Google Scholar 

  10. Izhikevich, E.M.: Polychronization: computation with spikes. Neural Comput. 18(2), 245–282 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  11. Kandel E.R., et al.: Principles of Neural Science, 5th edn. McGraw-Hill (2013)

    Google Scholar 

  12. Kohonen, T.: Self-organized formation of topologically correct feature maps. Biol. Cybern. 43, 59–69 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  13. Konorski J.: Conditioned Reflexes and Neuron Organization. Cambridge University Press, Cambridge (267 pp.); Reprinted with a supplementary chapter in 1968 by Hafner Publ. Co., New York (1948)

    Google Scholar 

  14. Krizhevsky, A., Sutskever, I., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: NIPS’ 2012 (2012)

    Google Scholar 

  15. Lim, H.K., Keniston, L.P., Cios, K.J.: Modeling of multisensory convergence with a network of spiking neurons: a reverse engineering approach. IEEE Trans. Biomed. Eng. 58(7), 1940–1949 (2011)

    Article  Google Scholar 

  16. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of IEEE, vol. 86, no. 11, pp. 2278–2324 (1998)

    Google Scholar 

  17. LeCun, Y., Kavukcuoglu, K., Farabet, C.: Convolutional networks and applications in vision. In: Proceedings of 2010 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 253–256. IEEE (2010)

    Google Scholar 

  18. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)

    Google Scholar 

  19. McCulloch, W.S., Pitts, W.H.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5, 115–133 (1943)

    Article  MathSciNet  MATH  Google Scholar 

  20. MacGregor, R.J.: Neural and Brain Modeling. Academic Press (1987)

    Google Scholar 

  21. Minsky, M., Papert, S.: An Introduction to Computational Geometry. MIT Press (1969)

    Google Scholar 

  22. Nguyen, A., Yosinski, J., Clune, J.: Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images (2014). arXiv:1412.1897v2 [cs.CV] 18 Dec 2014

  23. Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22(3), 400–407 (1951)

    Google Scholar 

  24. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323, 533–536 (1986)

    Google Scholar 

  25. Rosenblatt, F.: The Perceptron—a perceiving and recognizing automaton. Report 85-460-1, Cornell Aeronautical Laboratory (1957)

    Google Scholar 

  26. Sala, D.M., Cios, K.J.: Solving graph algorithms with networks of spiking neurons. IEEE Trans. Neural Netw. 10(4), 953–957 (1999)

    Article  Google Scholar 

  27. Shin, J.H., Smith, D., Swiercz, W., Staley, K., Rickard, T., Montero, J., Kurgan, L., Cios, K.J.: Recognition of partially occluded and rotated images with a network of spiking neurons. IEEE Trans. Neural Netw. 21(11), 1697–1708 (2010)

    Article  Google Scholar 

  28. Song, S., Miller, K.D., Abbot, L.F.: Competitive Hebbian learning through spike timing-dependent synaptic plasticity. Nat. Neurosci. 3(9) (2000)

    Google Scholar 

  29. Strack, B., Jacobs, K., Cios, K.J.: Biological restraint on the Izhikevich neuron model essential for seizure modeling. In: Proceedings of 6th International IEEE EMBS Conference on Neural Engineering, San Diego, 6–8 Nov, pp. 395–398 (2013)

    Google Scholar 

  30. Strack, B., Jacobs, K., Cios, K.J.: Simulating vertical and horizontal inhibition with short term dynamics in a multi-column multi-layer model of Neocortex. Int. J. Neural Syst. 24(5), 1440002 [19 pp.] (2014)

    Google Scholar 

  31. Swiercz, W., Cios, K.J., Staley, K., et al.: New synaptic plasticity rule for networks of spiking neurons. IEEE Trans. Neural Netw. 17(1), 94–105 (2006)

    Article  Google Scholar 

  32. Szeged, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, D., Fergus, R.: Intriguing properties of neural networks. In: International Conference on Learning Representations (2014)

    Google Scholar 

  33. Werbos, P.: Beyond regression: new tools for prediction and analysis in the behavioral sciences. Ph.D. thesis, Harvard University (1974)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Virginia Commonwealth University, Richmond, VA, 23284, USA

    Krzysztof J. Cios

  2. Institute of Theoretical and Applied Informatics, Polish Academy of Sciences, Bałtycka 5, 44-100, Gliwice, Poland

    Krzysztof J. Cios

Authors
  1. Krzysztof J. Cios
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Krzysztof J. Cios .

Editor information

Editors and Affiliations

  1. Department of Medicine, Division of Nephrology and Hypertension, University of Louisville, Louisville, Kentucky, USA

    Adam E Gawęda

  2. Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland

    Janusz Kacprzyk

  3. Institute of Computational Intelligence, Department of Mechanical Engineering and Computer Science, Częstochowa University of Technology, Częstochowa, Poland

    Leszek Rutkowski

  4. School of Electrical and Computer Engineering, Oklahoma State University, Stillwater, Oklahoma, USA

    Gary G. Yen

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this chapter

Cite this chapter

Cios, K.J. (2018). Deep Neural Networks—A Brief History. In: Gawęda, A., Kacprzyk, J., Rutkowski, L., Yen, G. (eds) Advances in Data Analysis with Computational Intelligence Methods. Studies in Computational Intelligence, vol 738. Springer, Cham. https://doi.org/10.1007/978-3-319-67946-4_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-67946-4_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-67945-7

  • Online ISBN: 978-3-319-67946-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation