Advertisement

Associative Networks and Perceptron Based on Memristors: Fundamentals and Algorithmic Implementation

  • Catarina DiasEmail author
  • Daniel J. Silva
  • Paulo Aguiar
  • João Ventura
Chapter
First Online:

Abstract

The present high demand for data classification in novel computing paradigms originated a huge growth in the machine learning field. The next step consists in the hardware implementation of the idealized artificial neural networks, for which memristors grant a low power and scalable solution. The conductance of a memristor (memristance) offers the possibility of working both in binary (0 or 1) or continuous ([0, 1]) states. In the first case, it can represent the nodes in an associative neural network (e.g. a Willshaw network), while in the later it can represent the trainable weights in a classifying perceptron algorithm. This chapter reviews the theoretical basics and algorithm implementation of Willshaw and single-layer perceptron memristor-based networks. The two algorithms, developed using the open-source python language, are made available to the public for particular testing, implementation and further development.

This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

This work was supported in part by projects PTDC/CTM-NAN/122868/2010, PTDC/CTM-NAN/3146/2014 and POCI-01-0145-FEDER-016623. This work was also partially supported by FEDER (Fundo Europeu de Desenvolvimento Regional) funds through the COMPETE 2020 Operacional Programme for Competitiveness and Internationalisation (POCI), Portugal 2020 and by Portuguese funds through FCT (Fundação para a Ciência e a Tecnologia) and Ministério da Ciência, Tecnologia e Inovação in the framework of the project “Institute for Research and Innovation in Health Sciences” (POCI-010145 FEDER-007274) and through the Associated Laboratory—Institute of Nanoscience and Nanotechnology. J. V. acknowledges financial support through FSE/POPH. C. Dias is thankful to FCT for grant SFRH/BD/101661/2014 and Bernardo D. Bordalo for all the help and useful comments.

References

  1. 1.
    Ha, S.D., Ramanathan, S.: Adaptive oxide electronics: a review. J. Appl. Phys. 110(7), 071101 (2011)Google Scholar
  2. 2.
    Merolla, P.A., Arthur, J.V., Alvarez-Icaza, R., Cassidy, A.S., Sawada, J., Akopyan, F., Jackson, B.L., Imam, N., Guo, C., Nakamura, Y., Brezzo, B., Vo, I., Esser, S.K., Appuswamy, R., Taba, B., Amir, A., Flickner, M.D., Risk, W.P., Manohar, R., Modha, D.S.: A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345(6197), 668–673 (2014)Google Scholar
  3. 3.
    Schemmel, J., Grubl, A.: Implementing synaptic plasticity in a VLSI spiking neural network model. In: International Joint Conference on Neural Networks, pp. 1–6 (2006)Google Scholar
  4. 4.
    Seo, K., Kim, I., Jung, S., Jo, M., Park, S., Park, J., Shin, J., Biju, K.P., Kong, J., Lee, K., Lee, B., Hwang, H.: Analog memory and spike-timing-dependent plasticity characteristics of a nanoscale titanium oxide bilayer resistive switching device. Nanotechnology 22(25), 254023 (2011)Google Scholar
  5. 5.
    Shi, L.P., Yi, K.J., Ramanathan, K., Zhao, R., Ning, N., Ding, D., Chong, T.C.: Artificial cognitive memory–changing from density driven to functionality driven. Appl. Phys. A 102(4), 865–875 (2011)Google Scholar
  6. 6.
    Chua, L.: Memristor-the missing circuit element. IEEE Trans. Circuit Theory 18(5), 507–519 (1971)Google Scholar
  7. 7.
    Strukov, D.B., Snider, G.S., Stewart, D.R., Williams, S.R.: The missing memristor found. Nature 453(7191), 80–83 (2008)Google Scholar
  8. 8.
    Dias, C., Ventura, J., Aguiar, P.: Memristive-based neuromorphic applications and associative memories. In: Vaidyanathan, S., Volos, C. (eds.) Memristors. Memristive Devices and Systems. Springer, Cham (2017)Google Scholar
  9. 9.
    Kozma, R., Pino, R.E., Pazienza, G.E.: Advances in Neuromorphic Memristor Science and Applications. Springer Publishing Company, Incorporated (2012)Google Scholar
  10. 10.
    Chen, A.: Ionic memory technology. In: Solid State Electrochemistry II, pp. 1–30. Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim, Germany (2011)Google Scholar
  11. 11.
    Jo, S.H., Chang, T., Ebong, I., Bhadviya, B.B., Mazumder, P., Lu, W.: Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10(4), 1297–301 (2010)Google Scholar
  12. 12.
    Kügeler, C., Rosezin, R., Linn, E., Bruchhaus, R., Waser, R.: Materials, technologies, and circuit concepts fornanocrossbar-based bipolar RRAM. Appl. Phys. A 102(4), 791–809 (2011)Google Scholar
  13. 13.
    Linn, E., Rosezin, R., Tappertzhofen, S., Böttger, U., Waser, R.: Beyond von Neumann-logic operations in passive crossbar arrays alongside memory operations. Nanotechnology 23(30), 305205 (2012)Google Scholar
  14. 14.
    Prezioso, M., Merrikh-Bayat, F., Hoskins, B.D., Adam, G.C., Likharev, K.K., Strukov, D.B.: Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature 521(7550), 61–64 (2015)Google Scholar
  15. 15.
    Mikhaylov, A.N., Morozov, O.A., Ovchinnikov, P.E., Antonov, I.N., Belov, A.I., Korolev, D.S., Koryazhkina, M.N., Sharapov, A.N., Gryaznov, E.G., Gorshkov, O.N., Kazantsev, V.B.: Towards Hardware Implementation of Double-Layer Perceptron Based on Metal-Oxide Memristive Nanostructures. 1–7 (2017) (November)Google Scholar
  16. 16.
    University of Michigan: Crossbar about to give Flash memory a serious run for its money with a faster, higher-capacity, and scalable alternative (2016)Google Scholar
  17. 17.
    Ziegler, M., Soni, R., Patelczyk, T., Ignatov, M., Bartsch, T., Meuffels, P., Kohlstedt, H.: An electronic version of Pavlov’s dog. Adv. Funct. Mater. 22(13), 2744–2749 (2012)Google Scholar
  18. 18.
    McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5(4), 115–133 (1943)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Dias, C., Guerra, L.M., Ventura, J., Aguiar, P.: Memristor-based Willshaw network: capacity and robustness to noise in the presence of defects. Appl. Phys. Lett. 106(22), 223505 (2015)Google Scholar
  20. 20.
    Lehtonen, E., Poikonen, J.H., Laiho, M., Kanerva, P.: Large-scale memristive associative memories. IEEE Trans. Very Large Scale Integr. (VLSI) Syst. 22(3), 562–574 (2014)Google Scholar
  21. 21.
    Duan, S., Dong, Z., Hu, X., Wang, L., Li, H.: Small-world Hopfield neural networks with weight salience priority and memristor synapses for digit recognition. Neural Comput. Appl. 27(4), 837–844 (2016)Google Scholar
  22. 22.
    Guo, X., Merrikh-Bayat, F., Gao, L., Hoskins, B.D., Alibart, F., Linares-Barranco, B., Theogarajan, L., Teuscher, C., Strukov, D.B.: Modeling and experimental demonstration of a Hopfield network analog-to-digital converter with hybrid CMOS/memristor circuits. Front. Neurosci. 9, 1–8 (2015)Google Scholar
  23. 23.
    Hu, S.G., Liu, Y., Liu, Z., Chen, T.P., Wang, J.J., Yu, Q., Deng, L.J., Yin, Y., Hosaka, S.: Associative memory realized by a reconfigurable memristive Hopfield neural network. Nat. Commun. 6, 7522 (2015)Google Scholar
  24. 24.
    Yang, J., Wang, L., Wang, Y., Guo, T.: A novel memristive Hopfield neural network with application in associative memory. Neurocomputing 227(2016), 142–148 (2017)Google Scholar
  25. 25.
    Willshaw, D.J., Buneman, O.P., Longuet-Higgins, H.C.: Non-holographic associative memory. Nature 222(5197), 960–962 (1969)Google Scholar
  26. 26.
    Heath, J.R., Kuekes, P.J., Snider, G.S., Williams, R.S.: A defect-tolerant computer architecture: opportunities for nanotechnology. Science 280(5370), 1716–1721 (1998)Google Scholar
  27. 27.
    Hogg, T., Snider, G.: Defect-tolerant logic with nanoscale crossbar circuits. J. Electron. Testing 23(2–3), 117–129 (2007)Google Scholar
  28. 28.
    Chabi, D., Zhao, W., Querlioz, D., Klein, J.-O.: Robust neural logic block (NLB) based on memristor crossbar array. In: 2011 IEEE/ACM International Symposium on Nanoscale Architectures, pp. 137–143. IEEE (2011)Google Scholar
  29. 29.
    Snider, G.: Computing with hysteretic resistor crossbars. Appl. Phys. A 80(6), 1165–1172 (2005)Google Scholar
  30. 30.
    Querlioz, D., Bichler, O., Dollfus, P., Gamrat, C.: Immunity to device variations in a spiking neural network with memristive nanodevices. IEEE Trans. Nanotechnol. 12(3), 288–295 (2013)Google Scholar
  31. 31.
    Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. U. S. A. 79(8), 2554–2558 (1982)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Hollis, P.W., Paulos, J.J.: An analog BiCMOS Hopfield neuron. Analog Integr. Circ. Sig. Process. 2(4), 273–279 (1992)Google Scholar
  33. 33.
    Likharev, K.K.: Neuromorphic CMOL circuits. Proceedings of the IEEE Conference on Nanotechnology 1, 339–342 (2003)Google Scholar
  34. 34.
    Wu, A., Zhang, J., Zeng, Z.: Dynamic behaviors of a class of memristor-based Hopfield networks. Phys. Lett. Sect. A Gen. At. Solid State Phys. 375(15), 1661–1665 (2011)MathSciNetzbMATHGoogle Scholar
  35. 35.
    Rosenblatt, F.: The Perceptron—A Perceiving and Recognizing Automaton (1957)Google Scholar
  36. 36.
    Daumé III, H.: The perceptron. In: A course in Machine Learning, pp. 37–50 (2012)Google Scholar
  37. 37.
    Park, H.: Multilayer perceptron and natural gradient learning. New Gener. Comput. 24, 79–95 (2006)zbMATHGoogle Scholar
  38. 38.
    Isa, N.A.M., Mamat, W.M.F.W.: Clustered-Hybrid Multilayer Perceptron network for pattern recognition application. Appl. Soft Comput. 11(1), 1457–1466 (2011)Google Scholar
  39. 39.
    Gatet, L., Tap-Béteille, H., Bony, F.: Comparison between analog and digital neural network implementations for range-finding applications. IEEE Trans. Neural Netw. 20(3), 460–70 (2009)Google Scholar
  40. 40.
    Wang, L., Duan, M., Duan, S.: Memristive perceptron for combinational logic classification. Math. Probl. Eng. 2013(1), 1–7 (2013)MathSciNetzbMATHGoogle Scholar
  41. 41.
    Agirre-Basurko, E., Ibarra-Berastegi, G., Madariaga, I.: Regression and multilayer perceptron-based models to forecast hourly \(\text{ O }_3\) and \(\text{ NO }_{2}\) levels in the Bilbao area. Environ. Model. Softw. 21(4), 430–446 (2006)Google Scholar
  42. 42.
    Rose, G.S., Pino, R., Wu, Q.: A low-power memristive neuromorphic circuit utilizing a global/local training mechanism. In: The 2011 International Joint Conference on Neural Networks, vol. 1, pp. 2080–2086. IEEE (2011)Google Scholar
  43. 43.
    Strukov, D.B., Kohlstedt, H.: Resistive switching phenomena in thin films: materials, devices, and applications. MRS Bull. 37(2), 108–114 (2012)Google Scholar
  44. 44.
    Thomas, A.: Memristor-based neural networks. J. Phys. D Appl. Phys. 46(9), 093001 (2013)Google Scholar
  45. 45.
    Yu, S., Wu, Y., Rakesh, J.: An electronic synapse device based on metal oxide resistive switching memory for neuromorphic computation. IEEE Trans. Electron Devices 58(8), 2729–2737 (2011)Google Scholar
  46. 46.
    Alibart, F., Zamanidoost, E., Strukov, D.B.: Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nat. Commun. 4, 2072 (2013)Google Scholar
  47. 47.
    Zamanidoost, E., Bayat, F.M., Strukov, D., Kataeva, I.: Manhattan rule training for memristive crossbar circuit pattern classifiers. In: WISP 2015—IEEE International Symposium on Intelligent Signal Processing, Proceedings (2015)Google Scholar
  48. 48.
    Lichman, M.: UCI Machine Learning Repository (2013)Google Scholar
  49. 49.
    Hasenjäger, M., Ritter, H.: Perceptron learning revisited: the sonar targets problem. Neural Process. Lett. 10(1), 17–24 (1999)Google Scholar
  50. 50.
    Agarap, A.F.: On breast cancer detection: an application of machine learning algorithms on the Wisconsin diagnostic dataset, vol. 1, Nov 2017Google Scholar
  51. 51.
    Zeid, M., Salama, G., Abdelhalim, M.B.: Breast cancer diagnosis on three different datasets using multi-classifiers. Int. J. Comput. Appl. Inf. Technol. 1, 36–43 (2012)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Catarina Dias
    • 1
    Email author
  • Daniel J. Silva
    • 1
  • Paulo Aguiar
    • 2
    • 3
  • João Ventura
    • 1
  1. 1.Department of Physics and Astronomy, Faculty of Sciences, IFIMUP and IN—Institute of NanotechnologyUniversity of PortoPortoPortugal
  2. 2.i3S Instituto de Investigação e Inovação em SaúdeUniversidade do PortoPortoPortugal
  3. 3.INEB Instituto de Engenharia BiomédicaUniversidade do PortoPortoPortugal

Personalised recommendations

Cite chapter

Buy options