Handbook of Memristor Networks pp 729-765 | Cite as

# Associative Networks and Perceptron Based on Memristors: Fundamentals and Algorithmic Implementation

## Abstract

The present high demand for data classification in novel computing paradigms originated a huge growth in the machine learning field. The next step consists in the hardware implementation of the idealized artificial neural networks, for which memristors grant a low power and scalable solution. The conductance of a memristor (memristance) offers the possibility of working both in binary (0 or 1) or continuous ([0, 1]) states. In the first case, it can represent the nodes in an associative neural network (e.g. a Willshaw network), while in the later it can represent the trainable weights in a classifying perceptron algorithm. This chapter reviews the theoretical basics and algorithm implementation of Willshaw and single-layer perceptron memristor-based networks. The two algorithms, developed using the open-source python language, are made available to the public for particular testing, implementation and further development.

## Notes

### Acknowledgements

This work was supported in part by projects PTDC/CTM-NAN/122868/2010, PTDC/CTM-NAN/3146/2014 and POCI-01-0145-FEDER-016623. This work was also partially supported by FEDER (Fundo Europeu de Desenvolvimento Regional) funds through the COMPETE 2020 Operacional Programme for Competitiveness and Internationalisation (POCI), Portugal 2020 and by Portuguese funds through FCT (Fundação para a Ciência e a Tecnologia) and Ministério da Ciência, Tecnologia e Inovação in the framework of the project “Institute for Research and Innovation in Health Sciences” (POCI-010145 FEDER-007274) and through the Associated Laboratory—Institute of Nanoscience and Nanotechnology. J. V. acknowledges financial support through FSE/POPH. C. Dias is thankful to FCT for grant SFRH/BD/101661/2014 and Bernardo D. Bordalo for all the help and useful comments.

## References

- 1.Ha, S.D., Ramanathan, S.: Adaptive oxide electronics: a review. J. Appl. Phys.
**110**(7), 071101 (2011)Google Scholar - 2.Merolla, P.A., Arthur, J.V., Alvarez-Icaza, R., Cassidy, A.S., Sawada, J., Akopyan, F., Jackson, B.L., Imam, N., Guo, C., Nakamura, Y., Brezzo, B., Vo, I., Esser, S.K., Appuswamy, R., Taba, B., Amir, A., Flickner, M.D., Risk, W.P., Manohar, R., Modha, D.S.: A million spiking-neuron integrated circuit with a scalable communication network and interface. Science
**345**(6197), 668–673 (2014)Google Scholar - 3.Schemmel, J., Grubl, A.: Implementing synaptic plasticity in a VLSI spiking neural network model. In: International Joint Conference on Neural Networks, pp. 1–6 (2006)Google Scholar
- 4.Seo, K., Kim, I., Jung, S., Jo, M., Park, S., Park, J., Shin, J., Biju, K.P., Kong, J., Lee, K., Lee, B., Hwang, H.: Analog memory and spike-timing-dependent plasticity characteristics of a nanoscale titanium oxide bilayer resistive switching device. Nanotechnology
**22**(25), 254023 (2011)Google Scholar - 5.Shi, L.P., Yi, K.J., Ramanathan, K., Zhao, R., Ning, N., Ding, D., Chong, T.C.: Artificial cognitive memory–changing from density driven to functionality driven. Appl. Phys. A
**102**(4), 865–875 (2011)Google Scholar - 6.Chua, L.: Memristor-the missing circuit element. IEEE Trans. Circuit Theory
**18**(5), 507–519 (1971)Google Scholar - 7.Strukov, D.B., Snider, G.S., Stewart, D.R., Williams, S.R.: The missing memristor found. Nature
**453**(7191), 80–83 (2008)Google Scholar - 8.Dias, C., Ventura, J., Aguiar, P.: Memristive-based neuromorphic applications and associative memories. In: Vaidyanathan, S., Volos, C. (eds.) Memristors. Memristive Devices and Systems. Springer, Cham (2017)Google Scholar
- 9.Kozma, R., Pino, R.E., Pazienza, G.E.: Advances in Neuromorphic Memristor Science and Applications. Springer Publishing Company, Incorporated (2012)Google Scholar
- 10.Chen, A.: Ionic memory technology. In: Solid State Electrochemistry II, pp. 1–30. Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim, Germany (2011)Google Scholar
- 11.Jo, S.H., Chang, T., Ebong, I., Bhadviya, B.B., Mazumder, P., Lu, W.: Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett.
**10**(4), 1297–301 (2010)Google Scholar - 12.Kügeler, C., Rosezin, R., Linn, E., Bruchhaus, R., Waser, R.: Materials, technologies, and circuit concepts fornanocrossbar-based bipolar RRAM. Appl. Phys. A
**102**(4), 791–809 (2011)Google Scholar - 13.Linn, E., Rosezin, R., Tappertzhofen, S., Böttger, U., Waser, R.: Beyond von Neumann-logic operations in passive crossbar arrays alongside memory operations. Nanotechnology
**23**(30), 305205 (2012)Google Scholar - 14.Prezioso, M., Merrikh-Bayat, F., Hoskins, B.D., Adam, G.C., Likharev, K.K., Strukov, D.B.: Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature
**521**(7550), 61–64 (2015)Google Scholar - 15.Mikhaylov, A.N., Morozov, O.A., Ovchinnikov, P.E., Antonov, I.N., Belov, A.I., Korolev, D.S., Koryazhkina, M.N., Sharapov, A.N., Gryaznov, E.G., Gorshkov, O.N., Kazantsev, V.B.: Towards Hardware Implementation of Double-Layer Perceptron Based on Metal-Oxide Memristive Nanostructures. 1–7 (2017) (November)Google Scholar
- 16.University of Michigan: Crossbar about to give Flash memory a serious run for its money with a faster, higher-capacity, and scalable alternative (2016)Google Scholar
- 17.Ziegler, M., Soni, R., Patelczyk, T., Ignatov, M., Bartsch, T., Meuffels, P., Kohlstedt, H.: An electronic version of Pavlov’s dog. Adv. Funct. Mater.
**22**(13), 2744–2749 (2012)Google Scholar - 18.McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys.
**5**(4), 115–133 (1943)MathSciNetzbMATHGoogle Scholar - 19.Dias, C., Guerra, L.M., Ventura, J., Aguiar, P.: Memristor-based Willshaw network: capacity and robustness to noise in the presence of defects. Appl. Phys. Lett.
**106**(22), 223505 (2015)Google Scholar - 20.Lehtonen, E., Poikonen, J.H., Laiho, M., Kanerva, P.: Large-scale memristive associative memories. IEEE Trans. Very Large Scale Integr. (VLSI) Syst.
**22**(3), 562–574 (2014)Google Scholar - 21.Duan, S., Dong, Z., Hu, X., Wang, L., Li, H.: Small-world Hopfield neural networks with weight salience priority and memristor synapses for digit recognition. Neural Comput. Appl.
**27**(4), 837–844 (2016)Google Scholar - 22.Guo, X., Merrikh-Bayat, F., Gao, L., Hoskins, B.D., Alibart, F., Linares-Barranco, B., Theogarajan, L., Teuscher, C., Strukov, D.B.: Modeling and experimental demonstration of a Hopfield network analog-to-digital converter with hybrid CMOS/memristor circuits. Front. Neurosci.
**9**, 1–8 (2015)Google Scholar - 23.Hu, S.G., Liu, Y., Liu, Z., Chen, T.P., Wang, J.J., Yu, Q., Deng, L.J., Yin, Y., Hosaka, S.: Associative memory realized by a reconfigurable memristive Hopfield neural network. Nat. Commun.
**6**, 7522 (2015)Google Scholar - 24.Yang, J., Wang, L., Wang, Y., Guo, T.: A novel memristive Hopfield neural network with application in associative memory. Neurocomputing
**227**(2016), 142–148 (2017)Google Scholar - 25.Willshaw, D.J., Buneman, O.P., Longuet-Higgins, H.C.: Non-holographic associative memory. Nature
**222**(5197), 960–962 (1969)Google Scholar - 26.Heath, J.R., Kuekes, P.J., Snider, G.S., Williams, R.S.: A defect-tolerant computer architecture: opportunities for nanotechnology. Science
**280**(5370), 1716–1721 (1998)Google Scholar - 27.Hogg, T., Snider, G.: Defect-tolerant logic with nanoscale crossbar circuits. J. Electron. Testing
**23**(2–3), 117–129 (2007)Google Scholar - 28.Chabi, D., Zhao, W., Querlioz, D., Klein, J.-O.: Robust neural logic block (NLB) based on memristor crossbar array. In: 2011 IEEE/ACM International Symposium on Nanoscale Architectures, pp. 137–143. IEEE (2011)Google Scholar
- 29.Snider, G.: Computing with hysteretic resistor crossbars. Appl. Phys. A
**80**(6), 1165–1172 (2005)Google Scholar - 30.Querlioz, D., Bichler, O., Dollfus, P., Gamrat, C.: Immunity to device variations in a spiking neural network with memristive nanodevices. IEEE Trans. Nanotechnol.
**12**(3), 288–295 (2013)Google Scholar - 31.Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. U. S. A.
**79**(8), 2554–2558 (1982)MathSciNetzbMATHGoogle Scholar - 32.Hollis, P.W., Paulos, J.J.: An analog BiCMOS Hopfield neuron. Analog Integr. Circ. Sig. Process.
**2**(4), 273–279 (1992)Google Scholar - 33.Likharev, K.K.: Neuromorphic CMOL circuits. Proceedings of the IEEE Conference on Nanotechnology
**1**, 339–342 (2003)Google Scholar - 34.Wu, A., Zhang, J., Zeng, Z.: Dynamic behaviors of a class of memristor-based Hopfield networks. Phys. Lett. Sect. A Gen. At. Solid State Phys.
**375**(15), 1661–1665 (2011)MathSciNetzbMATHGoogle Scholar - 35.Rosenblatt, F.: The Perceptron—A Perceiving and Recognizing Automaton (1957)Google Scholar
- 36.Daumé III, H.: The perceptron. In: A course in Machine Learning, pp. 37–50 (2012)Google Scholar
- 37.Park, H.: Multilayer perceptron and natural gradient learning. New Gener. Comput.
**24**, 79–95 (2006)zbMATHGoogle Scholar - 38.Isa, N.A.M., Mamat, W.M.F.W.: Clustered-Hybrid Multilayer Perceptron network for pattern recognition application. Appl. Soft Comput.
**11**(1), 1457–1466 (2011)Google Scholar - 39.Gatet, L., Tap-Béteille, H., Bony, F.: Comparison between analog and digital neural network implementations for range-finding applications. IEEE Trans. Neural Netw.
**20**(3), 460–70 (2009)Google Scholar - 40.Wang, L., Duan, M., Duan, S.: Memristive perceptron for combinational logic classification. Math. Probl. Eng.
**2013**(1), 1–7 (2013)MathSciNetzbMATHGoogle Scholar - 41.Agirre-Basurko, E., Ibarra-Berastegi, G., Madariaga, I.: Regression and multilayer perceptron-based models to forecast hourly \(\text{ O }_3\) and \(\text{ NO }_{2}\) levels in the Bilbao area. Environ. Model. Softw.
**21**(4), 430–446 (2006)Google Scholar - 42.Rose, G.S., Pino, R., Wu, Q.: A low-power memristive neuromorphic circuit utilizing a global/local training mechanism. In: The 2011 International Joint Conference on Neural Networks, vol. 1, pp. 2080–2086. IEEE (2011)Google Scholar
- 43.Strukov, D.B., Kohlstedt, H.: Resistive switching phenomena in thin films: materials, devices, and applications. MRS Bull.
**37**(2), 108–114 (2012)Google Scholar - 44.Thomas, A.: Memristor-based neural networks. J. Phys. D Appl. Phys.
**46**(9), 093001 (2013)Google Scholar - 45.Yu, S., Wu, Y., Rakesh, J.: An electronic synapse device based on metal oxide resistive switching memory for neuromorphic computation. IEEE Trans. Electron Devices
**58**(8), 2729–2737 (2011)Google Scholar - 46.Alibart, F., Zamanidoost, E., Strukov, D.B.: Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nat. Commun.
**4**, 2072 (2013)Google Scholar - 47.Zamanidoost, E., Bayat, F.M., Strukov, D., Kataeva, I.: Manhattan rule training for memristive crossbar circuit pattern classifiers. In: WISP 2015—IEEE International Symposium on Intelligent Signal Processing, Proceedings (2015)Google Scholar
- 48.Lichman, M.: UCI Machine Learning Repository (2013)Google Scholar
- 49.Hasenjäger, M., Ritter, H.: Perceptron learning revisited: the sonar targets problem. Neural Process. Lett.
**10**(1), 17–24 (1999)Google Scholar - 50.Agarap, A.F.: On breast cancer detection: an application of machine learning algorithms on the Wisconsin diagnostic dataset, vol. 1, Nov 2017Google Scholar
- 51.Zeid, M., Salama, G., Abdelhalim, M.B.: Breast cancer diagnosis on three different datasets using multi-classifiers. Int. J. Comput. Appl. Inf. Technol.
**1**, 36–43 (2012)Google Scholar