Advertisement

Mapping Inference Algorithms to DIMA

  • Mingu Kang
  • Sujan Gonugondla
  • Naresh R. Shanbhag
Chapter
  • 201 Downloads

Abstract

This chapter shows that diverse algorithms with significantly complex data-flow can also be mapped to DIMA. The mapping of a convolutional neural network (CNN) and a sparse distributed memory (SDM) to DIMA is demonstrated. Algorithmic opportunities such as the use of error-aware training in a DIMA-based CNN and the use of ensemble decision-making in SDM can be exploited to compensate for non-ideal analog computations in DIMA leading to even greater energy savings.

Keywords

Convolutional neural networks (CNN) Associative memory Brain-inspired computing Sparse distributed memory (SDM) 

References

  1. 7.
    P.N. Whatmough, S.K. Lee, H. Lee, S. Rama, D. Brooks, G.-Y. Wei, A 28nm SoC with a 1.2 GHz 568nJ/prediction sparse deep-neural-network engine with >0.1 timing error rate tolerance for IoT applications, in IEEE International Solid-State Circuits Conference (ISSCC) (2017), pp. 242–243Google Scholar
  2. 12.
    K. Mai, T. Paaske, N. Jayasena, R. Ho, W.J. Dally, M. Horowitz, Smart memories: a modular reconfigurable architecture, in ACM SIGARCH Computer Architecture News, vol. 28 (ACM, New York, 2000), pp. 161–171Google Scholar
  3. 14.
    D. Patterson, T. Anderson, N. Cardwell, R. Fromm, K. Keeton, C. Kozyrakis, R. Thomas, K. Yelick, Intelligent RAM (IRAM): chips that remember and compute, in IEEE International Solid-State Circuits Conference (ISSCC) (IEEE, Piscataway, 1997), pp. 224–225Google Scholar
  4. 29.
    M. Kang, M.-S. Keel, N.R. Shanbhag, S. Eilert, K. Curewitz, An energy-efficient VLSI architecture for pattern recognition via deep embedding of computation in SRAM, in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2014), pp. 8326–8330Google Scholar
  5. 32.
    M. Kang, S.K. Gonugondla, A. Patil, N.R. Shanbhag, A multi-functional in-memory inference processor using a standard 6T SRAM array. IEEE J. Solid State Circuits 53(2), 642–655 (2018)CrossRefGoogle Scholar
  6. 41.
    M. Kang, S.K. Gonugondla, M.-S. Keel, N.R. Shanbhag, An energy-efficient memory-based high-throughput VLSI architecture for convolutional networks, in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2015)Google Scholar
  7. 43.
    M. Kang, E.P. Kim, M.-S. Keel, N. R. Shanbhag, Energy-efficient and high throughput sparse distributed memory architecture, in IEEE International Symposium on Circuits and Systems (ISCAS) (2015), pp. 2505–2508Google Scholar
  8. 44.
    M. Kang, S. Gonugondla, A. Patil, N. Shanbhag, A 481pJ/decision 3.4M decision/s multifunctional deep in-memory inference processor using standard 6T SRAM array. arXiv:1610.07501 (preprint, 2016)Google Scholar
  9. 46.
    M. Yamaoka et al., A 300-MHz 25-μA/Mb-leakage on-chip SRAM module featuring process-variation immunity and low-leakage-active mode for mobile-phone application processor. IEEE J. Solid State Circuits 40(1), 186–194 (2005)CrossRefGoogle Scholar
  10. 64.
    Y. LeCun, C. Cortes, MNIST handwritten digit database. AT&T Labs (2010). http://yann.lecun.com/exdb/mnist
  11. 66.
    T. Kobayashi, K. Nogami, T. Shirotori, Y. Fujimoto, A current-controlled latch sense amplifier and a static power-saving input buffer for low-power architecture. IEEE J. Solid State Circuits76(5), 863–867 (1993)Google Scholar
  12. 74.
    M. Kang, N.R. Shanbhag, In-memory computing architectures for sparse distributed memory. IEEE Trans. Biomed. Circuits Syst.10(4), 855–863 (2016)CrossRefGoogle Scholar
  13. 75.
    Y. LeCun, L. Jackel, L. Bottou, A. Brunot, C. Cortes, J. Denker, H. Drucker, I. Guyon, U. Muller, E. Sackinger et al., Comparison of learning algorithms for handwritten digit recognition, in International Conference on Artificial Neural Networks, vol. 60 (1995), pp. 53–60Google Scholar
  14. 76.
    D. Strigl, K. Kofler, S. Podlipnig, Performance and scalability of GPU-based convolutional neural networks, in IEEE Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP) (2010), pp. 317–324Google Scholar
  15. 77.
    C. Farabet, C. Poulet, J.Y. Han, Y. LeCun, CNP: an FPGA-based processor for convolutional networks, in IEEE International Conference on Field Programmable Logic and Applications (FPL) (2009), pp. 32–37Google Scholar
  16. 78.
    K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (preprint, 2014)Google Scholar
  17. 79.
    K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 770–778Google Scholar
  18. 80.
    W. Rieutort-Louis, T. Moy, Z. Wang, S. Wagner, J.C. Sturm, N. Verma, A large-area image sensing and detection system based on embedded thin-film classifiers. IEEE J. Solid State Circuits51(1), 281–290 (2016)CrossRefGoogle Scholar
  19. 81.
    B. Moons, M. Verhelst, A 0.3-2.6 TOPS/W precision-scalable processor for real-time large-scale ConvNets, in IEEE Symposium on VLSI Circuits (VLSI Circuits) (2016), pp. 1–2Google Scholar
  20. 82.
    J.K. Kim, P. Knag, T. Chen, Z. Zhang, A 640M pixel/s 3.65 mW sparse event-driven neuromorphic object recognition processor with on-chip learning, in IEEE Symposium on VLSI Circuits (VLSI Circuits) (2015), pp. C50–C51Google Scholar
  21. 83.
    J.M. Cruz-Albrecht, M.W. Yung, N. Srinivasa, Energy-efficient neuron, synapse and STDP integrated circuits. IEEE Trans. Biomed. Circuits Syst.6(3), 246–256 (2012)CrossRefGoogle Scholar
  22. 86.
    V. Garg, R. Shekhar, J.G. Harris, Spiking neuron computation with the time machine. IEEE Trans. Biomed. Circuits Syst.6(2), 142–155 (2012)CrossRefGoogle Scholar
  23. 87.
    P. Kanerva, Sparse distributed memory (MIT Press, Cambridge, 1988)zbMATHGoogle Scholar
  24. 88.
    P.J. Denning, Sparse distributed memory. Research Institute for Advanced Computer Science (NASA Ames Research Center, Mountain View, 1989)Google Scholar
  25. 89.
    E. Lehtonen, J.H. Poikonen, M. Laiho, P. Kanerva, Large-scale memristive associative memories. IEEE Trans. Very Large Scale Integr. Syst.22(3), 562–574 (2014)CrossRefGoogle Scholar
  26. 90.
    M. Lindell et al., Configurable sparse distributed memory hardware implementation, in IEEE International Symposium on Circuits and Systems (ISCAS) (1991), 3078–3081Google Scholar
  27. 91.
    J. Saarinen et al., VLSI architectures of sparse distributed memory, in IEEE International Symposium on Circuits and Systems (ISCAS) (1991), pp. 3074–3077Google Scholar
  28. 92.
    J.D. Keeler et al., Notes on implementation of sparsely distributed memory, in NASA Research Institute for Advanced Computer Science (1986)Google Scholar
  29. 93.
    S.-I. Chien, I.-C. Kim, D.-Y. Kim, Iterative autoassociative memory models for image recalls and pattern classifications, in IEEE International Joint Conference on Neural Networks (IJCNN) (1991), pp. 30–35Google Scholar
  30. 94.
    I. Kim et al., High performance PRAM cell scalable to sub-20nm technology with below 4F 2 cell size, extendable to DRAM applications, in IEEE Symposium on VLSI Technology (VLSI Technology) (2010), pp. 203–204Google Scholar
  31. 95.
    S. Aritome, Advanced Flash memory technology and trends for file storage application, in IEEE International Electron Devices Meeting (IEDM) (2000), pp. 763–766Google Scholar
  32. 96.
    A. Verma, B. Razavi, Frequency-based measurement of mismatches between small capacitors, in IEEE Custom Integrated Circuits Conference (CICC) (2006), pp. 481–484Google Scholar
  33. 97.
    K. Kim, H. Mahmoodi, K. Roy, A low-power SRAM using bit-line charge-recycling technique, in International Symposium on Low Power Electronics and Design (ISLPED) (2007), pp. 177–182Google Scholar
  34. 84.
    S. Brink, S. Nease, P. Hasler, S. Ramakrishnan, R. Wunderlich, A. Basu, B. Degnan, A learning-enabled neuron array IC based upon transistor channel models of biological phenomena. IEEE Trans. Biomed. Circuits Syst.7(1), 71–81 (2013)CrossRefGoogle Scholar
  35. 85.
    S. Ramakrishnan, R. Wunderlich, J. Hasler, S. George, Neuron array with plastic synapses and programmable dendrites. IEEE Trans. Biomed. Circuits Syst.7(5), 631–642 (2013)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Mingu Kang
    • 1
  • Sujan Gonugondla
    • 2
  • Naresh R. Shanbhag
    • 2
  1. 1.IBM T. J. Watson Research CenterOld TappanUSA
  2. 2.University of Illinois at Urbana-ChampaignUrbanaUSA

Personalised recommendations