Advertisement

Training Convolutional Neural Networks Based on Ternary Optical Processor

  • Ruifen Zhang
  • Shan OuyangEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 699)

Abstract

A novel platform and algorithms of Ternary Optical Computer (TOC) are proposed to training Convolutional Neural Network (CNN). It can significantly improve the concurrency and throughput of the training process of CNN. Reviewing the irrelevance data and the inherent parallelism module of the CNN, this paper discusses the preprocessing way of arbitrary number of two-dimensional data which include feature maps, convolutional kernels and mini-batches. Then strategies of parallel training of CNN based on the reconfigurable flexible arithmetic operator are proposed. All these arithmetic units are implemented by the optical Modified Signed Digit (MSD) adder and optical MSD multiplier, which are carry-free differing from the electronic computers. The massive data-bits of TOC are reconfigurable and redistributable, so fully parallel pipeline of the CNN can be sufficiently achieved. The computational complexity of the algorithms in time are analyzed. The result shows that TOC has great benefits comparing to the GPU and FPGA in concurrency, needed cycle and hardware resources resumed. This paper provides a new perspective to efficiently address computation-intensive and data-intensive issues.

Keywords

Massive data-bits Reconfigurable and redistributable processor Parallel processing Convolutional Neural Network Ternary Optical Computer 

References

  1. 1.
    Le Cun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Handwritten digit recognition with a back-propagation network. In: Advances in Neural Information Processing Systems (1990)Google Scholar
  2. 2.
    Sackinger, E., Boser, B., Bromley, J., LeCun, Y., Jackel, L.D.: Application of the ANNA neural network chip to high-speed character recognition. IEEE Trans. Neural Netw. 3(2), 498–505 (1992)CrossRefGoogle Scholar
  3. 3.
    Uetz, R., Behnke, S.: Large-scale object recognition with CUDA-accelerated hierarchical neural networks. In: IEEE International Conference on Intelligent Computing and Intelligent Systems, ICIS 2009, vol. 1. IEEE (2009)Google Scholar
  4. 4.
    Yi, J., Huacan, H., Lü, Y.: Ternary optical computer architecture. Phys. Scr. T118, 98 (2005)CrossRefGoogle Scholar
  5. 5.
    Jin, Y., Ouyang, S., Song, K., Shen, Y.F., Peng, J.J., Liu, X.: Management of many data bits in ternary optical computers. Sci. Sin. Inf. 43, 361–373 (2013). doi: 10.1360/112012-260 Google Scholar
  6. 6.
    Shen, Y.F., Pan, L.: Principle of a one-step MSD adder for a ternary optical computer. Sci. China Inf. Sci. 57(1), 1–10 (2014)CrossRefGoogle Scholar
  7. 7.
    Jin, Y., et al.: Principles structures and implementation of reconfigurable ternary optical processors. Sci. China Inf. Sci. 54(11), 2236–2246 (2011)CrossRefGoogle Scholar
  8. 8.
    Farabet, C., Martini, B., Akselrod, P., Talay, S., LeCun, Y., Culurciello, E.: Hardware accelerated convolutional neural networks for synthetic vision systems. In: Proceedings of 2010 IEEE International Symposium on Circuits and Systems, pp. 257–260. IEEE, May 2010Google Scholar
  9. 9.
    Shen, Y., Pengfei, H., Wang, H.: The computational complexity of arithmetic based on ternary optical computer. J. Inf. Comput. Sci. 8(5), 850–857 (2011)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2017

Authors and Affiliations

  1. 1.School of Computer Engineering and ScienceShanghai UniversityShanghaiChina
  2. 2.Computer Science BuildingShanghai UniversityShanghai CityChina

Personalised recommendations