Training Convolutional Neural Networks Based on Ternary Optical Processor
A novel platform and algorithms of Ternary Optical Computer (TOC) are proposed to training Convolutional Neural Network (CNN). It can significantly improve the concurrency and throughput of the training process of CNN. Reviewing the irrelevance data and the inherent parallelism module of the CNN, this paper discusses the preprocessing way of arbitrary number of two-dimensional data which include feature maps, convolutional kernels and mini-batches. Then strategies of parallel training of CNN based on the reconfigurable flexible arithmetic operator are proposed. All these arithmetic units are implemented by the optical Modified Signed Digit (MSD) adder and optical MSD multiplier, which are carry-free differing from the electronic computers. The massive data-bits of TOC are reconfigurable and redistributable, so fully parallel pipeline of the CNN can be sufficiently achieved. The computational complexity of the algorithms in time are analyzed. The result shows that TOC has great benefits comparing to the GPU and FPGA in concurrency, needed cycle and hardware resources resumed. This paper provides a new perspective to efficiently address computation-intensive and data-intensive issues.
KeywordsMassive data-bits Reconfigurable and redistributable processor Parallel processing Convolutional Neural Network Ternary Optical Computer
- 1.Le Cun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Handwritten digit recognition with a back-propagation network. In: Advances in Neural Information Processing Systems (1990)Google Scholar
- 3.Uetz, R., Behnke, S.: Large-scale object recognition with CUDA-accelerated hierarchical neural networks. In: IEEE International Conference on Intelligent Computing and Intelligent Systems, ICIS 2009, vol. 1. IEEE (2009)Google Scholar
- 8.Farabet, C., Martini, B., Akselrod, P., Talay, S., LeCun, Y., Culurciello, E.: Hardware accelerated convolutional neural networks for synthetic vision systems. In: Proceedings of 2010 IEEE International Symposium on Circuits and Systems, pp. 257–260. IEEE, May 2010Google Scholar
- 9.Shen, Y., Pengfei, H., Wang, H.: The computational complexity of arithmetic based on ternary optical computer. J. Inf. Comput. Sci. 8(5), 850–857 (2011)Google Scholar