Research on CNN Parallel Computing and Learning Architecture Based on Real-Time Streaming Architecture

  • Yuting ZhuEmail author
  • Liang Qian
  • Chuyan Wang
  • Lianghui Ding
  • Feng Yang
  • Hao Wang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11319)


Convolutional neural network (CNN) is a deep feed-forward artificial neural network, which is widely used in image recognition. However, this mode highlights the problems that the training time is too long and memory is insufficient. Traditional acceleration methods are mainly limited to optimizing for an algorithm. In this paper, we propose a method, namely CNN-S, to improve training efficiency and cost based on Storm and is suitable for every algorithm. This model divides data into several sub sets and processes data on several machine in parallel flexibly. The experimental results show that in the case of achieving a recognition accuracy rate of 95%, the training time of single serial model is around 913 s, and in CNN-S model only needs 248 s. The acceleration ratio can reach 3.681. This shows that the CNN-S parallel model has better performance than single serial mode on training efficiency and cost of system resource.


CNN Parallel computing Apache storm Real time 



Yuting Zhu is also with Shanghai Microwave Research Institute and CETC Key Laboratory of Data Link Technology. This paper is supported in part by NSFC China (61771309, 61671301, 61420106008, 61521062), Shanghai Key Laboratory Funding (STCSM15DZ2270400), CETC Key Laboratory of Data Link Technology Foundation (CLDL-20162306), and Medical Engineering Cross Research Foundation of Shanghai Jiao Tong University (YG2017QN47).


  1. 1.
    Lecun, Y., Bottou, L., Bengio, Y., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  2. 2.
    Venkatraman, S., Kulkarni, S.: Map Reduce neural network framework for efficient content based image retrieval from large datasets in the cloud. In: 12th International Conference on Hybrid Intelligent Systems, HIS (2012)Google Scholar
  3. 3.
    Yang, W., Liu, X., Zhang, L., et al.: Big data real-time processing based on storm. In: Proceedings of IEEE International Conference on Trust, Security and Privacy in Computing and Communications, pp. 1784–1787 (2013)Google Scholar
  4. 4.
    Hubel, D.H., Wiesel, T.N.: Binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160(1), 106–154 (1962)CrossRefGoogle Scholar
  5. 5.
    Xiang, D., Wu, Y., Shang, P., Jiang, J., Wu, J., Yu, K.: RB-storm: resource balance scheduling in apache storm. In: 2017 6th IIAI International Congress on Advanced Applied Informatics, IIAI-AAI, Hamamatsu, pp. 419–423 (2017)Google Scholar
  6. 6.
    Xie, C., Qian, L., Ding, L., Yang, F.: Adaptive topology decomposition for storm. In: 2017 International Conference on Electrical Engineering and Informatics, ICELTICs, Banda Aceh, pp. 269–273 (2017)Google Scholar
  7. 7.
    Batyuk, A., Voityshyn, V.: Apache storm based on topology for real-time processing of streaming data from social networks. In: Proceedings of the 1st IEEE International Conference on Data Stream Mining & Processing. IEEE (2016)Google Scholar
  8. 8.
    Ranjan, R.: Streaming big data processing in datacenter clouds. IEEE Cloud Comput. 1(1), 78–83 (2014)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Yuting Zhu
    • 1
    Email author
  • Liang Qian
    • 1
  • Chuyan Wang
    • 1
  • Lianghui Ding
    • 1
  • Feng Yang
    • 1
  • Hao Wang
    • 2
  1. 1.School of Electronic Information and Electrical EngineeringShanghai Jiao Tong UniversityShanghaiChina
  2. 2.The Air Force of Military Representative Office in Shanghai-NanjingNanjingChina

Personalised recommendations