Advertisement

Deep ensemble network based on multi-path fusion

  • Enhui Lv
  • Xuesong Wang
  • Yuhu ChengEmail author
  • Qiang Yu
Article
  • 75 Downloads

Abstract

Deep convolutional network is commonly stacked by vast number of nonlinear convolutional layers. Deep fused network can improve the training process of deep convolutional network due to its capability of learning multi-scale representations and of optimizing information flow. However, the depth in a deep fused network does not contribute to the overall performance significantly. Therefore, a deep ensemble network consisting of deep fused network and branch channel is proposed. First, two base networks are combined in a concatenation and fusion manner to generate a deep fused network architecture. Then, an ensemble block with embedded learning mechanisms is formed to improve feature representation power of the model. Finally, the computational efficiency is improved by introducing group convolution without loss of performance. Experimental results on the standard recognition tasks have shown that the proposed network achieves better classification performance and has superior generalization ability compared to the original residual networks.

Keywords

Deep convolution network Deep fusion Learning mechanisms Group convolution 

Notes

References

  1. Abdi M, Nahavandi S (2017) Multi-residual networks: improving the speed and accuracy of residual networks. arXiv preprint. arXiv: 1609.05672
  2. Alom MZ, Hasan M, Yakopcic C et al (2017) Inception recurrent convolutional neural network for object recognition. arXiv preprint. arXiv: 1704.07709
  3. Bell S, Zitnick CL, Bala K et al (2016) Inside-outside net: detecting objects in context with skip pooling and recurrent neural networks. In: Proceedings of the 2016 IEEE computer society conference on computer vision and pattern recognition, pp 2874–2883Google Scholar
  4. Deng J, Dong W, Socher R et al (2009) ImageNet: a large-scale hierarchical image database. In: Proceedings of the 2009 IEEE computer society conference on computer vision and pattern recognition, pp 248–255Google Scholar
  5. Girshick R, Donahue J, Darrell T et al (2016) Region-based convolutional networks for accurate object detection and segmentation. IEEE Trans Pattern Anal Mach Intell 38(1):142–158CrossRefGoogle Scholar
  6. Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. J Mach Learn Res 9:249–256Google Scholar
  7. Guo JM, Zhang SF, Li JM (2016) Hash learning with convolutional neural networks for semantic based image retrieval. Lecture Notes in Computer Science, vol 9561, pp 227–238Google Scholar
  8. He KM, Zhang XY, Ren SQ et al (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the 2015 IEEE international conference on computer vision, pp 1026–1034Google Scholar
  9. He KM, Zhang XY, Ren SQ et al (2016) Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE computer society conference on computer vision and pattern recognition, pp 770–778Google Scholar
  10. He K, Zhang X, Ren SQ et al (2016) Identity mappings in deep residual networks. arXiv preprint. arXiv: 1603.05027
  11. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. arXiv preprint. arXiv: 1709.01507
  12. Huang G, Liu S C, Maaten et al (2018) CondenseNet: an efficient DenseNet using learned group convolutions. arXiv preprint. arXiv: 1711.09224
  13. Huang G, Liu Z, Weinberger KQ et al (2017) Densely connected convolutional networks. In: Proceedings of the 2017 IEEE computer society conference on computer vision and pattern recognition, pp 2261–2269Google Scholar
  14. Huang G, Sun Y, Liu Z et al (2016) Deep networks with stochastic depth. Leture Notes in Computer Science, vol 9908, pp 646–661Google Scholar
  15. Ioannou Y, Robertson D, Cipolla R et al (2017) Deep roots: improving CNN efficiency with hierarchical filter groups. In: Proceedings of the 2017 IEEE computer society conference on computer vision and pattern recognition, pp 5977–5986Google Scholar
  16. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd international conference on machine learning, vol 1, pp 448–456Google Scholar
  17. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90CrossRefGoogle Scholar
  18. Larsson G, Maire M, Shakhnarovich G (2017) FractalNet: ultra-deep neural networks without residuals. arXiv preprint. arXiv: 1605.07648
  19. Lin M, Chen Q, Yan S (2014) Network in network. arXiv preprint. arXiv: 1312.4400
  20. Liu XL, Deng ZD, Yang YH (2018) Recent progress in semantic image segmentation. Artif Intell Rev.  https://doi.org/10.1007/s10462-018-9641-3 Google Scholar
  21. Lv EH, Wang XS, Cheng YH et al (2018) Deep convolutional network based on pyramid architecture. IEEE Access 6:43125–43135CrossRefGoogle Scholar
  22. Patil H, Kothari A, Bhurchandi K (2015) 3-D face recognition: features, databases, algorithms and challenges. Artif Intell Rev 44(3):393–441CrossRefGoogle Scholar
  23. Romero A, Ballas N, Kahou SE et al (2015) Fitnets: hints for thin deep nets. arXiv preprint. arXiv: 1412.6550
  24. Shang WL, Sohn K, Almeida D et al (2016) Understanding and improving convolutional neural networks via concatenated rectified linear units. In: Proceedings of the 33rd international conference on machine learning, vol 5, pp 3276–3284Google Scholar
  25. Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651CrossRefGoogle Scholar
  26. Shen L, Lin ZC, Huang QM (2016) Relay backpropagation for effective learning of deep convolutional neural networks. In: Proceedings of the 14 European conference on computer vision, vol 9911, pp 467–482Google Scholar
  27. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. arXiv preprint. arXiv: 1409.1556
  28. Srivastava RK, Greff K, Schmidhuber JG (2015) Training very deep networks. Advances in Neural Information Processing Systems 2377–2385:Google Scholar
  29. Srivastava N, Hinton G, Krizhevsky A et al (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958MathSciNetzbMATHGoogle Scholar
  30. Szegedy C, Liu W, Jia YQ et al (2015) Going deeper with convolutions. In: Proceedings of the 2015 IEEE computer society conference on computer vision and pattern recognition, pp 1-9Google Scholar
  31. Szegedy C, Vanhoucke V, Ioffe S et al (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the 2016 IEEE computer society conference on computer vision and pattern recognition, pp 2818–2826Google Scholar
  32. Szegedy C, Ioffe S, Vanhoucke V et al (2017) Inception-v4, inception-resNet and the impact of residual connections on learning. In: Proceedings of the 31st AAAI conference on artificial intelligence, pp 4278–4284Google Scholar
  33. Targ S, Almeida D, Lyman K (2016) Resnet in resnet: generalizing residual architectures. arXiv preprint. arXiv: 1603.08029
  34. Veit A, Wilber M, Belongie S (2016) Residual networks are exponential ensembles of relatively shallow networks. arXiv preprint. arXiv: 1605.06431
  35. Wang JD, Zhen W, Zhang T et al (2016) Deeply-fused nets. arXiv preprint. arXiv: 1605.07716
  36. Wang F, Jiang MQ, Qian C et al (2017) Residual attention network for image classification. In: Proceedings of the 2017 IEEE computer society conference on computer vision and pattern recognition, pp 6450–6458Google Scholar
  37. Xie SN, Girshick R, Dollr P et al (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the 2017 IEEE computer society conference on computer vision and pattern recognition, pp 5987–5995Google Scholar
  38. Zagoruyko S, Komodakis N (2017) Wide residual networks. arXiv preprint. arXiv: 1605.07146
  39. Zhang T, Qi GJ, Xiao B et al (2017) Interleaved group convolutions for deep neural networks. arXiv preprint. arXiv: 1707.02725
  40. Zhao LM, Wang JD, Li X et al (2016) On the connection of deep fusion to ensembling. arXiv preprint. arXiv: 1611.07718

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.School of Information and Control EngineeringChina University of Mining and TechnologyXuzhouChina
  2. 2.Xuzhou Key Laboratory of Artificial Intelligence and Big DataXuzhouChina

Personalised recommendations