Advertisement

Slice Operator for Efficient Convolutional Neural Network Architecture

  • Van-Thanh Hoang
  • Kang-Hyun JoEmail author
Conference paper
  • 290 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12034)

Abstract

Convolutional neural networks (CNNs) have shown remarkable performance in various computer vision tasks in recent years. However, the increasing model size has raised challenges in adopting them in real-time applications as well as mobile and embedded vision applications. A number of efficient architectures have been proposed in recent years, for example, MobileNet, ShuffleNet, MobileNetV2, and ShuffleNetV2. This paper describes an improved version of ShuffleNetV2, which uses the Channel Slice operator with slice-step parameters to make information interaction between two channels, instead of using Channel Shuffle and Channel Split operators. Because the Channel Slice and Channel Split operators are similar and the proposed architecture does not have Channel Shuffle operator, it has lower memory access cost than ShuffleNetV2. The experiments on ImageNet demonstrate that the proposed network is faster than ShuffleNetV2 while still achieves similar accuracy.

Notes

Acknowledgments

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP, Ministry of Science, ICT & Future Planning) (No. 2019R1F1A1061659).

References

  1. 1.
    Chen, T., et al.: Mxnet: a flexible and efficient machine learning library for heterogeneous distributed systems. In: Proceedings of the Neural Information Processing Systems, Workshop on Machine Learning Systems (2015)Google Scholar
  2. 2.
    Chetlur, S., Woolley, C., Vandermersch, P., Cohen, J., Tran, J.: cuDNN: efficient primitives for deep learning. arXiv preprint arXiv:1410.0759 (2014)
  3. 3.
    Chollet, F.: Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1251–1258 (2017)Google Scholar
  4. 4.
    Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)Google Scholar
  5. 5.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  6. 6.
    He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46493-0_38CrossRefGoogle Scholar
  7. 7.
    Hoang, V.T., Jo, K.H.: Pydmobilenet: improved version of mobilenets with pyramid depthwise separable convolution. arXiv preprint arXiv:1811.07083 (2018)
  8. 8.
    Howard, A.G., et al.: Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)
  9. 9.
    Huang, G., Liu, S., van der Maaten, L., Weinberger, K.Q.: Condensenet: an efficient densenet using learned group convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2752–2761 (2018)Google Scholar
  10. 10.
    Huang, G., Liu, Z., Maaten, L.V.D., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)Google Scholar
  11. 11.
    LeCun, Y., et al.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)CrossRefGoogle Scholar
  12. 12.
    Lin, M., Chen, Q., Yan, S.: Network in network. In: Proceedings of the International Conference on Learning Representations (2014)Google Scholar
  13. 13.
    Ma, N., Zhang, X., Zheng, H.-T., Sun, J.: ShuffleNet V2: practical guidelines for efficient CNN architecture design. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision – ECCV 2018. LNCS, vol. 11218, pp. 122–138. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01264-9_8CrossRefGoogle Scholar
  14. 14.
    Nesterov, Y.E.: A method for solving the convex programming problem with convergence rate o (\(1/\text{k}^ 2\)). In: Dokl. Akad. Nauk SSSR, vol. 269, pp. 543–547 (1983)Google Scholar
  15. 15.
    Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22, 400–407 (1951)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Russakovsky, O., et al.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vision 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv 2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)Google Scholar
  18. 18.
    Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)Google Scholar
  19. 19.
    Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)Google Scholar
  20. 20.
    Wang, R.J., Li, X., Ling, C.X.: Pelee: A real-time object detection system on mobile devices. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 1967–1976 (2018)Google Scholar
  21. 21.
    Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1492–1500 (2017)Google Scholar
  22. 22.
    Zagoruyko, S., Komodakis, N.: Wide residual networks. In: Proceedings of the British Machine Vision Conference (2016)Google Scholar
  23. 23.
    Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: Mixup: beyond empirical risk minimization. In: Proceedings of the International Conference on Learning Representations (2018)Google Scholar
  24. 24.
    Zhang, X., Zhou, X., Lin, M., Sun, J.: Shufflenet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6848–6856 (2018)Google Scholar
  25. 25.
    Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V.: Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8697–8710 (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.School of Electrical EngineeringUniversity of UlsanUlsanKorea

Personalised recommendations