Abstract
In this paper, we focus on the embedded deployment of real-time semantic segmentation network. Semantic segmentation based on convolutional neural networks has achieved impressive success in recent years, thus raising interests of researchers in many related application fields. The practical applications such as autonomous driving raise challenge to the lightweight of networks. Many previous achievements lighten the network by reducing layers, channels and applying group convolution, depthwise convolution to get real-time performance. At a cost, the learning ability of the network decreases. To break out of the dilemma, we propose MFANet with efficient multi-fiber unit and attention module, which obtains well balance between speed and performance. Testing on single NVIDIA 1080Ti GPU, the network achieves 65.71% MIoU with only 3.472 GFLOPs and speed of 135 FPS on the Cityscapes dataset as the input size 512 \( \times \) 1024. Further on, after some adjustments, we deploy the network to NVIDIA jetson nano embedded system, it achieves 55.33% MIoU and speed of 12 FPS, to further accelerate the model, we converted the trained model into tensorrt model of type int8, it achieves 54. 47% MIoU and speed of 47 FPS, which is capable of industrial deployment.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Treml M, Arjona-Medina J, Unterthiner T et al (2016) Speeding up semantic segmentation for autonomous driving. In: MLITS, NIPS workshop, vol 1, p 5
Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE TPAMI 39(12):2481–2495
Yu C, Kun Y, Chi Z, et al (2018) Bisenet: bilateral segmentation network for real-time semantic segmentation. In: Proceedings of the european conference on computer vision (ECCV), pp 325–341
Hang Z, Kristin D, Shi J, et al (2018) Context encoding for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7151–7160
Zhao H, Qi X, Shen X, et al (2018) ICNet for real-time semantic segmentation on high-resolution images. In: Proceedings of the european conference on computer vision (ECCV), pp 405–420
Wu Z, Shen C, van den Hengel A (2017) Real-time semantic image segmentation via spatial sparsity. arXiv preprint arXiv:1712.00213
Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, Cham, pp 234–241
Mehta S, Rastegari M, Caspi A, et al (2018) Espnet: efficient spatial pyramid of dilated convolutions for semantic segmentation. In: Proceedings of the european conference on computer vision (ECCV), pp 552–568
Chen Y, Yannis K, Li J, et al (2018) Multi-fiber networks for video recognition. In: Proceedings of the european conference on computer vision (ECCV), pp. 352–367
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
Lin G, Milan A, Shen C, et al (2017) Refinenet: Multi-path refinement networks for high-resolution semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1925–1934
Howard AG, Zhu M, Chen B, et al. (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861
Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251–1258
Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122
Siam M, Gamal M, Abdel-Razek M, et al (2018) A comparative study of real-time semantic segmentation for autonomous driving. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 587–597
Sutskever I, Vinyals O, Le QV (2014). Sequence to sequence learning with neural networks. In: Advances in neural information processing systems, pp 3104–3112
Fu J, Liu J, Tian H, et al (2018) Dual attention network for scene segmentation. arXiv preprint arXiv:1809.02983
He K, Zhang X, Ren S, et al (2018) Delving deep into rectifiers: surpassing human‐level performance on ImageNet classification. ArXiv e‐prints 2015
Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495
Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zheng, J., Li, J., Liu, Y., Zhang, W. (2020). Real-Time Semantic Segmentation Network for Edge Deployment. In: Jia, Y., Du, J., Zhang, W. (eds) Proceedings of 2019 Chinese Intelligent Systems Conference. CISC 2019. Lecture Notes in Electrical Engineering, vol 594. Springer, Singapore. https://doi.org/10.1007/978-981-32-9698-5_28
Download citation
DOI: https://doi.org/10.1007/978-981-32-9698-5_28
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-32-9697-8
Online ISBN: 978-981-32-9698-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)