Advertisement

Geometric Constrained Joint Lane Segmentation and Lane Boundary Detection

  • Jie Zhang
  • Yi XuEmail author
  • Bingbing Ni
  • Zhenyu Duan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11205)

Abstract

Lane detection is playing an indispensable role in advanced driver assistance systems. The existing approaches for lane detection can be categorized as lane area segmentation and lane boundary detection. Most of these methods abandon a great quantity of complementary information, such as geometric priors, when exploiting the lane area and the lane boundaries alternatively. In this paper, we establish a multiple-task learning framework to segment lane areas and detect lane boundaries simultaneously. The main contributions of the proposed framework are highlighted in two facets: (1) We put forward a multiple-task learning framework with mutually interlinked sub-structures between lane segmentation and lane boundary detection to improve overall performance. (2) A novel loss function is proposed with two geometric constraints considered, as assumed that the lane boundary is predicted as the outer contour of the lane area while the lane area is predicted as the area integration result within the lane boundary lines. With an end-to-end training process, these improvements extremely enhance the robustness and accuracy of our approach on several metrics. The proposed framework is evaluated on KITTI dataset, CULane dataset and RVD dataset. Compared with the state of the arts, our approach achieves the best performance on the metrics and a robust detection in varied traffic scenes.

Keywords

Lane segmentation Semantic segmentation 

Notes

Acknowledgement

This work was supported by National Science Foundation of China 61671298 and STCSM 17511105400, 18DZ2270700. This work was supported by SJTU-UCLA Joint Center for Machine Perception and Inference. The work was also partially supported by NSFC (U1611461, 61502301, 61521062), China’s Thousand Youth Talents Plan, the 111 project B07022 and MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, China.

References

  1. 1.
    Abadi, M., et a.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015). http://tensorflow.org/
  2. 2.
    Alon, Y., Ferencz, A., Shashua, A.: Off-road path following using region classification and geometric projection constraints. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), New York, NY, USA, pp. 689–696, 17–22 June 2006. https://doi.org/10.1109/CVPR.2006.213
  3. 3.
    Álvarez, J.M., López, A.M., Baldrich, R.: Shadow resistant road segmentation from a mobile monocular system. In: Martí, J., Benedí, J.M., Mendonça, A.M., Serrat, J. (eds.) IbPRIA 2007, Part II. LNCS, vol. 4478, pp. 9–16. Springer, Heidelberg (2007).  https://doi.org/10.1007/978-3-540-72849-8_2CrossRefGoogle Scholar
  4. 4.
    Aly, M.: Real time detection of lane markers in urban streets. CoRR abs/1411.7113 (2014). http://arxiv.org/abs/1411.7113
  5. 5.
    Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. (2017)Google Scholar
  6. 6.
    Barnes, D., Maddern, W., Posner, I.: Find your own way: weakly-supervised segmentation of path proposals for urban autonomy. In: 2017 IEEE International Conference on Robotics and Automation, ICRA 2017, Singapore, Singapore, pp. 203–210, May 29–June 3 2017. https://doi.org/10.1109/ICRA.2017.7989025
  7. 7.
    Bergasa, L.M., Almeria, D., Almazán, J., Torres, J.J.Y., Arroyo, R.: DriveSafe: An app for alerting inattentive drivers and scoring driving behaviors. In: 2014 IEEE Intelligent Vehicles Symposium Proceedings, Dearborn, MI, USA, pp. 240–245, 8–11 June 2014. https://doi.org/10.1109/IVS.2014.6856461
  8. 8.
    Chen, S., Zhang, S., Shang, J., Chen, B., Zheng, N.: Brain inspired cognitive model with attention for self-driving cars. CoRR abs/1702.05596 (2017). http://arxiv.org/abs/1702.05596
  9. 9.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981).  https://doi.org/10.1145/358669.358692MathSciNetCrossRefGoogle Scholar
  10. 10.
    Gao, J., Wang, Q., Yuan, Y.: Embedding structured contour and location prior in siamesed fully convolutional networks for road detection. In: 2017 IEEE International Conference on Robotics and Automation, ICRA 2017, Singapore, Singapore, pp. 219–224 29 May–3 June 2017. https://doi.org/10.1109/ICRA.2017.7989027
  11. 11.
    Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32(11), 1231–1237 (2013).  https://doi.org/10.1177/0278364913491297CrossRefGoogle Scholar
  12. 12.
    Haloi, M., Jayagopi, D.B.: Vehicle local position estimation system. CoRR abs/1503.06648 (2015). http://arxiv.org/abs/1503.06648
  13. 13.
    Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Improving neural networks by preventing co-adaptation of feature detectors. CoRR abs/1207.0580 (2012). http://arxiv.org/abs/1207.0580
  14. 14.
    Huval, B., et al.: An empirical evaluation of deep learning on highway driving. CoRR abs/1504.01716 (2015). http://arxiv.org/abs/1504.01716
  15. 15.
    Katramados, I., Crumpler, S., Breckon, T.P.: Real-time traversable surface detection by colour space fusion and temporal analysis. In: Fritz, M., Schiele, B., Piater, J.H. (eds.) ICVS 2009. LNCS, vol. 5815, pp. 265–274. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-04667-4_27CrossRefGoogle Scholar
  16. 16.
    Kim, J., Park, C.: End-to-end ego lane estimation based on sequential transfer learning for self-driving cars. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops, Honolulu, HI, USA, pp. 1194–1202, 21–26 July 2017. https://doi.org/10.1109/CVPRW.2017.158
  17. 17.
    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. CoRR abs/1412.6980 (2014). http://arxiv.org/abs/1412.6980
  18. 18.
    Kortli, Y., Marzougui, M., Bouallegue, B., Bose, J.S.C., Rodrigues, P., Atri, M.: A novel illumination-invariant lane detection system. In: 2017 2nd International Conference on Anti-Cyber Crimes (ICACC), pp. 166–171, March 2017Google Scholar
  19. 19.
    Kotikalapudi, R., contributors: keras-vis (2017). https://github.com/raghakot/keras-vis
  20. 20.
    Levinson, J., et al.: Towards fully autonomous driving: systems and algorithms. In: 2011 IEEE Intelligent Vehicles Symposium (IV), Baden-Baden, Germany, pp. 163–168, 5–9 June 2011Google Scholar
  21. 21.
    Li, J., Mei, X., Prokhorov, D.V., Tao, D.: Deep neural network for structural prediction and lane detection in traffic scene. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 690–703 (2017).  https://doi.org/10.1109/TNNLS.2016.2522428CrossRefGoogle Scholar
  22. 22.
    Li, J., Jin, L., Fei, S., Ma, J.: Robust urban road image segmentation. In: Proceedings of the 11th World Congress on Intelligent Control and Automation, pp. 2923–2928, June 2014Google Scholar
  23. 23.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. CoRR abs/1411.4038 (2014). http://arxiv.org/abs/1411.4038
  24. 24.
    Lu, K., Li, J., An, X., He, H.: A hierarchical approach for road detection. In: 2014 IEEE International Conference on Robotics and Automation, ICRA 2014, Hong Kong, China, pp. 517–522, 31 May–7 June 2014. https://doi.org/10.1109/ICRA.2014.6906904
  25. 25.
    Oliveira, G.L., Burgard, W., Brox, T.: Efficient deep models for monocular road segmentation. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2016, Daejeon, South Korea, pp. 4885–4891, 9–14 October 2016. https://doi.org/10.1109/IROS.2016.7759717
  26. 26.
    Rahman, M.A., Wang, Y.: Optimizing intersection-over-union in deep neural networks for image segmentation. In: Bebis, G. (ed.) ISVC 2016. LNCS, vol. 10072, pp. 234–244. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-50835-1_22CrossRefGoogle Scholar
  27. 27.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-24574-4_28. http://lmb.informatik.uni-freiburg.de/Publications/2015/RFB15a. arXiv:1505.04597 [cs.CV]CrossRefGoogle Scholar
  28. 28.
    Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vision (IJCV) 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: OverFeat: integrated recognition, localization and detection using convolutional networks. CoRR abs/1312.6229 (2013). http://arxiv.org/abs/1312.6229
  30. 30.
    Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. CoRR abs/1312.6034 (2013). http://arxiv.org/abs/1312.6034
  31. 31.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014). http://arxiv.org/abs/1409.1556
  32. 32.
    Teichmann, M., Weber, M., Zöllner, J.M., Cipolla, R., Urtasun, R.: MultiNet: real-time joint semantic reasoning for autonomous driving. CoRR abs/1612.07695 (2016). http://arxiv.org/abs/1612.07695
  33. 33.
    Valente, M., Stanciulescu, B.: Real-time method for general road segmentation. In: IEEE Intelligent Vehicles Symposium, IV 2017, Los Angeles, CA, USA, pp. 443–447, 11–14 June 2017. https://doi.org/10.1109/IVS.2017.7995758
  34. 34.
    Pan, X., Shi, J., Luo, P., Wang, X., Tang, X.: Spatial as deep: Spatial CNN for traffic scene understanding. In: AAAI Conference on Artificial Intelligence (AAAI), February 2018Google Scholar
  35. 35.
    Zhang, G., Zheng, N., Cui, C., Yang, G.: An efficient road detection method in noisy urban environment. In: Intelligent Vehicles Symposium, pp. 556–561 (2009)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.SJTU-UCLA Joint Center for Machine Perception and InferenceShanghai Jiao Tong UniversityShanghaiChina

Personalised recommendations