Geometric Constrained Joint Lane Segmentation and Lane Boundary Detection

  • Jie Zhang
  • Yi XuEmail author
  • Bingbing Ni
  • Zhenyu Duan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11205)


Lane detection is playing an indispensable role in advanced driver assistance systems. The existing approaches for lane detection can be categorized as lane area segmentation and lane boundary detection. Most of these methods abandon a great quantity of complementary information, such as geometric priors, when exploiting the lane area and the lane boundaries alternatively. In this paper, we establish a multiple-task learning framework to segment lane areas and detect lane boundaries simultaneously. The main contributions of the proposed framework are highlighted in two facets: (1) We put forward a multiple-task learning framework with mutually interlinked sub-structures between lane segmentation and lane boundary detection to improve overall performance. (2) A novel loss function is proposed with two geometric constraints considered, as assumed that the lane boundary is predicted as the outer contour of the lane area while the lane area is predicted as the area integration result within the lane boundary lines. With an end-to-end training process, these improvements extremely enhance the robustness and accuracy of our approach on several metrics. The proposed framework is evaluated on KITTI dataset, CULane dataset and RVD dataset. Compared with the state of the arts, our approach achieves the best performance on the metrics and a robust detection in varied traffic scenes.


Lane segmentation Semantic segmentation 



This work was supported by National Science Foundation of China 61671298 and STCSM 17511105400, 18DZ2270700. This work was supported by SJTU-UCLA Joint Center for Machine Perception and Inference. The work was also partially supported by NSFC (U1611461, 61502301, 61521062), China’s Thousand Youth Talents Plan, the 111 project B07022 and MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, China.


  1. 1.
    Abadi, M., et a.: TensorFlow: Large-scale machine learning on heterogeneous systems (2015).
  2. 2.
    Alon, Y., Ferencz, A., Shashua, A.: Off-road path following using region classification and geometric projection constraints. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), New York, NY, USA, pp. 689–696, 17–22 June 2006.
  3. 3.
    Álvarez, J.M., López, A.M., Baldrich, R.: Shadow resistant road segmentation from a mobile monocular system. In: Martí, J., Benedí, J.M., Mendonça, A.M., Serrat, J. (eds.) IbPRIA 2007, Part II. LNCS, vol. 4478, pp. 9–16. Springer, Heidelberg (2007). Scholar
  4. 4.
    Aly, M.: Real time detection of lane markers in urban streets. CoRR abs/1411.7113 (2014).
  5. 5.
    Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. (2017)Google Scholar
  6. 6.
    Barnes, D., Maddern, W., Posner, I.: Find your own way: weakly-supervised segmentation of path proposals for urban autonomy. In: 2017 IEEE International Conference on Robotics and Automation, ICRA 2017, Singapore, Singapore, pp. 203–210, May 29–June 3 2017.
  7. 7.
    Bergasa, L.M., Almeria, D., Almazán, J., Torres, J.J.Y., Arroyo, R.: DriveSafe: An app for alerting inattentive drivers and scoring driving behaviors. In: 2014 IEEE Intelligent Vehicles Symposium Proceedings, Dearborn, MI, USA, pp. 240–245, 8–11 June 2014.
  8. 8.
    Chen, S., Zhang, S., Shang, J., Chen, B., Zheng, N.: Brain inspired cognitive model with attention for self-driving cars. CoRR abs/1702.05596 (2017).
  9. 9.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981). Scholar
  10. 10.
    Gao, J., Wang, Q., Yuan, Y.: Embedding structured contour and location prior in siamesed fully convolutional networks for road detection. In: 2017 IEEE International Conference on Robotics and Automation, ICRA 2017, Singapore, Singapore, pp. 219–224 29 May–3 June 2017.
  11. 11.
    Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32(11), 1231–1237 (2013). Scholar
  12. 12.
    Haloi, M., Jayagopi, D.B.: Vehicle local position estimation system. CoRR abs/1503.06648 (2015).
  13. 13.
    Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Improving neural networks by preventing co-adaptation of feature detectors. CoRR abs/1207.0580 (2012).
  14. 14.
    Huval, B., et al.: An empirical evaluation of deep learning on highway driving. CoRR abs/1504.01716 (2015).
  15. 15.
    Katramados, I., Crumpler, S., Breckon, T.P.: Real-time traversable surface detection by colour space fusion and temporal analysis. In: Fritz, M., Schiele, B., Piater, J.H. (eds.) ICVS 2009. LNCS, vol. 5815, pp. 265–274. Springer, Heidelberg (2009). Scholar
  16. 16.
    Kim, J., Park, C.: End-to-end ego lane estimation based on sequential transfer learning for self-driving cars. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops, Honolulu, HI, USA, pp. 1194–1202, 21–26 July 2017.
  17. 17.
    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. CoRR abs/1412.6980 (2014).
  18. 18.
    Kortli, Y., Marzougui, M., Bouallegue, B., Bose, J.S.C., Rodrigues, P., Atri, M.: A novel illumination-invariant lane detection system. In: 2017 2nd International Conference on Anti-Cyber Crimes (ICACC), pp. 166–171, March 2017Google Scholar
  19. 19.
    Kotikalapudi, R., contributors: keras-vis (2017).
  20. 20.
    Levinson, J., et al.: Towards fully autonomous driving: systems and algorithms. In: 2011 IEEE Intelligent Vehicles Symposium (IV), Baden-Baden, Germany, pp. 163–168, 5–9 June 2011Google Scholar
  21. 21.
    Li, J., Mei, X., Prokhorov, D.V., Tao, D.: Deep neural network for structural prediction and lane detection in traffic scene. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 690–703 (2017). Scholar
  22. 22.
    Li, J., Jin, L., Fei, S., Ma, J.: Robust urban road image segmentation. In: Proceedings of the 11th World Congress on Intelligent Control and Automation, pp. 2923–2928, June 2014Google Scholar
  23. 23.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. CoRR abs/1411.4038 (2014).
  24. 24.
    Lu, K., Li, J., An, X., He, H.: A hierarchical approach for road detection. In: 2014 IEEE International Conference on Robotics and Automation, ICRA 2014, Hong Kong, China, pp. 517–522, 31 May–7 June 2014.
  25. 25.
    Oliveira, G.L., Burgard, W., Brox, T.: Efficient deep models for monocular road segmentation. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2016, Daejeon, South Korea, pp. 4885–4891, 9–14 October 2016.
  26. 26.
    Rahman, M.A., Wang, Y.: Optimizing intersection-over-union in deep neural networks for image segmentation. In: Bebis, G. (ed.) ISVC 2016. LNCS, vol. 10072, pp. 234–244. Springer, Cham (2016). Scholar
  27. 27.
    Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). arXiv:1505.04597 [cs.CV]CrossRefGoogle Scholar
  28. 28.
    Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vision (IJCV) 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: OverFeat: integrated recognition, localization and detection using convolutional networks. CoRR abs/1312.6229 (2013).
  30. 30.
    Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. CoRR abs/1312.6034 (2013).
  31. 31.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014).
  32. 32.
    Teichmann, M., Weber, M., Zöllner, J.M., Cipolla, R., Urtasun, R.: MultiNet: real-time joint semantic reasoning for autonomous driving. CoRR abs/1612.07695 (2016).
  33. 33.
    Valente, M., Stanciulescu, B.: Real-time method for general road segmentation. In: IEEE Intelligent Vehicles Symposium, IV 2017, Los Angeles, CA, USA, pp. 443–447, 11–14 June 2017.
  34. 34.
    Pan, X., Shi, J., Luo, P., Wang, X., Tang, X.: Spatial as deep: Spatial CNN for traffic scene understanding. In: AAAI Conference on Artificial Intelligence (AAAI), February 2018Google Scholar
  35. 35.
    Zhang, G., Zheng, N., Cui, C., Yang, G.: An efficient road detection method in noisy urban environment. In: Intelligent Vehicles Symposium, pp. 556–561 (2009)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.SJTU-UCLA Joint Center for Machine Perception and InferenceShanghai Jiao Tong UniversityShanghaiChina

Personalised recommendations