Tracking a visual path to perform image-based control requires additional processing to extract valuable information even in the presence of inconveniences, such as failures or deformation in the lane line or even restrictive environmental conditions. Dealing with faulty in the path to be followed, non-homogeneous floors and bad lighting condition are some of the difficulties encountered that compromise the accurate of feature extraction and consequently the controller effectiveness. In this paper, a novel visual system is proposed to detect and extract useful lane line parameters and to use them in a NMPC-based Visual Path Following Control Scheme. To cope with the above-mentioned problems, the visual path is approximated by a quadratic function and to robustness improve, this novel algorithm was modified to use RANSAC as a model estimation approach instead of using the classical Least Square Method. Experimental results demonstrate the superiority of the proposed system with respect to two others, in environments with visual disturbance, faulty paths, noise and bad lighting conditions.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Arakeri, M.P., Vijaya Kumar, B.P., Barsaiya, S., Sairam, H.V.: Computer vision based robotic weed control system for precision agriculture. In: 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI) (2017)
Borkar, A., Hayes, M., Smith, M.T.: Robust lane detection and tracking with ransac and kalman filter. In: 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 3261–3264 (2009)
Castano, M., Tan, X.: Model predictive control-based path-following for tail-actuated robotic fish. J. Dyn. Syst. Meas. Control, 141 (2019)
Cheong, H.W., Lee, H.: Concept design of agv (automated guided vehicle) based on image detection and positioning. Procedia Comput. Sci. 139, 104–107 (2018). https://doi.org/10.1016/j.procs.2018.10.224
Derpanis, K.G.: Overview of the RANSAC Algorithm, vol. 4. ImageRochester, Rochester (2010)
Du, X., Tan, K.K., Htet, K.K.K.: Vision-based lane line detection for autonomous vehicle navigation and guidance. In: 2015 10th Asian Control Conference (ASCC), pp. 1–5 (2015)
Faulwasser, T., Findeisen Rolf adn Magni, L., Raimondo, D.M., Allgöwer, F.: Nonlinear Model Predictive Path-Following Control, pp 335–343. Springer, Berlin (2009)
Fischler, M.A., Bolles, R.C.: Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)
Franco, I.J.P.B., Ribeiro, T.T., Conceicao, A.G.S.: A novel approach for parameter extraction of an nmpc-based visual follower model. In: 2019 19th International Conference on Advanced Robotics (ICAR), pp. 117–122 (2019)
Gorbunov, V., Bobkov, V., Htet, N.W., Ionov, E.: Automated control system of fabrics parameters that uses computer vision. In: 2018 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (EIConRus), pp 1728–1730 (2018)
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2004)
Huang, A.S., Moore, D., Antone, M., Olson, E., Teller, S.: Finding multiple lanes in urban road networks with vision and lidar. Auton. Robot. 26(2-3), 103–122 (2009)
Jang, H.J., Baek, S.H., Park, S.Y.: Lane marking detection in various lighting conditions using robust feature extraction. In: Computer Science (2014)
Kim, Z.: Robust lane detection and tracking in challenging scenarios. IEEE Trans. Intell. Transp. Syst. 9(1), 16–26 (2008)
Kuo, Y.C., Pai, N.S., Li, Y.F.: Vision-based vehicle detection for a driver assistance system. Advances in Nonlinear Dynamics, vol. 61, pp 2096–2100 (2011)
Li, S., Xu, J., Wei, W., Qi, H.: Curve lane detection based on the binary particle swarm optimization. In: 2017 29th Chinese Control And Decision Conference (CCDC), pp. 75–80 (2017)
Lu, W., Florez, S.A.R., Seignez, E., Reynaud, R.: An improved approach for vision-based lane marking detection and tracking. In: 2013 International Conference on Electrical, Control and Automation Engineering, Dec 2013, Hong Kong, China. DEStech Publications, pp. 382–386 (2014)
Mammeri, A., Boukerche, A., Lu, G.: Lane detection and tracking system based on the mser algorithm, hough transform and kalman filter. In: MSWiM ’14 (2014)
Mammeri, A., Boukerche, A., Tang, Z.: A real-time lane marking localization, tracking and communication system. Comput. Commun. 73, 132–143 (2016)
Raković, S.V., Levine, W.: Handbook of Model Predictive Control. Springer, Berlin (2018)
Ribeiro, T.T., Conceicao, A.G.S.: Nonlinear model predictive visual path following control to autonomous mobile robots. J. Intell. Robot. Syst. 95(2), 731–743 (2019)
Ribeiro, T.T., Fernandez, R.O., Conceicao, A.G.S.: Nmpc-based visual leader-follower formation control for wheeled mobile robots. In: 2018 IEEE 16th International Conference on Industrial Informatics (INDIN), pp. 406–411 (2018)
Sun, T.Y., Tsai, S.J., Chan, V.: Hsi color model based lane-marking detection. In: 2006 IEEE Intelligent Transportation Systems Conference, pp. 1168–1172 (2006)
Vetrella, A.R., Savvaris, A., Fasano, G., Accardo, D.: Rgb-d camera-based quadrotor navigation in gps-denied and low light environments using known 3d markers. In: 2015 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 185–192 (2015)
Yuan, C., Chen, H., Liu, J., Zhu, D., Xu, Y.: Robust lane detection for complicated road environment based on normal map. IEEE Access 6, 49,679–49,689 (2018)
We would like to thank the SEPIN/MCTI and the European Union’s Horizon 2020 Research and Innovation Programme through the Grant Agreement No̱ 777096, and the Brazilian funding agency (CNPq) grant number [306852/2017-9].
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Franco, I.J.P.B., Ribeiro, T.T. & Conceição, A.G.S. A Novel Visual Lane Line Detection System for a NMPC-based Path Following Control Scheme. J Intell Robot Syst 101, 12 (2021). https://doi.org/10.1007/s10846-020-01278-x
- Autonomous mobile robots
- Visual path following
- Computer vision
- Nonlinear model predictive control