A Practical Obstacle Avoidance Method Using Q-Learning with Local Information

  • Eric J. Tzeng
  • Eugene Yang
  • S. C. Chen
  • J. L. ChenEmail author
Conference paper
Part of the Mechanisms and Machine Science book series (Mechan. Machine Science, volume 73)


Various methods have been proposed for solving the obstacle avoidance problem. However, many of them are based on information that might not be available for robots in real-world settings. We focus on the generalizability and the practical aspects of the problem instead of studying yet another obstacle avoidance method. We propose a simple but robust method based on reinforcement learning for obstacle avoidance using only local information that could be gathered by the sensors on the robot. We train the model with simple and random cases having only static obstacles in a simulated environment and deploy the trained model to an actual robot car. The robot successfully avoided the static and, surprisingly, dynamic obstacles and eventually reached the target.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Algabri, M., Mathkour, H., Ramdane, H., Alsulaiman, M.: Comparative study of soft computing techniques for mobile robot navigation in an unknown environment. Computers in Human Behavior 50, 42–56 (2015)CrossRefGoogle Scholar
  2. 2.
    Bhattacharya, P., Gavrilova, M.L.: Roadmap-based path planning-using the voronoi diagram for a clearance-based shortest path. IEEE Robotics & Automation Magazine 15(2) (2008)Google Scholar
  3. 3.
    Borenstein, J., Koren, Y.: Real-time obstacle avoidance for fast mobile robots. IEEE Transactions on systems, Man, and Cybernetics 19(5), 1179–1187 (1989)CrossRefGoogle Scholar
  4. 4.
    Borenstein, J., Koren, Y.: The vector field histogram-fast obstacle avoidance for mobile robots. IEEE transactions on robotics and automation 7(3), 278–288 (1991)CrossRefGoogle Scholar
  5. 5.
    Dechter, R., Pearl, J.: Generalized best-first search strategies and the optimality of a. Journal of the ACM (JACM) 32(3), 505–536 (1985)CrossRefGoogle Scholar
  6. 6.
    Dijkstra, E.W.: A note on two problems in connexion with graphs. Numerische mathematik 1(1), 269–271 (1959)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Duguleana, M., Mogan, G.: Neural networks based reinforcement learning for mobile robots obstacle avoidance. Expert Systems with Applications 62, 104–115 (2016)CrossRefGoogle Scholar
  8. 8.
    Garrido, S., Moreno, L., Blanco, D.: Voronoi diagram and fast marching applied to path planning. In: Robotics and Automation, 2006. ICRA 2006. Proceedings 2006 IEEE International Conference on, pp. 3049–3054. IEEE (2006)Google Scholar
  9. 9.
    Keerthi, S.S., Ong, C.J., Huang, E., Gilbert, E.G.: Equidistance diagram-a new roadmap method for path planning. In: Robotics and Automation, 1999. Proceedings. 1999 IEEE International Conference on, vol. 1, pp. 682–687. IEEE (1999)Google Scholar
  10. 10.
    Sutton, R.S., Barto, A.G., et al.: Reinforcement learning: An introduction. MIT press (1998)Google Scholar
  11. 11.
    Tusi, Y., Chung, H.Y.: Using abc and rrt algorithms to improve mobile robot path planning with danger degree. In: Future Generation Communication Technologies (FGCT), 2016 Fifth International Conference on, pp. 21–26. IEEE (2016)Google Scholar
  12. 12.
    Watkins, C.J.C.H., Dayan, P.: Q-learning. Machine Learning 8(3), 279–292 (1992). DOI 10.1007/BF00992698. Scholar
  13. 13.
    Wong, C., Yang, E., Yan, X.T., Gu, D.: Adaptive and intelligent navigation of autonomous planetary roversa survey. In: Adaptive Hardware and Systems (AHS), 2017 NASA/ESA Conference on, pp. 237–244. IEEE (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Eric J. Tzeng
    • 1
  • Eugene Yang
    • 2
  • S. C. Chen
    • 1
  • J. L. Chen
    • 1
    Email author
  1. 1.Mechanical Engineering DepartmentNational Chung-Hsing UniversityTaichungTaiwan
  2. 2.IR LabGeorgetown UniversityWashington, D.C.USA

Personalised recommendations