Mobile Robot Path-Tracking Using an Adaptive Critic Learning PD Controller

  • Xin Xu
  • Xuening Wang
  • Dewen Hu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3174)


This paper proposes a novel self-learning PD (Proportional- Derivative) control method for mobile robot path-tracking problems. In the self-learning PD control method, a reinforcement-learning (RL) module is used to automatically fine-tune the PD coefficients with only evaluative feedback. The optimization of the PD coefficients is modeled as a Markov decision problem (MDP) with continuous state space. Using an improved AHC (Adaptive Heuristic Critic) learning control method based on recursive least-squares algorithms, the near-optimal control policies of the MDP are approximated efficiently. Besides its simplicity, the self-learning PD controller can be adaptive to uncertainties in the environment as well as the mobile robot dynamics. Simulation and experimental results on a real mobile robot illustrate the effectiveness of the proposed method.


Mobile Robot Reward Function Critic Network Continuous State Space Adaptive Critic 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Choi, S.B.: The Design of a Look-Down Feedback Adaptive Controller for the Lateral Control of Front-Wheel-Steering Autonomous Highway Vehicles. In: Proceedings of the American Control Conference, Albuquerque, New Mexico, vol. 3, pp. 1603–1607 (1997)Google Scholar
  2. 2.
    Shin, D.H., Singh, S., Lee, J.J.: Explicit Path Tracking by Autonomous Vehicles. Robotica 10, 539–554 (1992)CrossRefGoogle Scholar
  3. 3.
    Aguilar, M., et al.: Robust Path Following Control with Exponential Stability for Mobile Robots. In: IEEE International Conference on Robotics and Automation, Leuven, Belgium, vol. 4, pp. 3279–3284 (1998)Google Scholar
  4. 4.
    Fierro, R., Lewis, F.L.: Control of a Nonholonomic Mobile Robot Using Neural Networks. IEEE Trans. on Neural Networks 9, 589–600 (1998)CrossRefGoogle Scholar
  5. 5.
    Sanchez, O., Ollero, A., Heredia, G.: Adaptive Fuzzy Control for Automatic Path Tracking of Outdoor Mobile Robots: Application to Romeo 3R. In: IEEE International Conference on Fuzzy Systems, Barcelona, Spain, vol. 1, pp. 593–599 (1997)Google Scholar
  6. 6.
    Kaelbling, L.P., et al.: Reinforcement Learning: a Survey. Journal of Artificial Intelligence Research 12, 237–285 (1996)Google Scholar
  7. 7.
    Xu, X., He, H., Hu, D.: Efficient Reinforcement Learning using Recursive Leastsquares Methods. Journal of Artificial Intelligence Research 16, 259–292 (2002)zbMATHMathSciNetGoogle Scholar
  8. 8.
    Lee, C.H., Teng, C.C.: Tuning of PID Controllers for Stable and Unstable Processes with Specifications on Gain and Phase Margins. International Journal of Fuzzy Systems 3(1), 346–355 (2001)Google Scholar
  9. 9.
    Campion, G., et al.: Structural Properties and Classification of Dynamic Models of Wheeled Mobile Robots. IEEE Trans. on Robotics and Automation 12(1), 47–62 (1996)CrossRefMathSciNetGoogle Scholar
  10. 10.
    Puterman, M.L.: Markov Decision Processes. John Wiley and Sons, New York (1994)zbMATHCrossRefGoogle Scholar
  11. 11.
    Barto, A.G., Sutton, R.S., Anderson, C.W.: Neuron-like Adaptive Elements that can Solve Difficult Learning Control Problems. IEEE Transactions on System, Man and Cybernetics 13, 834–846 (1983)Google Scholar
  12. 12.
    Berenji, H.R., Khedkar, P.: Learning and Tuning Fuzzy Logic Controllers Through Reinforcement. IEEE Trans.on Neural Networks 3(5), 724–740 (1992)CrossRefGoogle Scholar
  13. 13.
    Sutton, R.: Learning to Predict by the Method of Temporal Differences. Machine Learning 3(1), 9–44 (1988)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Xin Xu
    • 1
    • 2
  • Xuening Wang
    • 2
  • Dewen Hu
    • 2
  1. 1.School of ComputerNational University of Defense TechnologyChangshaP.R. China
  2. 2.Department of Automatic ControlNational University of Defense TechnologyChangshaP.R. China

Personalised recommendations