Abstract
In this paper, a reinforcement learning method called DAQL is proposed to solve the problem of seeking and homing onto a fast maneuvering target, within the context of mobile robots. This Q-learning based method considers both target and obstacle actions when determining its own action decisions, which enables the agent to learn more effectively in a dynamically changing environment. It particularly suits fast-maneuvering target cases, in which maneuvers of the target are unknown a priori. Simulation result depicts that the proposed method is able to choose a less convoluted path to reach the target when compared to the ideal proportional navigation (IPN) method in handling fast maneuvering and randomly moving target. Furthermore, it can learn to adapt to the physical limitation of the system and do not require specific initial conditions to be satisfied for successful navigation towards the moving target.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Ge, S.S., Cui, Y.J.: New potential functions for mobile robot path planning. IEEE Transactions on Robotics and Automation 16, 615–620 (2000)
Conte, G., Zulli, R.: Hierarchical Path Planning in a Multirobot Environment with a Simple Navigation Function. IEEE Transactions on Systems Man and Cybernetics 25, 651–654 (1995)
Li, T.H.S., Chang, S.J., Tong, W.: Fuzzy target tracking control of autonomous mobile robots by using infrared sensors. IEEE Transactions on Fuzzy Systems 12, 491–501 (2004)
Luo, R.C., Chen, T.M., Su, K.L.: Target tracking using hierarchical grey-fuzzy motion decision-making method. IEEE Transactions on Systems Man and Cybernetics Part a-Systems and Humans 31, 179–186 (2001)
Dias, J., Paredes, C., Fonseca, I., Araujo, H., Batista, J., Almeida, A.T.: Simulating pursuit with machine experiments with robots and artificial vision. IEEE Transactions on Robotics and Automation 14, 1–18 (1998)
Adams, M.D.: High speed target pursuit and asymptotic stability in mobile robotics. IEEE Transactions on Robotics and Automation 15, 230–237 (1999)
Ge, S.S., Cui, Y.J.: Dynamic motion planning for mobile robots using potential field method. Autonomous Robots 13, 207–222 (2002)
Belkhouche, F., Belkhouche, B., Rastgoufard, P.: Line of sight robot navigation toward a moving goal. IEEE Transactions on System Man and Cybernetic Part b-Cybernetic 36, 255–267 (2006)
Yang, C.D., Yang, C.C.: A unified approach to proportional navigation. IEEE Transactions on Aerospace and Electronic Systems 33, 557–567 (1997)
Shukla, U.S., Mahapatra, P.R.: The Proportional Navigation Dilemma - Pure or True. IEEE Transactions on Aerospace and Electronic Systems 26, 382–392 (1990)
Yuan, P.J., Chern, J.S.: Ideal Proportional Navigation. Journal of Guidance Control and Dynamics 15, 1161–1165 (1992)
Borg, J.M., Mehrandezh, M., Fenton, R.G., Benhabib, B.: An Ideal Proportional Navigation Guidance system for moving object interception-robotic experiments. In: Systems, Man, and Cybernetics, 2000 IEEE International Conference, vol. 5, pp. 3247–3252 (2000)
Mehrandezh, M., Sela, N.M., Fenton, R.G., Benhabib, B.: Robotic interception of moving objects using an augmented ideal proportional navigation guidance technique. IEEE Transactions on Systems Man and Cybernetics Part a-Systems and Humans 30, 238–250 (2000)
Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction. MIT Press, Cambridge (1998)
Kaelbling, L.P.: Learning in embedded systems. MIT Press, Cambridge (1993)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)
Sutton, R.S.: Reinforcement Learning. The International Series in Engineering and Computer Science, vol. 173. Kluwer Academic Publishers, Dordrecht (1992)
Ngai, D.C.K., Yung, N.H.C.: Double action Q-learning for obstacle avoidance in a dynamically changing environment. In: Proceedings of the 2005 IEEE Intelligent Vehicles Symposium, Las Vegas, pp. 211–216 (2005)
Ngai, D.C.K., Yung, N.H.C.: Performance Evaluation of Double Action Q-Learning in Moving Obstacle Avoidance Problem. In: Proceedings of the 2005 IEEE International Conference on Systems, Man, and Cybernetics, Hawaii, October 2005, pp. 865–870 (2005)
Watkins, C.J.C.H., Dayan, P.: Technical Note: Q-learning. Machine Learning 8, 279–292 (1992)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ngai, D.C.K., Yung, N.H.C. (2007). Fast-Maneuvering Target Seeking Based on Double-Action Q-Learning. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2007. Lecture Notes in Computer Science(), vol 4571. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73499-4_49
Download citation
DOI: https://doi.org/10.1007/978-3-540-73499-4_49
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73498-7
Online ISBN: 978-3-540-73499-4
eBook Packages: Computer ScienceComputer Science (R0)