Abstract
Ant colony optimization (ACO) is an optimization heuristic for solving combinatorial optimization problems and is inspired by the swarming behavior of foraging ants. ACO has been successfully applied in various domains, such as routing and scheduling. In particular, the agents, called ants here, are very efficient at sampling the problem space and quickly finding good solutions. Motivated by the advantages of ACO in combinatorial optimization, we develop a novel framework for finding optimal control policies that we call Ant Colony Learning (ACL). In ACL, the ants all work together to collectively learn optimal control policies for any given control problem for a system with nonlinear dynamics. In this chapter, we discuss the ACL framework and its implementation with crisp and fuzzy partitioning of the state space. We demonstrate the use of both versions in the control problem of two-dimensional navigation in an environment with variable damping and discuss their performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Dorigo, M., Blum, C.: Ant colony optimization theory: a survey. Theoretical Computer Science 344(2-3), 243–278 (2005)
Colorni, A., Dorigo, M., Maniezzo, V.: Distributed optimization by ant colonies. In: Varela, F.J., Bourgine, P. (eds.) Towards a Practice of Autonomous Systems: Proceedings of the First European Conference on Artificial Life, pp. 134–142. MIT Press, Cambridge (1992)
Dorigo, M., Stützle, T.: Ant Colony Optimization. The MIT Press, Cambridge (2004)
Sim, K.M., Sun, W.H.: Ant colony optimization for routing and load-balancing: survey and new directions. IEEE Transactions on Systems, Man, Cybernetics, Part A 33(5), 560–572 (2003)
Huang, R.H., Yang, C.L.: Ant colony system for job shop scheduling with time windows. International Journal of Advanced Manufacturing Technology 39(1-2), 151–157 (2008)
Alaykran, K., Engin, O., Dyen, A.: Using ant colony optimization to solve hybrid flow shop scheduling problems. International Journal of Advanced Manufacturing Technology 35(5-6), 541–550 (2007)
Fan, X., Luo, X., Yi, S., Yang, S., Zhang, H.: Optimal path planning for mobile robots based on intensified ant colony optimization algorithm. In: Proceedings of the IEEE International Conference on Robotics, Intelligent Systems and Signal Processing (RISSP 2003), Changsha, Hunan, China, October 2003, pp. 131–136 (2003)
Wang, J., Osagie, E., Thulasiraman, P., Thulasiram, R.K.: HOPNET: A hybrid ant colony optimization routing algorithm for mobile ad hoc network. Ad Hoc Networks 7(4), 690–705 (2009)
Purnamadjaja, A.H., Russell, R.A.: Pheromone communication in a robot swarm: necrophoric bee behaviour and its replication. Robotica 23(6), 731–742 (2005)
Fox, B., Xiang, W., Lee, H.P.: Industrial applications of the ant colony optimization algorithm. International Journal of Advanced Manufacturing Technology 31(7-8), 805–814 (2007)
Bianchi, L., Dorigo, M., Gambardella, L.M., Gutjahr, W.J.: Metaheuristics in stochastic combinatorial optimization: a survey. IDSIA, Manno, Switzerland, Tech. Rep. 08 (March 2006)
van Ast, J.M., Babuška, R., De Schutter, B.: Novel ant colony optimization approach to optimal control. International Journal of Intelligent Computing and Cybernetics 2(3), 414–434 (2009)
van Ast, J.M., Babuška, R., De Schutter, B.: Fuzzy ant colony optimization for optimal control. In: Proceedings of the American Control Conference (ACC 2009), Saint Louis, MO, USA, June 2009, pp. 1003–1008 (2009)
Socha, K., Blum, C.: An ant colony optimization algorithm for continuous optimization: application to feed-forward neural network training. Neural Computing & Applications 16(3), 235–247 (2007)
Socha, K., Dorigo, M.: Ant colony optimization for continuous domains. European Journal of Operational Research 185(3), 1155–1173 (2008)
Bilchev, G., Parmee, I.C.: The ant colony metaphor for searching continuous design spaces. In: Fogarty, T. (ed.) AISB-WS 1995. LNCS, vol. 993, pp. 25–39. Springer, Heidelberg (1995)
Tsutsui, S., Pelikan, M., Ghosh, A.: Performance of aggregation pheromone system on unimodal and multimodal problems. In: Proceedings of the 2005 Congress on Evolutionary Computation (CEC 2005), Edinburgh, Scotland, September 2005, pp. 880–887 (2005)
Korosec, P., Silc, J., Oblak, K., Kosel, F.: The differential ant-stigmergy algorithm: an experimental evaluation and a real-world application. In: Proceedings of the 2007 Congress on Evolutionary Computation (CEC 2007), Singapore, September 2007, pp. 157–164 (2007)
Birattari, M., Caro, G.D., Dorigo, M.: Toward the formal foundation of Ant Programming. In: Proceedings of the International Workshop on Ant Algorithms (ANTS 2002), pp. 188–201. Springer, Brussels (2002)
Gambardella, L.M., Dorigo, M.: Ant-Q: A reinforcement learning approach to the traveling salesman problem. In: Prieditis, A., Russell, S. (eds.) Machine Learning: Proceedings of the Twelfth International Conference on Machine Learning, pp. 252–260. Morgan Kaufmann Publishers, San Francisco (1995)
Casillas, J., Cordn, O., Herrera, F.: Learning fuzzy rule-based systems using ant colony optimization algorithms. In: Proceedings of the ANTS 2000. From Ant Colonies to Artificial Ants: Second International Workshop on Ant Algorithms, Brussels, Belgium, September 2000, pp. 13–21 (2000)
Zhao, B., Li, S.: Design of a fuzzy logic controller by ant colony algorithm with application to an inverted pendulum system. In: Proceedings of the IEEE International Conference on Systems, Man, Cybernetics, Taipei, Taiwan, October 2006, pp. 3790–3794 (2006)
Zhu, W., Chen, J., Zhu, B.: Optimal design of fuzzy controller based on ant colony algorithms. In: Proceedings of the IEEE International Conference on Mechatronics and Automation, Luoyang, China, June 2006, pp. 1603–1607 (2006)
Buşoniu, L., Ernst, D., De Schutter, B., Babuška, R.: Continuous-state reinforcement learning with fuzzy approximation. In: Tuyls, K., Nowe, A., Guessoum, Z., Kudenko, D. (eds.) ALAMAS 2005, ALAMAS 2006, and ALAMAS 2007. LNCS (LNAI), vol. 4865, pp. 27–43. Springer, Heidelberg (2008)
Dorigo, M., Maniezzo, V., Colorni, A.: Ant system: optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics, Part B 26(1), 29–41 (1996)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Dorigo, M., Gambardella, L.: Ant Colony System: a cooperative learning approach to the traveling salesman problem. IEEE Transactions on Evolutionary Computation 1(1), 53–66 (1997)
Stützle, T., Hoos, U.: MAX MIN Ant System. Journal of Future Generation Computer Systems 16, 889–914 (2000)
Åström, K.J., Wittenmark, B.: Computer Controlled Systems—Theory and Design. Prentice-Hall, Englewood Cliffs (1990)
Watkins, C., Dayan, P.: Q-learning. Machine Learning 8(3-4), 279–292 (1992)
Rummery, G.A., Niranjan, M.: On-line Q-learning using connectionist systems. Cambridge University, Tech. Rep. CUED/F-INFENG/TR166 (1994)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
van Ast, J.M., Babuška, R., De Schutter, B. (2010). Ant Colony Learning Algorithm for Optimal Control. In: Babuška, R., Groen, F.C.A. (eds) Interactive Collaborative Information Systems. Studies in Computational Intelligence, vol 281. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-11688-9_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-11688-9_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-11687-2
Online ISBN: 978-3-642-11688-9
eBook Packages: EngineeringEngineering (R0)