Abstract
In many multi-agent control problems, the ability to compute an optimal feedback control is severely limited by the dimension of the state space. In this work, deterministic, nonholonomic agents are tasked with creating and maintaining a formation based on observations of their neighbors, and each agent in the formation independently computes its feedback control from a Hamilton-Jacobi-Bellman (HJB) equation. Since an agent does not have knowledge of its neighbors’ future motion, we assume that the unknown control to be applied by neighbors can be modeled as Brownian motion. The resulting probability distribution of its neighbors’ future trajectory allows the HJB equation to be written as a path integral over the distribution of optimal trajectories. We describe how the path integral approach to stochastic optimal control allows the distributed control problems to be written as independent Kalman smoothing problems over the probability distribution of the connected agents’ future trajectories. Simulations show five unicycles achieving the formation of a regular pentagon.
Keywords
- Multiagent System
- Stochastic Optimal Control
- Future Trajectory
- Optimal Feedback Control
- Path Integral Approach
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Anderson, B., Fidan, B., Yu, C., Walle, D.: UAV formation control: theory and application. In: Blondel, V., Boyd, S., Kimura, H. (eds.) Recent Advances in Learning and Control, pp. 15–34. Springer (2008)
Bell, B.M., Burke, J.V., Pillonetto, G.: An inequality constrained nonlinear Kalman-Bucy smoother by interior point likelihood maximization. Automatica 45(1), 25–33 (2009)
Bellingham, J.G., Rajan, K.: Robotics in remote and hostile environments. Science 318(5853), 1098–1102 (2007)
Bertsekas, D.P., Tsitsiklis, J.N.: Neuro-dynamic programming. Athena Scientific, Belmont (1996)
van den Broek, B., Wiegerinck, W., Kappen, B.: Graphical model inference in optimal control of stochastic multi-agent systems. Journal of Artificial Intelligence Research 32(1), 95–122 (2008)
van den Broek, B., Wiegerinck, W., Kappen, B.: Optimal Control in Large Stochastic Multi-agent Systems. In: Tuyls, K., Nowe, A., Guessoum, Z., Kudenko, D. (eds.) Adaptive Agents and MAS III. LNCS (LNAI), vol. 4865, pp. 15–26. Springer, Heidelberg (2008)
Bullo, F., Cortes, J., Martinez, S.: Distributed control of robotic networks: A mathematical approach to motion coordination algorithms. Princeton University Press, Princeton (2009)
Dimarogonas, D.: On the rendezvous problem for multiple nonholonomic agents. IEEE Transactions on Automatic Control 52(5), 916–922 (2007)
Elkaim, G., Kelbley, R.: A Lightweight Formation Control Methodology for a Swarm of Non-Holonomic Vehicles. In: IEEE Aerospace Conference. IEEE, Big Sky (2006)
Fleming, W., Soner, H.: Logarithmic Transformations and Risk Sensitivity. In: Controlled Markov Processes and Viscosity Solutions, ch.6. Springer, Berlin (1993)
Freidlin, M., Wentzell, A.: Random Perturbations of Dynamical Systems. Springer (1984)
Gamerman, D.: Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference. Chapman and Hall (1997)
Gelb, A.: Applied Optimal Estimation. The M.I.T. Press, Cambridge (1974)
Goldstein, H.: Classical Mechanics, 2nd edn. Addison-Wesley (1980)
van Kampen, N.G.: Stochastic Processes in Physics and Chemistry, 3rd edn. North-Holland (2007)
Kappen, H.: Linear Theory for Control of Nonlinear Stochastic Systems. Physical Review Letters 95(20), 1–4 (2005)
Kappen, H.: Path integrals and symmetry breaking for optimal control theory. Journal of Statistical Mechanics: Theory and Experiment 2005, P11,011 (2005)
Kappen, H., Wiegerinck, W., van den Broek, B.: A path integral approach to agent planning. In: Autonomous Agents and Multi-Agent Systems, Citeseer (2007)
Kappen, H.J.: Path integrals and symmetry breaking for optimal control theory. Journal of Statistical Mechanics, Theory and Experiment 2005, 21 (2005)
Kappen, H.J., Gómez, V., Opper, M.: Optimal control as a graphical model inference problem. Machine Learning 87(2), 159–182 (2012)
Khas’minskii, R.: On the principle of averaging the Itô’s stochastic differential equations. Kybernetika 4(3), 260–279 (1968)
Kumar, V., Rus, D., Sukhatme, G.S.: Networked Robotics. In: Sciliano, B., Khatib, O. (eds.) Springer Handbook of Robotics. ch. 41, pp. 943–958 (2008)
Kushner, H.J., Dupuis, P.: Numerical Methods for Stochastic Control Problems in Continuous Time, 2nd edn. Springer (2001)
Milutinović, D.: Utilizing Stochastic Processes for Computing Distributions of Large-Size Robot Population Optimal Centralized Control. In: Martinoli, A., Mondada, F., Correll, N., Mermoud, G., Egerstedt, M., Hsieh, M.A., Parker, L.E., Støy, K. (eds.) Distributed Autonomous Robotic Systems. STAR, vol. 83, pp. 359–372. Springer, Heidelberg (2013)
Milutinović, D., Garg, D.P.: Stochastic model-based control of multi-robot systems. Tech. Rep. 0704, Duke University, Durham, NC (2009)
Milutinović, D., Garg, D.P.: A sampling approach to modeling and control of a large-size robot population. In: Proceedings of the 2010 ASME Dynamic Systems and Control Conference. ASME, Boston (2010)
Oksendal, B.: Stochastic Differential Equations: An Introduction with Applications, 6th edn. Springer, Berlin (2003)
Palmer, A., Milutinović, D.: A Hamiltonian Approach Using Partial Differential Equations for Open-Loop Stochastic Optimal Control. In: Proceedings of the 2011 American Control Conference, San Francisco, CA (2011)
Parker, L.E.: Multiple Mobile Robot Systems. In: Sciliano, B., Khatib, O. (eds.) Springer Handbook of Robotics, ch.40, pp. 921–941. Springer (2008)
Powell, W.B.: Approximate Dynamic Programming: Solving the Curses of Dimensionality. Wiley Interscience, Hoboken (2007)
Ren, W., Beard, R.: Distributed consensus in multi-vehicle cooperative control: Theory and applications. Springer, New York (2007)
Ren, W., Beard, R., Atkins, E.: A survey of consensus problems in multi-agent coordination. In: Proceedings of the 2005, American Control Conference, pp. 1859–1864 (2005)
Ryan, A., Zennaro, M., Howell, A., Sengupta, R., Hedrick, J.: An overview of emerging results in cooperative UAV control. In: 2004 43rd IEEE Conference on Decision and Control, vol. 1, pp. 602–607 (2004)
Särkkä, S.: Continuous-time and continuous-discrete-time unscented Rauch-Tung-Striebel smoothers. Signal Processing 90(1), 225–235 (2010)
Singh, A., Batalin, M., Stealey, M., Chen, V., Hansen, M., Harmon, T., Sukhatme, G., Kaiser, W.: Mobile robot sensing for environmental applications. In: Field and Service Robotics, pp. 125–135. Springer (2008)
Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Theodorou, E., Buchli, J., Schaal, S.: A Generalized Path Integral Control Approach to Reinforcement Learning. The Journal of Machine Learning Research 11, 3137–3181 (2010)
Theodorou, E., Buchli, J., Schaal, S.: Reinforcement learning of motor skills in high dimensions: A path integral approach. In: 2010 IEEE International Conference on Robotics and Automation (ICRA), vol. 4, pp. 2397–2403. IEEE (2010)
Todorov, E.: General duality between optimal control and estimation. In: 47th IEEE Conference on Decision and Control, vol. 5, pp. 4286–4292. IEEE, Cancun (2008)
Wang, M.C., Uhlenbeck, G.: On the theory of Brownian Motion II. Reviews of Modern Physics 17(2-3), 323–342 (1945)
Wiegerinck, W., van den Broek, B., Kappen, B.: Stochastic optimal control in continuous space-time multi-agent systems. In: Proceedings UAI. Citeseer (2006)
Wiegerinck, W., van den Broek, B., Kappen, B.: Optimal on-line scheduling in stochastic multiagent systems in continuous space-time. In: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems, p. 1 (2007)
Yin, G.G., Zhu, C.: Hybrid Switching Diffusions. Springer, New York (2010)
Yong, J.: Relations among ODEs, PDEs, FSDEs, BDSEs, and FBSDEs. In: Proceedings of the 36th IEEE Conference on Decision and Control, pp. 2779–2784. IEEE, San Diego (1997)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Anderson, R.P., Milutinović, D. (2013). Kalman Smoothing for Distributed Optimal Feedback Control of Unicycle Formations. In: Milutinović, D., Rosen, J. (eds) Redundancy in Robot Manipulators and Multi-Robot Systems. Lecture Notes in Electrical Engineering, vol 57. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33971-4_9
Download citation
DOI: https://doi.org/10.1007/978-3-642-33971-4_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33970-7
Online ISBN: 978-3-642-33971-4
eBook Packages: EngineeringEngineering (R0)