Abstract
This chapter describes a methodology to deal with the interaction (negotiation) between MPC controllers in a distributed MPC architecture. This approach combines ideas from Distributed Artificial Intelligence (DAI) and Reinforcement Learning (RL) in order to provide a controller interaction based on negotiation, cooperation and learning techniques. The aim of this methodology is to provide a general structure to perform optimal control in networked distributed environments, where multiple dependencies between subsystems are found. Those dependencies or connections often correspond to control variables. In that case, the distributed control has to be consistent in each subsystem. One of the main new concepts of this architecture is the negotiator agent. Negotiator agents interact with MPC agents to reach an agreement on the optimal value of the shared control variables. The optimal value of those shared control variables has to accomplish a common goal, probably incompatible with the specific goals of each partition that share the variable. Two cases of study are discussed, a small water distribution network and the Barcelona water network. The results suggest that this approach is a promising strategy when centralized control is not a reasonable choice.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
D. Barcelli, Decomposizione ottima e controllo predittivo distribuito della rete idrica di Barcellona (Università di Siena, Facoltà di Ingegneria Informatica, Master’s thesis, 2008)
M. Brdys, B. Ulanicki, Operational Control of Water Systems: Structures, Algorithms and Applications (Prentice Hall International, Hemel Hempstead, Hertfordshire, 1994)
V. Fambrini, C. Ocampo-Martinez, Modelling and decentralized Model Predictive Control of drinking water networks. Technical Report IRI-TR-04-09, Institut de Robòtica i Informàtica Industrial (CSIC-UPC), April 2009
H. El Fawal, D. Georges, and G. Bornard, Optimal control of complex irrigation systems via descomposition-coordination and the use of augmented Lagrangian, in IEEE International Conference Systems, Man and Cybernetics, San Diego, 1998, pp. 3874–3879
M. Gómez, J. Rodellar, F. Vea, J. Mantecón, and J. Cardona, Decentralized adaptive control for water distribution, in IEEE International on Systems, Man and Cybernetics, San Diego, 1998, pp. 1411–1416
T. Jaakkola, M.I. Jordan, S.P. Singh, Q-learning. Mach. Learn. 8, 1185–1201 (1994)
V. Javalera, B. Morcego, V. Puig, Distributed MPC for large scale systems using agent-based reinforcement learning, in IFAC Symposium Large Scale Systems, Lille 2010
R.R. Negenborn, B. De Schutter, J. Hellendoorn, Multi-agent model predictive control for transportation networks: Serial vs. parallel schemes. Eng. Appl. Artif. Intell. 21(3), 353–366 (April 2008)
C. Ocampo-Martinez, D. Barcelli, V. Puig, and A. Bemporad, Hierarchical and decentralised model predictive control of drinking water networks: Application to the barcelona case study. IET Control Theory & Applications, Conditionally accepted, 2011
C. Ocampo-Martinez, S. Bovo, V. Puig, Partitioning approach oriented to the decentralised predictive control of large-scale systems. J. Process Control 21(5), 775–786 (2011)
J. Quevedo, V. Puig, G. Cembrano, J. Blanch, Validation and reconstruction of flow meter data in the Barcelona water distribution network. Control Eng. Pract. 11(6), 640–651 (June 2010)
J.B. Rawlings, B. Stewart, Coordinating multiple optimization-based controllers: New opportunities and challenges. J. Process Control 18(9), 839–845 (2008)
D.D. Šiljak, Decentralized Control of Complex Systems (Academic Press, New York, 1991)
J.N. Tsitsiklis, Asynchronous stochastic approximation and Q-learning. Mach. Learn. 16, 185–202 (1994)
A.N. Venkat, J.B. Rawlings, S.J. Wrigth, Stability and optimality of distributed model predictive control, in IEEE Conference on Decision and Control and European Control Conference, Seville, 2005
C.I.C.H. Watkins, Learning from Delayed Rewards. Doctoral Dissertation (University of Cambridge, Cambridge, 1989)
C.I.C.H. Watkins, P. Dayan, Q-learning. Mach. Learn. 8, 279–292 (1992)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Morcego, B., Javalera, V., Puig, V., Vito, R. (2014). Distributed MPC Using Reinforcement Learning Based Negotiation: Application to Large Scale Systems. In: Maestre, J., Negenborn, R. (eds) Distributed Model Predictive Control Made Easy. Intelligent Systems, Control and Automation: Science and Engineering, vol 69. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-7006-5_32
Download citation
DOI: https://doi.org/10.1007/978-94-007-7006-5_32
Published:
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-007-7005-8
Online ISBN: 978-94-007-7006-5
eBook Packages: EngineeringEngineering (R0)