Control Strategies for the Dynamics of Large Particle Systems

Michael Herty; Lorenzo Pareschi; Sonja Steffensen

doi:10.1007/978-3-030-20297-2_5

Active Particles, Volume 2 pp 149-171 | Cite as

Control Strategies for the Dynamics of Large Particle Systems

Authors
Authors and affiliations

Michael Herty
Lorenzo Pareschi
Sonja Steffensen

Chapter

First Online: 23 August 2019

231 Downloads

Part of the Modeling and Simulation in Science, Engineering and Technology book series (MSSET)

Abstract

We survey some recent approaches to control problems for large particle systems. Particle systems are transversal to many applications, ranging from classical physics to social sciences. The temporal evolution of the particles is determined by deterministic or stochastic dynamics and they are additionally able to optimize their trajectory over a large time. In particular, we investigate the limit of infinitely many particles leading to control of kinetic partial differential equations. To this goal a different notion of differentiability of the meanfield equation is introduced. Different mathematical methods based on meanfield games, model predictive control, and optimal control techniques will be discussed.

This is a preview of subscription content, to check access.

Notes

Acknowledgements

This work has been supported by DFG HE5386/13,14,15-1, BMBF 05M18PAA, and the DAAD-MIUR project.

Appendix: Notion of Differentiability and Calculus for Consistent Derivation

This section is devoted to more details on the notion of differentiability. In order to keep the presentation simple on a technical level we consider the following particle system. Consider N particles i with state space $x_i\in \mathbb {R}$, a sufficiently smooth function $\phi , \psi :\mathbb {R} \to \mathbb {R}$ and assume each particle fulfills

$$\displaystyle \begin{aligned} x^{\prime}_i(t) = \frac{1}N \sum_{j=1}^N \phi(x_j(t)), x_i(0)=u_{i}. \end{aligned} $$

(27)

Here, u_i is the unknown control applied as initial datum and we assume that the previous dynamics is solved on a time interval t ∈ (0, T). The control $U=(u_i)_{i=1}^N \in \mathbb {R}^N$ is chosen to minimize the cost $J_N:\mathbb {R}^N \to \mathbb {R}$

$$\displaystyle \begin{aligned} J_N(U)=\int_0^T \frac{1}{N}\sum_{i=1}^N \psi(x_i(t)) dt, \end{aligned} $$

(28)

where x_i are the solution to (27). Standard theory like Pontryagin’s maximum principle can be applied to solve the minimization problem

$$\displaystyle \begin{aligned} U^*_N = \mbox{ argmin }_{U\in\mathbb{R}^N} J_N(U). \end{aligned} $$

(29)

We are interested in the relation of the first-order optimality conditions to (29) in the case N →∞. We introduce for each fixed t the probability density $\mu (t,\cdot )\in \mathcal {P}(\mathbb {R})$, i.e., μ(t, ⋅) ≥ 0 and $ \int _{\mathbb {R}} \mu (t,x)dx = 1.$ The probability density describes the probability to have particles at time t with property x. In the meanfield limit the dynamics of μ is obtained by (27) as

$$\displaystyle \begin{aligned} \partial_t \mu(t,x) + \partial_x \int \phi(y) \mu(t,y) \mu(t,x) dy = 0, \quad \mu(0,x) = v(x), \end{aligned} $$

(30)

where $\int v(x) dx=1$ is the corresponding control. We introduce the empirical measure on $\mathbb {R}$ as $\nu _{X}(x)=\frac {1}N\sum _i \delta (x-x_i)$ with $X=(x_i)_{i=1}^N.$ Then, for $\mu (t,x)=\nu _{X(t)}= \frac {1}N\sum _i \delta (x-x_i(t))$ and v = ν_U we recover (27) from the weak form of Eq. (30) and vice versa. The meanfield limit of the sequence (J_N)_N of cost functional for N →∞ is obtained as $\mathcal {J}:\mathcal {P}(\mathbb {R})\to \mathbb {R}$, where

$$\displaystyle \begin{aligned} \mathcal{J}(v)=\int_0^T \int \psi(x) \mu(t,x) dx dt, \end{aligned} $$

(31)

and where μ solves Eq. (30). Hence, on the level of the meanfield limit we may formulate the problem

$$\displaystyle \begin{aligned} v^* = \mbox{ argmin }_{v \in\mathcal{P}(\mathbb{R})} \mathcal{J}(v). \end{aligned} $$

(32)

The latter is an optimal control problem with differential equations as constraints and may be solved using techniques from infinite-dimensional optimal control theory. The main result of this section is to establish the relation between the meanfield limit in the Pontryagin’s maximum principle and the formal adjoint calculus applied to the optimal control problem (32).

Remark 2

A simple application of formal Lagrange calculus to problem (32) yields as the adjoint equation

$$\displaystyle \begin{aligned} - \partial_t \lambda - \partial_x \lambda \left( \int \phi \mu dy \right) - \int \mu \partial_x \lambda dy \phi = \psi(x) , \qquad \lambda(T,0)=0. \end{aligned} $$

(33)

From the previous equation we observe that λ = ν_L with $L=(\ell _i)_{i=1}^N$ is not a solution since $\int \lambda dx$ is not conserved. Further, from Pontryagin’s maximum principle we expect the N adjoints ℓ_i(t) to fulfill

$$\displaystyle \begin{aligned} - \ell^{\prime}_i(t) = \psi'(x_i(t)) + \frac{1}N \sum_{j=1}^N \ell_j(t) \phi'(x_i(t)), \qquad \ell_i(T)=0, \end{aligned} $$

(34)

involving the derivatives of ψ and ϕ that are not present in (33).

The problem with the formal Lagrange calculus is the fact that Eq. (30) is posed on the space $\mathcal {P}(\mathbb {R})$ that is not a vector space. Therefore, we consider geometric derivatives also used, e.g., in [8]. As preliminary discussion note that if

$$\displaystyle \begin{aligned}\frac{d}{ds} y_i(s) = c,\qquad y_i(0)=x_{i,0}\end{aligned}$$

holds for i = 1 : N, then the empirical measure fulfills in weak form

$$\displaystyle \begin{aligned}\partial_s \nu_{Y(s)}(z) + \partial_z c, \qquad \nu_{Y(s)}(z) = 0\end{aligned}$$

for initial condition

$$\displaystyle \begin{aligned}\nu_{Y(0)}(z) = \nu_{X_{0}}.\end{aligned}$$

Clearly, ν is a probability measure. It also serves as first-order variation of the points x_i,0, i = 1 : N, in the direction of the given velocity field c in the following sense: We define the derivative of $\mathcal {J}$ at measure $\mu =\nu _{X_{0}}$ in direction c by considering the measure associated with moving all points X₀ with velocity c. This measure is $\eta (t,\cdot ):=\nu _{Y(t)} \in \mathcal {P}(\mathbb {R}).$ The derivative of $\mathcal {J}$ is then infinitesimal difference at t = 0 between the limit

$$\displaystyle \begin{aligned}\frac{d\mathcal{J}}{d\mu}(\mu) c := \lim\limits_{t\to 0} \frac{1}t \left( \mathcal{J}(\eta(t,\cdot))- \mathcal{J}(\eta(0,\cdot)) \right),\end{aligned}$$

where ∂_tη + ∂_x(cη) = 0, η(0) = μ. For the example $\mathcal {J}(\mu ) = \int \psi (x) \mu (x) dx$ we obtain

$$\displaystyle \begin{aligned} \frac{dJ}{d\mu}(\mu) c = \lim_{t\to 0} \frac{ J(\eta(t,\cdot))-J(\mu)}t = \\ \lim_{t\to 0} \frac{1}t \int \psi( \eta(t,x) - \eta(0,x) ) dx = \int \psi \eta_t(t,x) dx = \\ - \int \psi ( \eta(t,x) c )_x dx = \int \psi'(x) c(x) \mu(x) dx. \end{aligned} $$

A similar formalism is applied to the weak form of Eq. (30). We write for a function $\rho \in C^2(\mathbb {R})$ and probability measures μ(t, x) and μ(t, y) the weak form as operator A as

$$\displaystyle \begin{aligned}<A \mu, \rho>:= \int_0^T \int \rho_t + \rho_x \int \phi d\mu(t,y) d\mu(t,x) dt.\end{aligned}$$

Applying a similar calculus we obtain

$$\displaystyle \begin{aligned} \frac{d}{d\mu} <A \mu, \rho>c &= \int_0^T \int \rho_{tx} c(x) d\mu(t,x) dt \\ &\quad + \int_0^T \int \int \rho_{x} \phi'(y) c(y) d\mu(t,y)d\mu(t,x) dt\\ &\quad + \int_0^T \int \int \rho_{xx} \phi(y) c(x)d\mu(t,y)d\mu(t,x) dt. \end{aligned} $$

In order to derive first-order optimality conditions for the problem (32) we consider the Lagrangian $\mathcal {L}:\mathcal {P}(\mathbb {R}) \times \mathcal {P}(\mathbb {R}) \times (\mathcal {P}(\mathbb {R}))' \to \mathbb {R}$ given by

$$\displaystyle \begin{aligned}\mathcal{L}(v,\mu,\rho ) = \int_0^T \int \psi(x) d\mu(t,x) dt + <A\mu,\rho>+ \int \rho(0,x) dv(x).\end{aligned}$$

Then, the formal optimality conditions are obtained as saddle point of the Lagrangian $\mathcal {L}$ for everey field c in weak form and are given by

$$\displaystyle \begin{aligned} &\int_0^T \int \rho_t + \rho_x \int \phi d\mu(t,y) d\mu(t,x) dt + \int \rho dv(x) =0, {} \end{aligned} $$

(35a)

$$\displaystyle \begin{aligned} &\int_0^T \int \rho_{tx} c(x) d\mu(t,x) dt + \int_0^T \int \int \rho_{x} \phi'(y) c(y) d\mu(t,y) d\mu(t,x) dt{} \end{aligned} $$

(35b)

$$\displaystyle \begin{aligned} &\quad +\int_0^T \int \int \rho_{xx} \phi(y) c(x)d\mu(t,y) d\mu(t,x) dt + \int_0^T \int \psi'(x) c(x) d\mu(t,x) dt =0, \\ &\int \rho_x(0,x)c(x) d v(x) = 0.{} \end{aligned} $$

(35c)

Finally, we show that Eqs. (34) and (27) are recovered from (35) using the empirical measure. Fix N and $U=(u_i)_{i=1}^N.$ Let $v(x):=\frac {1}N\sum _{i=1}^N \delta (x-u_i).$ Then, a weak solution to (30) is given by $\mu (t,x):=\frac {1}N\sum _i \delta (x-x_i(t))$ provided that x_i fulfills (27). Further, we denote by $c_i(t) = \int c(x) \delta (x-x_i(t)) dx$ for any c to get

$$\displaystyle \begin{aligned} &\frac{1}N \int_0^T \sum_i c_i(t) \left( \psi'(x_i) + \rho_{tx}(t,x_i) +\rho_{xx}(t,x_i) \left( \frac{1}N\sum_j \phi(x_j) \right) \right.\\ &\left. \qquad + \phi'(x_i) \left( \frac{1}N\sum_j \rho_x(t,x_j) \right) \right) dt = 0. \end{aligned} $$

Since the previous equation has to hold true for any function c this yields

$$\displaystyle \begin{aligned} \psi'(x_i) + \rho_{tx}(t,x_i) +\rho_{xx}(t,x_i) \left ( \frac{1}N\sum\phi(x_j) \right) + \phi'(x_i) \left( \frac{1}N\sum\rho_x(t,x_j) \right) = 0. \end{aligned} $$

Define now

$$\displaystyle \begin{aligned}\lambda_i(t):= \rho_x(t,x_i(t)), i=1,\dots,N.\end{aligned}$$

Then,

$$\displaystyle \begin{aligned}\lambda_i^{\prime}(t) = \rho_{tx}(t,x_i) + \rho_{xx}(t,x_i)x_i^{\prime}(t)\end{aligned}$$

and therefore

$$\displaystyle \begin{aligned}\psi'(x_i) + \lambda^{\prime}_i(t) + \rho_{xx}(t,x_i)\left( - x_i^{\prime} + \frac{1}N\sum_j\phi(x_j) \right) + \phi'(x_i) \frac{1}N \sum \lambda_j =0.\end{aligned}$$

Since $x_i^{\prime }$ fulfills (27) we obtain that λ_i ≡ ℓ_i for all i and fulfills (34). Summarizing, the first-order optimality conditions for the meanfield problem (32) coincide with the first-order optimality conditions for (29) provided the previous notion of differentiability in $\mathcal {P}(\mathbb {R})$ is used. The following relation holds true for the corresponding Lagrange multipliers:

$$\displaystyle \begin{aligned}\ell_i(t) := \partial_x \rho(t,x_i(t)).\end{aligned}$$

References

1.
G. Albi, M. Bongini, E. Cristiani, and D. Kalise, Invisible control of self-organizing agents leaving unknown environments, SIAM J. Appl. Math., 76 (2016), pp. 1683–1710.MathSciNetCrossRefGoogle Scholar
2.
G. Albi, Y.-P. Choi, M. Fornasier, and D. Kalise, Mean field control hierarchy, Appl. Math. Optim., 76 (2017), pp. 93–135.MathSciNetCrossRefGoogle Scholar
3.
G. Albi, M. Fornasier, and D. Kalise, A Boltzmann approach to mean-field sparse feedback control, IFAC-PapersOnLine, 50 (2017), pp. 2898–2903. 20th IFAC World Congress.Google Scholar
4.
G. Albi, M. Herty, and L. Pareschi, Kinetic description of optimal control problems and applications to opinion consensus, Comm. Math. Sci., 13 (2015), pp. 1407–1429.MathSciNetCrossRefGoogle Scholar
5.
G. Albi and L. Pareschi, Modeling of self-organized systems interacting with a few individuals: from microscopic to macroscopic dynamics, Appl. Math. Lett., 26 (2013), pp. 397–401.MathSciNetCrossRefGoogle Scholar
6.
G. Albi, L. Pareschi, and M. Zanella, Boltzmann type control of opinion consensus through leaders, Philos. Trans. R. Soc. Lond. Ser. A, 372 (2014).Google Scholar
7.
G. Albi, L. Pareschi, and M. Zanella, Uncertainty quantification in control problems for flocking models, Math. Probl. Eng., (2015), pp. Art. ID 850124, 14.Google Scholar
8.
L. Ambrosio, N. Gigli, and G. Savare, Gradient Flows in Metric Spaces of Probability Measures, Lectures in Mathematics, Birkhäuser Verlag, Basel, Boston, Berlin, 2008.zbMATHGoogle Scholar
9.
N. Bellomo, G. Ajmone Marsan, and A. Tosin, Complex Systems and Society. Modeling and Simulation, SpringerBriefs in Mathematics, Springer, 2013.Google Scholar
10.
N. Bellomo, P. Degond, and E. Tadmor, Active Particles, Volume 1 : Advances in Theory, Models, and Applications, Modeling and Simulation in Science, Engineering and Technology, Birkhäuser Verlag, 2017.Google Scholar
11.
N. Bellomo and J. Soler, On the mathematical theory of the dynamics of swarms viewed as complex systems, Math. Models Methods Appl. Sci., 22 (2012), p. 1140006.MathSciNetCrossRefGoogle Scholar
12.
A. Bensoussan, M. H. M. Chau, Y. Lai, and S. C. P. Yam, Linear-quadratic mean field Stackelberg games with state and control delays, SIAM J. Control Optim., 55 (2017), pp. 2748–2781.MathSciNetCrossRefGoogle Scholar
13.
A. Bensoussan, J. Frehse, and P. Yam, Mean Field Games and Mean Field Type Control Theory, Series: SpringerBriefs in Mathematics, New York, 2013.Google Scholar
14.
M. Bongini, M. Fornasier, O. Junge, and B. Scharf, Sparse control of alignment models in high dimension, Netw. Heterog. Media, 10 (2015), pp. 647–697.MathSciNetCrossRefGoogle Scholar
15.
A. Borzì and S. Wongkaew, Modeling and control through leadership of a refined flocking system, Math. Models Methods Appl. Sci., 25 (2015), pp. 255–282.MathSciNetCrossRefGoogle Scholar
16.
M. Burger, M. Di Francesco, P. A. Markowich, and M.-T. Wolfram, Mean field games with nonlinear mobilities in pedestrian dynamics, Discrete Contin. Dyn. Syst. Ser. B, 19 (2014), pp. 1311–1333.MathSciNetCrossRefGoogle Scholar
17.
M. Caponigro, M. Fornasier, B. Piccoli, and E. Trélat, Sparse stabilization and optimal control of the Cucker-Smale model, Math. Control Relat. Fields, 3 (2013), pp. 447–466.MathSciNetCrossRefGoogle Scholar
18.
E. Cristiani, B. Piccoli, and A. Tosin, Multiscale modeling of pedestrian dynamics, vol. 12 of MS&A. Modeling, Simulation and Applications, Springer, Cham, 2014.Google Scholar
19.
P. Degond, M. Herty, and J.-G. Liu, Meanfield games and model predictive control, Comm. Math. Sci., 5 (2017), pp. 1403–1422.MathSciNetCrossRefGoogle Scholar
20.
P. Degond, J.-G. Liu, and C. Ringhofer, Evolution of the distribution of wealth in an economic environment driven by local Nash equilibria, Journal of Statistical Physics, 154 (2014), pp. 751–780.MathSciNetCrossRefGoogle Scholar
21.
P. Degond, J.-G. Liu, and C. Ringhofer, Evolution of wealth in a nonconservative economy driven by local Nash equilibria, Phl. Trans. Roy. Soc. A, 372 (2014).Google Scholar
22.
M. Fornasier, B. Piccoli, and F. Rossi, Mean-field sparse optimal control, Philos. Trans. R. Soc. Lond. Ser. A Math. Phys. Eng. Sci., 372 (2014), pp. 20130400, 21.MathSciNetCrossRefGoogle Scholar
23.
M. Fornasier and F. Solombrino, Mean-field optimal control, ESAIM Control Optim. Calc. Var., 20 (2014), pp. 1123–1152.MathSciNetCrossRefGoogle Scholar
24.
F. Golse, On the dynamics of large particle systems in the mean field limit, in Macroscopic and large scale phenomena: coarse graining, mean field limits and ergodicity, vol. 3 of Lect. Notes Appl. Math. Mech., Springer, [Cham], 2016, pp. 1–144.Google Scholar
25.
L. Grüne, Analysis and design of unconstrained nonlinear MPC schemes for finite and infinite dimensional systems, SIAM J. Control Optim., 48 (2009), pp. 1206–1228.MathSciNetCrossRefGoogle Scholar
26.
R. Hegselmann and U. Krause, Opinion dynamics and bounded confidence, models, analysis and simulation, Journal of Artificial Societies and Social Simulation, 5 (2002), p. 2.Google Scholar
27.
M. Herty, S. Steffensen, and L. Pareschi, Mean-field control and Riccati equations, Netw. Heterog. Media, 10 (2015).Google Scholar
28.
M. Herty and M. Zanella, Performance bounds for the mean-field limit of constrained dynamics, Discrete Contin. Dyn. Syst., 37 (2017), pp. 2023–2043.MathSciNetCrossRefGoogle Scholar
29.
J.-M. Lasry and P.-L. Lions, Mean field games, Japanese Journal of Mathematics, 2 (2007), pp. 229–260.MathSciNetCrossRefGoogle Scholar
30.
D. Q. Mayne and H. Michalska, Receding horizon control of nonlinear systems, IEEE Trans. Automat. Control, 35 (1990), pp. 814–824.MathSciNetCrossRefGoogle Scholar
31.
S. Motsch and E. Tadmor, Heterophilious dynamics enhances consensus, SIAM Rev., 4 (2014), pp. 577–621.MathSciNetCrossRefGoogle Scholar
32.
G. Naldi, L. Pareschi, and G. Toscani, Mathematical Modeling of Collective Behavior in Socio-Economic and Life Sciences, Series: Modeling and Simulation in Science, Engineering and Technology, Birkhauser, Boston, 2010.Google Scholar
33.
L. Pareschi and G. Toscani, Interacting Multiagent Systems: Kinetic Equations and Monte Carlo Methods, Oxford University Press, 2013.zbMATHGoogle Scholar
34.
E. D. Sontag, Mathematical control theory, vol. 6 of Texts in Applied Mathematics, Springer-Verlag, New York, second ed., 1998. Deterministic finite-dimensional systems.Google Scholar

Copyright information

Authors and Affiliations

Michael Herty
- 1
Email author
Lorenzo Pareschi
- 2
Sonja Steffensen
- 1

1.RWTH Aachen UniversityIGPMAachenGermany
2.Department of Mathematics and Computer ScienceUniversity of FerraraFerraraItaly

Control Strategies for the Dynamics of Large Particle Systems

Abstract

Notes

Acknowledgements

References

Copyright information

Authors and Affiliations

Personalised recommendations

Cite chapter