Control Strategies for the Dynamics of Large Particle Systems

Abstract

We survey some recent approaches to control problems for large particle systems. Particle systems are transversal to many applications, ranging from classical physics to social sciences. The temporal evolution of the particles is determined by deterministic or stochastic dynamics and they are additionally able to optimize their trajectory over a large time. In particular, we investigate the limit of infinitely many particles leading to control of kinetic partial differential equations. To this goal a different notion of differentiability of the meanfield equation is introduced. Different mathematical methods based on meanfield games, model predictive control, and optimal control techniques will be discussed.

Appendix: Notion of Differentiability and Calculus for Consistent Derivation

This section is devoted to more details on the notion of differentiability. In order to keep the presentation simple on a technical level we consider the following particle system. Consider N particles i with state space $x_i\in \mathbb {R}$, a sufficiently smooth function $\phi , \psi :\mathbb {R} \to \mathbb {R}$ and assume each particle fulfills

$$\displaystyle \begin{aligned} x^{\prime}_i(t) = \frac{1}N \sum_{j=1}^N \phi(x_j(t)), x_i(0)=u_{i}. \end{aligned} $$

(27)

Here, u_i is the unknown control applied as initial datum and we assume that the previous dynamics is solved on a time interval t ∈ (0, T). The control $U=(u_i)_{i=1}^N \in \mathbb {R}^N$ is chosen to minimize the cost $J_N:\mathbb {R}^N \to \mathbb {R}$

$$\displaystyle \begin{aligned} J_N(U)=\int_0^T \frac{1}{N}\sum_{i=1}^N \psi(x_i(t)) dt, \end{aligned} $$

(28)

where x_i are the solution to (27). Standard theory like Pontryagin’s maximum principle can be applied to solve the minimization problem

$$\displaystyle \begin{aligned} U^*_N = \mbox{ argmin }_{U\in\mathbb{R}^N} J_N(U). \end{aligned} $$

(29)

We are interested in the relation of the first-order optimality conditions to (29) in the case N →∞. We introduce for each fixed t the probability density $\mu (t,\cdot )\in \mathcal {P}(\mathbb {R})$, i.e., μ(t, ⋅) ≥ 0 and $\int _{\mathbb {R}} \mu (t,x)dx = 1.$ The probability density describes the probability to have particles at time t with property x. In the meanfield limit the dynamics of μ is obtained by (27) as

$$\displaystyle \begin{aligned} \partial_t \mu(t,x) + \partial_x \int \phi(y) \mu(t,y) \mu(t,x) dy = 0, \quad \mu(0,x) = v(x), \end{aligned} $$

(30)

where $\int v(x) dx=1$ is the corresponding control. We introduce the empirical measure on $\mathbb {R}$ as $\nu _{X}(x)=\frac {1}N\sum _i \delta (x-x_i)$ with $X=(x_i)_{i=1}^N.$ Then, for $\mu (t,x)=\nu _{X(t)}= \frac {1}N\sum _i \delta (x-x_i(t))$ and v = ν_U we recover (27) from the weak form of Eq. (30) and vice versa. The meanfield limit of the sequence (J_N)_N of cost functional for N →∞ is obtained as $\mathcal {J}:\mathcal {P}(\mathbb {R})\to \mathbb {R}$, where

$$\displaystyle \begin{aligned} \mathcal{J}(v)=\int_0^T \int \psi(x) \mu(t,x) dx dt, \end{aligned} $$

(31)

and where μ solves Eq. (30). Hence, on the level of the meanfield limit we may formulate the problem

$$\displaystyle \begin{aligned} v^* = \mbox{ argmin }_{v \in\mathcal{P}(\mathbb{R})} \mathcal{J}(v). \end{aligned} $$

(32)

The latter is an optimal control problem with differential equations as constraints and may be solved using techniques from infinite-dimensional optimal control theory. The main result of this section is to establish the relation between the meanfield limit in the Pontryagin’s maximum principle and the formal adjoint calculus applied to the optimal control problem (32).

Remark 2

A simple application of formal Lagrange calculus to problem (32) yields as the adjoint equation

$$\displaystyle \begin{aligned} - \partial_t \lambda - \partial_x \lambda \left( \int \phi \mu dy \right) - \int \mu \partial_x \lambda dy \phi = \psi(x) , \qquad \lambda(T,0)=0. \end{aligned} $$

(33)

From the previous equation we observe that λ = ν_L with $L=(\ell _i)_{i=1}^N$ is not a solution since $\int \lambda dx$ is not conserved. Further, from Pontryagin’s maximum principle we expect the N adjoints ℓ_i(t) to fulfill

$$\displaystyle \begin{aligned} - \ell^{\prime}_i(t) = \psi'(x_i(t)) + \frac{1}N \sum_{j=1}^N \ell_j(t) \phi'(x_i(t)), \qquad \ell_i(T)=0, \end{aligned} $$

(34)

involving the derivatives of ψ and ϕ that are not present in (33).

The problem with the formal Lagrange calculus is the fact that Eq. (30) is posed on the space $\mathcal {P}(\mathbb {R})$ that is not a vector space. Therefore, we consider geometric derivatives also used, e.g., in [8]. As preliminary discussion note that if

$$\displaystyle \begin{aligned}\frac{d}{ds} y_i(s) = c,\qquad y_i(0)=x_{i,0}\end{aligned}$$

holds for i = 1 : N, then the empirical measure fulfills in weak form

$$\displaystyle \begin{aligned}\partial_s \nu_{Y(s)}(z) + \partial_z c, \qquad \nu_{Y(s)}(z) = 0\end{aligned}$$

for initial condition

$$\displaystyle \begin{aligned}\nu_{Y(0)}(z) = \nu_{X_{0}}.\end{aligned}$$

Clearly, ν is a probability measure. It also serves as first-order variation of the points x_i,0, i = 1 : N, in the direction of the given velocity field c in the following sense: We define the derivative of $\mathcal {J}$ at measure $\mu =\nu _{X_{0}}$ in direction c by considering the measure associated with moving all points X₀ with velocity c. This measure is $\eta (t,\cdot ):=\nu _{Y(t)} \in \mathcal {P}(\mathbb {R}).$ The derivative of $\mathcal {J}$ is then infinitesimal difference at t = 0 between the limit

$$\displaystyle \begin{aligned}\frac{d\mathcal{J}}{d\mu}(\mu) c := \lim\limits_{t\to 0} \frac{1}t \left( \mathcal{J}(\eta(t,\cdot))- \mathcal{J}(\eta(0,\cdot)) \right),\end{aligned}$$

where ∂_tη + ∂_x(cη) = 0, η(0) = μ. For the example $\mathcal {J}(\mu ) = \int \psi (x) \mu (x) dx$ we obtain

$$\displaystyle \begin{aligned} \frac{dJ}{d\mu}(\mu) c = \lim_{t\to 0} \frac{ J(\eta(t,\cdot))-J(\mu)}t = \\ \lim_{t\to 0} \frac{1}t \int \psi( \eta(t,x) - \eta(0,x) ) dx = \int \psi \eta_t(t,x) dx = \\ - \int \psi ( \eta(t,x) c )_x dx = \int \psi'(x) c(x) \mu(x) dx. \end{aligned} $$

A similar formalism is applied to the weak form of Eq. (30). We write for a function $\rho \in C^2(\mathbb {R})$ and probability measures μ(t, x) and μ(t, y) the weak form as operator A as

$$\displaystyle \begin{aligned}<A \mu, \rho>:= \int_0^T \int \rho_t + \rho_x \int \phi d\mu(t,y) d\mu(t,x) dt.\end{aligned}$$

Applying a similar calculus we obtain

$$\displaystyle \begin{aligned} \frac{d}{d\mu} <A \mu, \rho>c &= \int_0^T \int \rho_{tx} c(x) d\mu(t,x) dt \\ &\quad + \int_0^T \int \int \rho_{x} \phi'(y) c(y) d\mu(t,y)d\mu(t,x) dt\\ &\quad + \int_0^T \int \int \rho_{xx} \phi(y) c(x)d\mu(t,y)d\mu(t,x) dt. \end{aligned} $$

In order to derive first-order optimality conditions for the problem (32) we consider the Lagrangian $\mathcal {L}:\mathcal {P}(\mathbb {R}) \times \mathcal {P}(\mathbb {R}) \times (\mathcal {P}(\mathbb {R}))' \to \mathbb {R}$ given by

$$\displaystyle \begin{aligned}\mathcal{L}(v,\mu,\rho ) = \int_0^T \int \psi(x) d\mu(t,x) dt + <A\mu,\rho>+ \int \rho(0,x) dv(x).\end{aligned}$$

Then, the formal optimality conditions are obtained as saddle point of the Lagrangian $\mathcal {L}$ for everey field c in weak form and are given by

$$\displaystyle \begin{aligned} &\int_0^T \int \rho_t + \rho_x \int \phi d\mu(t,y) d\mu(t,x) dt + \int \rho dv(x) =0, {} \end{aligned} $$

(35a)

$$\displaystyle \begin{aligned} &\int_0^T \int \rho_{tx} c(x) d\mu(t,x) dt + \int_0^T \int \int \rho_{x} \phi'(y) c(y) d\mu(t,y) d\mu(t,x) dt{} \end{aligned} $$

(35b)

$$\displaystyle \begin{aligned} &\quad +\int_0^T \int \int \rho_{xx} \phi(y) c(x)d\mu(t,y) d\mu(t,x) dt + \int_0^T \int \psi'(x) c(x) d\mu(t,x) dt =0, \\ &\int \rho_x(0,x)c(x) d v(x) = 0.{} \end{aligned} $$

(35c)

Finally, we show that Eqs. (34) and (27) are recovered from (35) using the empirical measure. Fix N and $U=(u_i)_{i=1}^N.$ Let $v(x):=\frac {1}N\sum _{i=1}^N \delta (x-u_i).$ Then, a weak solution to (30) is given by $\mu (t,x):=\frac {1}N\sum _i \delta (x-x_i(t))$ provided that x_i fulfills (27). Further, we denote by $c_i(t) = \int c(x) \delta (x-x_i(t)) dx$ for any c to get

$$\displaystyle \begin{aligned} &\frac{1}N \int_0^T \sum_i c_i(t) \left( \psi'(x_i) + \rho_{tx}(t,x_i) +\rho_{xx}(t,x_i) \left( \frac{1}N\sum_j \phi(x_j) \right) \right.\\ &\left. \qquad + \phi'(x_i) \left( \frac{1}N\sum_j \rho_x(t,x_j) \right) \right) dt = 0. \end{aligned} $$

Since the previous equation has to hold true for any function c this yields

$$\displaystyle \begin{aligned} \psi'(x_i) + \rho_{tx}(t,x_i) +\rho_{xx}(t,x_i) \left ( \frac{1}N\sum\phi(x_j) \right) + \phi'(x_i) \left( \frac{1}N\sum\rho_x(t,x_j) \right) = 0. \end{aligned} $$

Define now

$$\displaystyle \begin{aligned}\lambda_i(t):= \rho_x(t,x_i(t)), i=1,\dots,N.\end{aligned}$$

Then,

$$\displaystyle \begin{aligned}\lambda_i^{\prime}(t) = \rho_{tx}(t,x_i) + \rho_{xx}(t,x_i)x_i^{\prime}(t)\end{aligned}$$

and therefore

$$\displaystyle \begin{aligned}\psi'(x_i) + \lambda^{\prime}_i(t) + \rho_{xx}(t,x_i)\left( - x_i^{\prime} + \frac{1}N\sum_j\phi(x_j) \right) + \phi'(x_i) \frac{1}N \sum \lambda_j =0.\end{aligned}$$

Since $x_i^{\prime }$ fulfills (27) we obtain that λ_i ≡ ℓ_i for all i and fulfills (34). Summarizing, the first-order optimality conditions for the meanfield problem (32) coincide with the first-order optimality conditions for (29) provided the previous notion of differentiability in $\mathcal {P}(\mathbb {R})$ is used. The following relation holds true for the corresponding Lagrange multipliers:

$$\displaystyle \begin{aligned}\ell_i(t) := \partial_x \rho(t,x_i(t)).\end{aligned}$$