Linear turnpike theorem

Abstract

The turnpike phenomenon stipulates that the solution of an optimal control problem in large time remains essentially close to a steady-state of the dynamics, itself being the optimal solution of an associated static optimal control problem. Under general assumptions, it is known that not only the optimal state and the optimal control, but also the adjoint state coming from the application of the Pontryagin maximum principle, are exponentially close to that optimal steady-state, except at the beginning and at the end of the time frame. In such a result, the turnpike set is a singleton, which is a steady-state. In this paper, we establish a turnpike result for finite-dimensional optimal control problems in which some of the coordinates evolve in a monotone way, and some others are partial steady-states of the dynamics. We prove that the discrepancy between the optimal trajectory and the turnpike set is linear, but not exponential: we thus speak of a linear turnpike theorem.

A Appendix

This appendix is devoted to illustrating the comments done in Remark 2. We take a very simple example and we give numerical simulations showing the competition between two global turnpikes, or, at the level of the initialization of numerical methods, between local and global turnpikes.

We consider the one-dimensional optimal control problem

$$\begin{aligned} \begin{aligned}&\dot{x} = -3x + 3x^3 + u, \qquad x(0)=x_0,\quad x(T)=x_f\text { or free} \\&\min \int _0^T \left( (x(t)-x_d)^2 + (u(t)-u_d)^2 \right) dt \end{aligned} \end{aligned}$$

(89)

The corresponding static problem is

$$\begin{aligned} \min _{(x,u)\ \mid \ -3x + 3x^3 + u=0} \left( (x-x_d)^2 + (u-u_d)^2\right) . \end{aligned}$$

(90)

1.1 A.1 Competition between two (global) turnpikes

We take $x_d=1$ and $u_d=3.47197$. The choice of $u_d$ is done so that the static problem (90) has two global minima, at ${\bar{x}}_1=-1.347372066$ and ${\bar{x}}_2=0.5939615956$, see Fig. 4.

Fig. 4

Plot of the function $x\mapsto (x-1)^2 + (3x - 3x^3-3.47197)^2$

In Fig. 5, in dashed blue, we have computed the optimal trajectory with $x_0=-5$, $x_f=-1$, $T=10$: we observe a turnpike phenomenon around ${\bar{x}}_1$. In solid red, we have computed the optimal trajectory with $x_0=2$, $x_f=1$, $T=10$: we observe a turnpike phenomenon around ${\bar{x}}_2$.

Fig. 5

Global turnpikes around ${\bar{x}}_1$ and ${\bar{x}}_2$

The fact that the turnpike phenomenon is either around ${\bar{x}}_1$ or around ${\bar{x}}_2$ depends on the terminal conditions. For instance, if $x_0$ and $x_f$ are close to ${\bar{x}}_1$ (resp., ${\bar{x}}_2$) then the optimal trajectory will make a turnpike around ${\bar{x}}_1$ (resp., ${\bar{x}}_2$). But when the terminal conditions are farther, it is not clear to predict the behavior of the optimal trajectory.

Since the minima are global, we expect to observe a competition between both turnpikes, depending on the terminal conditions (see [37]). Let us provide some numerical simulations illustrating this competition. To facilitate the understanding, we consider the problem with $x_f$ free. It is interesting to note that the numerical result strongly depends on the initialization of the numerical method. Here, to compute numerically the optimal solutions of (89), we use AMPL (see [25]) combined with IpOpt (see [46]): the trajectory and the controls are discretized (the control is piecewise constant and the trajectory is piecewise linear, on a given subdivision that is chosen fine enough), and we initialize the trajectory with the same constant value over all the subdivision. Then according to the value of this initialization, we can make emerge such or such turnpike, and all solutions are anyway optimal.

Fig. 6

$x_0=-2$, $x_f$ free, $T=20$

In Fig. 6, we have taken $T=20$, $x_0=-2$ (and $x_f$ is let free). In dashed blue, we have initialized the trajectory to the constant trajectory ${\bar{x}}_1$, and we then obtain an optimal trajectory which stays essentially near ${\bar{x}}_1=-1.347372066$, with a kind of “hesitation” towards ${\bar{x}}_2=0.5939615956$ near the end. In solid red, we have initialized the trajectory to the constant trajectory ${\bar{x}}_2$, and we obtain a trajectory staying essentially near ${\bar{x}}_2$, with a kind of “hesitation” towards ${\bar{x}}_1$ near the beginning. We stress that the two solutions are optimal: both have a cost $C\simeq 6.2822$. We could make emerge other similar trajectories, which “hesitate” between the two turnpikes. All of them are optimal, or, at least, “quasi-optimal” (there is a small error due to switches from one turnpike to the other).

1.2 A.3 Local versus global turnpike

We take $x_d=1$ and $u_d=1$. The choice of $u_d$ is now such that the static problem (90) has a unique global solution ${\bar{x}} = 0.781538640850898$, see Fig. 7. But it has also a local solution ${\bar{x}}_{loc}=-1.10551208794920$.

Fig. 7

Plot of the function $x\mapsto (x-1)^2 + (3x - 3x^3-1)^2$

The global minimum ${\bar{x}}$ is a global turnpike, while the local minimum ${\bar{x}}$ is a local turnpike. In the numerical simulations, when performing either an optimization or a Newton method (shooting), we compute local solutions. Hence, we must expect that the numerical results depend on the initialization. To check global optimality, we have to compare the costs.

Fig. 8

$x_0=-2$, $x_f=-1$, $T=10$

In Fig. 8, in dashed blue, we have initialized the code with the constant trajectory ${\bar{x}}_{loc}$. We observe a turnpike around ${\bar{x}}_{loc}=-1.10551208794920$. The cost is $C\simeq 7.008$. But this is a local turnpike only. This trajectory that we obtain is only locally optimal, and is not globally optimal. In solid red, we have initialized the code with the constant trajectory ${\bar{x}}$. We observe a turnpike around ${\bar{x}} = 0.781538640850898$. The cost is $C\simeq 3.825$ and is lower than the one of the previous one. Here, we have actually computed the globally optimal trajectory. This is the global turnpike.

It is also interesting to see what happens if we take T much smaller. Let us take $T=2$. In Fig. 9, in dashed blue, we have initialized the code with the constant trajectory ${\bar{x}}$. We observe a trend to the turnpike around ${\bar{x}} = 0.781538640850898$. The cost is $C\simeq 17.792$. But this trajectory, now, is not globally optimal. In solid red, we have initialized the code with the constant trajectory ${\bar{x}}_{loc}$. We observe a turnpike around ${\bar{x}}_{loc}=-1.347372066$. The cost is $C\simeq 16.322$. It is the globally optimal trajectory.

Fig. 9

$x_0=-2$, $x_f=-1$, $T=2$

We can search a time $2<T_c<10$ for which both previous initializations give equivalent turnpikes around ${\bar{x}}_{loc}$ and ${\bar{x}}$ (with same cost). This is done hereafter. What is important is that, in large time T, we have indeed the global turnpike around ${\bar{x}}$.

In Fig. 10, at the left, the code is initialized with $x(t)\equiv -1.1$, in order to promote the turnpike around $\bar{x}_{loc}=-1.347372066$. At the right, the code is initialized with $x(t)\equiv 0.78$, in order to promote the turnpike around ${\bar{x}} = 0.781538640850898$. We have computed trajectories for the following successives values of T: 2.5, 2.7, 2.9, 3.1, 3.3. We observe that the blue and cyan trajectories at the top are optimal (see the value of their cost); and that the red, green and black trajectories at the bottom are optimal. The bifurcation occurs around $T=2.9$.

Finally, in Fig. 11, we represent the global optimal trajectory. For $T\lesssim 2.9$, the global optimal trajectory makes a turnpike around ${\bar{x}}_{loc}=-1.347372066$, which is a local minimizer of the optimal static problem. For $T\geqslant 2.9$ we have a bifurcation and the global optimal trajectory makes a turnpike around ${\bar{x}} = 0.781538640850898$, which is the global minimizer of the optimal static problem.

Fig. 10

$x_0=-2$, $x_f=-1$

Fig. 11

$x_0=-2$, $x_f=-1$