On the convergence of the gradient projection method for convex optimal control problems with bang–bang solutions
 283 Downloads
Abstract
We revisit the gradient projection method in the framework of nonlinear optimal control problems with bang–bang solutions. We obtain the strong convergence of the iterative sequence of controls and the corresponding trajectories. Moreover, we establish a convergence rate, depending on a constant appearing in the corresponding switching function and prove that this convergence rate estimate is sharp. Some numerical illustrations are reported confirming the theoretical results.
Keywords
Gradient projection method Strong convergence Convergence rate Optimal control Bang–bang controlMathematics Subject Classification
47J20 49J15 49M05 90C25 90C301 Introduction
Numerical solution methods for various optimal control problems have been investigated during the last decades [6, 8, 9, 10, 11]. However, in most of the literature, the optimal controls are assumed to be at least Lipschitz continuous. This assumption is rather strong, as whenever the control appears linearly in the problem, the lack of coercivity typically leads to discontinuities of the optimal controls. Recently, optimal control problems with bang–bang solutions attract more attention. Stability and error analysis of bang–bang controls can be found in [14, 26, 32]. Euler discretizations for linear–quadratic optimal control problems with bang–bang solutions were studied in [1, 2, 5, 29]. Higher order schemes for linear and linear–quadratic optimal control problems with bang–bang solutions were developed in [24, 27].
On the other hand, among many traditional solution methods in optimization, projectiontype methods are widely applied because of their simplicity and efficiency [13, 15, 31].
Recently, the gradient projection method has been reconsidered for solving general optimal control problems [22, 28]. Under some suitable conditions, it was proved that the control sequence converges weakly to an optimal control and the corresponding trajectory sequence converges strongly to an optimal trajectory. However, no convergence rate result has been established.
Further we assume (see the next section for precise formulations) that the data are smooth enough, that the problem (1.1)–(1.3) is convex and that for the (unique) optimal control \(u^*\) the objective function fulfills a certain growth condition. In particular we show that this condition is satisfied in the bang–bang case if each component of the associated switching function satisfies a growth condition as given in [25, 29].
Under these assumptions, we prove that the control sequence actually converges strongly to the solution. Moreover, the convergence rates for both controls and states are provided, depending on the constant appearing in the growth condition for the switching function. An example is analysed showing that the estimation for these convergence rates is sharp.
The paper is organized as follows: In Sect. 2, we specify the assumptions we use and recall some facts which will be useful in the sequel. Section 3 discusses the convergence properties of the gradient projection method. Some numerical examples of linear–quadratic type are reported in Sect. 4 illustrating the results in the previous section. Some final remarks are given in the last section.
2 Preliminaries
In this section, we will clarify the assumptions used and recall some important facts which are necessary to establish our result.
By \({\mathcal {U}}:=L^2([0,T],U)\) we denote the set of all admissible controls and if not stated otherwise \(\Vert \cdot \Vert \) denotes the \(L^2\)norm. The first two assumptions guarantee that the problem (1.1)–(1.3) is meaningful.
Assumption A1
For any given control \(u\in {\mathcal {U}}\) there is a unique solution \(x=x(u)\) of (1.2) on [0, T].
Assumption A3
The objective function J is continuously differentiable on \({\mathcal {U}}\) with Lipschitz derivative.
We denote by L the Lipschitz modulus of the gradient \(\nabla J\) of J and write \(J^*:=J(u^*)\) for its optimal value. The following result is well known (see e.g. [23, Lemma 1.30]).
Lemma 2.1
Assumptions A1–A3 are common in optimal control. For example the following two Assumptions B1–B2 imply A1–A3 (cf. [22])
Assumption B1
The functions f and h are of the form \(f(t,x,u)=f_0(x)+f_1(x)u\) and \(h(t,x,u)=h_0(x)+\langle h_1(x),u\rangle \) respectively, where \(f_0:{\mathbb {R}}^n\rightarrow {\mathbb {R}}^n, f_1:{\mathbb {R}}^n\rightarrow {\mathbb {R}}^{n\times m}, h_0:{\mathbb {R}}^n\rightarrow {\mathbb {R}}\) and \(h_1:{\mathbb {R}}^n\rightarrow {\mathbb {R}}^m\) are twice continuously differentiable.
Assumption B2
Additionally we assume the following.
Assumption A4
The objective function J is convex.
Note that if the set \({\mathcal {F}}\) of admissible pairs is convex this assumption is equivalent to the statement that the function \(\psi \) is convex on \({\mathcal {F}}\). In particular this is the case if f is affine (i.e. f is of the form \(f(t,x,u)=A(t)x+B(t)u+d(t)\)) as in [25, 29].
Further we will assume a growth condition for J that is similar to (4.7) in [3].
Assumption A5
Note that in particular A5 implies that the solution \(u^*\) is unique.
Remark 2.2
For coercive optimal control problems (in the sense of [12]) Assumptions A1–A4 are fulfilled as well as A5 for \(\theta =0\). In these problems the objective function J however is even strongly convex and therefore one can apply known results (e.g. [21, Theorem 2.1.15]) directly to show linear convergence of the gradient projection method in this case.
In the following we will show that Assumption A5 is fulfilled for bang–bang controls with no singular arcs. We recall that in the case of bang–bang controls the function \(\sigma ^*:=H_u(\cdot ,x^*,u^*,p^*)\) is called switching function corresponding to the triple \((x^*,u^*,p^*)\). For every \(j\in \{1,\ldots ,m\}\) denote by \(\sigma ^*_j\) its jth component. The following assumption says that the switching function \(\sigma ^*\) satisfies a growth condition around the switching points, which implies that \(u^*\) is strictly bang–bang.
Assumption B3
Assumption B3 plays the main role in the study of regularity, stability and error analysis of discretization techniques for optimal control problems with bang–bang solutions. Many variations of this assumption are used in the literature about bang–bang controls. To our knowledge the first assumption of this type was introduced by Felgenhauer [14] for continuously differentiable switching functions with \(\theta =1\) to study the stability of bang–bang controls. Alt et al. [1, 2, 4] used a slightly stronger version of B3 with \(\theta =1\), that additionally excludes the endpoints 0 and T as zeros of the switching function, to investigate the error bound for Euler approximation of linear–quadratic optimal control problems with bang–bang solutions. Quincampoix and Veliov [26] used a rank condition which implies B3 (including cases where \(\theta \ne 1\)) to obtain the metric regularity and stability of Mayer problems for linear systems. Seydenschwanz [29], Preininger et al. [25], Pietrus, Scarinci and Veliov [24, 27] used this assumption in the study of metric (sub)regularity, stability and error estimate for discretized schemes of linear–quadratic optimal control problems with bang–bang solutions.
To prove that B3 implies A5 we need the following lemma, which is a simplified version of [29, Lemma 1.3] (see also, [1, Lemma 4.1]).
Lemma 2.3
Proposition 2.4
Let Assumptions A1, A2 and A4 be fulfilled and let \(u^*\) be a solution of (1.1)–(1.3) such that B3 is fulfilled. Then A5 holds.
Proof
Lemma 2.5
Lemma 2.6
3 Convergence analysis
We consider the following Gradient Projection Method (GPM):

Step 0: Choose a sequence \(\{\lambda _k\}\) of positive real numbers and an initial control \(u_0\in {\mathcal {U}}\). Set \(k=0\).
 Step 1: Compute the gradient \(\nabla J(u_k)(t):=f_u(t,x_k(t),u_k(t))^\top p_k(t)+ h_u(t,x_k(t),u_k(t))^\top \) by solving the following differential equations$$\begin{aligned} \dot{x}_k(t)= & {} f(t,x_k(t),u_k(t)), \quad x_k(0)=x_0;\nonumber \\ \dot{p}_k(t)= & {} f_x(t,x_k(t),u_k(t))^\top p_k(t) h_x(t,x_k(t),u_k(t))^\top , \nonumber \\ p_k(T)= & {} \nabla g(x_k(T)). \end{aligned}$$(3.1)
 Step 2: Compute$$\begin{aligned} {u}_{k+1} = P_{\mathcal {U}}(u_k\lambda _k \nabla J(u_k)). \end{aligned}$$(3.2)

Step 3: If \(u_{k+1}=u_k\) then Stop. Otherwise replace k by \(k+1\) and go to Step 1.
The following estimate will be used repeatedly in our convergence analysis.
Proposition 3.1
Proof
We are now in the position to establish the strong convergence and the convergence rate of \(\left\{ u_k \right\} \) to a solution.
Theorem 3.2
 (i)
\(\Vert u_{k}u^*\Vert ^2 \le \eta k^{\frac{1}{\theta }},\) for all k, where \(\eta >0\) is a constant;
 (ii)
The sequence \(\{J(u_k)\}\) is monotonically decreasing. Moreover \( \sum _{k=0}^{\infty } \left( J(u_k)\right. \left. J(u^*)\right) < +\infty .\)
Proof
Now we can apply Lemma 2.5 for \(s_k= \Vert u_{k}u^*\Vert ^2, \alpha =\theta \) and \(\delta _k= 2\lambda _{\min } \beta \) to obtain the convergence rate (i) for \(\left\{ \Vert u_{k}u^*\Vert \right\} \).
Remark 3.3
From (ii) in Theorem 3.2, we can conclude that \(J(u_{k}) J(u^*)=o(\frac{1}{k})\), which significantly improves the error estimate \(J(u_{k}) J(u^*)=O(\frac{1}{k})\) in (3.3).
The following example illustrates that the estimation (i) in Theorem 3.2 cannot be improved when \(\lambda _k\) is bounded from below by a constant \(\lambda _{\min }\).
Example 3.4
Using the stronger Assumptions B1–B2 the convergence rate of the corresponding trajectories can be obtained as a corollary of Theorem 3.2 and [22, Lemma 2].
Corollary 3.5
When the Lipschitz modulus L is difficult to estimate, one can consider the nonsummable diminishing stepsizes as follow.
Theorem 3.6
 (i)
\( \Vert u_{k}u^*\Vert ^2 \le C\mu _k^{\frac{1}{\theta }}\)
 (ii)
\(J(u_k)J(u^*)=o\left( \frac{1}{\mu _k}\right) \),
Proof
Using the same example as above we can again show that the estimation (i) cannot be improved.
Example 3.7
Similar to Corollary 3.5 we obtain
Corollary 3.8
4 Numerical illustrations
The following example is taken from [27].
Example 4.1
Convergence rates for Example 4.1
N  10  20  50  100  200  500 

\(\rho _N \)  0.7701  0.9181  0.9839  0.9902  0.9964  0.9976 
The following second example is taken from [1, Example 6.1]
Example 4.2
Convergence rates for Example 4.2
N  10  20  50  100  200  500 

\(\rho _N \)  0.9625  0.9724  0.9905  0.9937  0.9943  0.9944 
In the next example, we consider a problem in which Assumption A5 is satisfied for \(\theta \not =1\) (see also [27, 29]).
Example 4.3
Convergence rates for Example 4.3
N  10  20  50  100  200  500 

\(\theta =2\)  
\(\rho _N \)  0.9418  0.9686  0.9865  0.9962  0.9953  0.9947 
\(\theta =3\)  
\(\rho _N \)  0.9245  0.9781  0.9936  0.9922  0.9968  0.9986 
5 Concluding remarks
Note that the main results in Theorems 3.2 and 3.6 use Assumption A5 which is more general than just the bang–bang case. For example Assumption A5 is also satisfied in the strongly convex case, where even better convergence results are known. Further it would be interesting to see under what assumptions our results still apply in the case of singular arcs. This is challenging due to the fact that currently there is no condition similar to the bang–bang Assumption B3 that ensures Assumption A5 and therefore remains as a topic for future research.
Notes
Acknowledgements
Open access funding provided by Austrian Science Fund (FWF). The authors thank Vladimir Veliov for introducing them to the topic and for fruitful discussions. They are also thankful to Ursula Felgenhauer and the two anonymous referees for constructive comments which helped improving the presentation of the paper significantly.
References
 1.Alt, W., Baier, R., Gerdts, M., Lempio, F.: Error bounds for Euler approximation of linear–quadratic control problems with bang–bang solutions. Numer. Algebra Control Optim. 2(3), 547–570 (2012)MathSciNetCrossRefMATHGoogle Scholar
 2.Alt, W., Baier, R., Gerdts, M., Lempio, F.: Approximations of linear control problems with bang–bang solutions. Optimization 62(1), 9–32 (2013)MathSciNetCrossRefMATHGoogle Scholar
 3.Alt, W., Felgenhauer, U., Seydenschwanz, M.: Euler discretization for a class of nonlinear optimal control problems with control appearing linearly. Comput. Optim. Appl. (2017). https://doi.org/10.1007/s1058901799697
 4.Alt, W., Schneider, C., Seydenschwanz, M.: An implicit discretization scheme for linear–quadratic control problems with bangbang solutions. Optim. Methods Softw. 29(3), 535–560 (2014)MathSciNetCrossRefMATHGoogle Scholar
 5.Alt, W., Schneider, C., Seydenschwanz, M.: Regularization and implicit Euler discretization of linear–quadratic optimal control problems with bang–bang solutions. Appl. Math. Comput. 287–288, 104–124 (2016)MathSciNetGoogle Scholar
 6.Bonnans, J.F., Festa, A.: Error estimates for the Euler discretization of an optimal control problem with firstorder state constraints. SIAM J. Numer. Anal. 55(2), 445–471 (2017)MathSciNetCrossRefMATHGoogle Scholar
 7.Dong, Y.: Comments on the proximal point algorithm revisited. J. Optim. Theory Appl. 166(1), 343–349 (2015)MathSciNetCrossRefMATHGoogle Scholar
 8.Dontchev, A.L.: An a priori estimate for discrete approximations in nonlinear optimal control. SIAM J. Control Optim. 34(4), 1315–1328 (1996)MathSciNetCrossRefMATHGoogle Scholar
 9.Dontchev, A.L., Hager, W.W.: Lipschitzian stability in nonlinear control and optimization. SIAM J. Control Optim. 31(3), 569–603 (1993)MathSciNetCrossRefMATHGoogle Scholar
 10.Dontchev, A.L., Hager, W.W., Malanowski, K.: Error bounds for Euler approximation of a state and control constrained optimal control problem. Numer. Funct. Anal. Optim. 21(5–6), 653–682 (2000)MathSciNetCrossRefMATHGoogle Scholar
 11.Dontchev, A.L., Hager, W.W., Veliov, V.M.: Secondorder Runge–Kutta approximations in control constrained optimal control. SIAM J. Numer. Anal. 38(1), 202–226 (2000)MathSciNetCrossRefMATHGoogle Scholar
 12.Dontchev, A.L., Hager, W.W., Veliov, V.M.: Uniform convergence and mesh independence of Newton’s method for discretized variational problems. SIAM J. Control Optim. 39(3), 961–980 (2000)MathSciNetCrossRefMATHGoogle Scholar
 13.Dunn, J.C.: Global and asymptotic convergence rate estimates for a class of projected gradient processes. SIAM J. Control Optim. 19(3), 368–400 (1981)MathSciNetCrossRefMATHGoogle Scholar
 14.Felgenhauer, U.: On stability of bang–bang type controls. SIAM J. Control Optim. 41(6), 1843–1867 (2003)MathSciNetCrossRefMATHGoogle Scholar
 15.Kelley, C.T., Sachs, E.W.: Mesh independence of the gradient projection method for optimal control problems. SIAM J. Control Optim. 30(2), 477–493 (1991)MathSciNetCrossRefMATHGoogle Scholar
 16.Khoroshilova, E.V.: Extragradienttype method for optimal control problem with linear constraints and convex objective function. Optim. Lett. 7, 1193–1214 (2012)MathSciNetCrossRefMATHGoogle Scholar
 17.Kinderlehrer, D., Stampacchia, G.: An Introduction to Variational Inequalities and Their Applications. Academic Press, New York (1980)MATHGoogle Scholar
 18.Li, G., Mordukhovich, B.S.: Hölder metric subregularity with applications to proximal point method. SIAM J. Optim. 22(4), 1655–1684 (2012)MathSciNetCrossRefMATHGoogle Scholar
 19.Luo, Z.Q., Tseng, P.: Error bounds and convergence analysis of feasible descent methods: a general approach. Ann. Oper. Res. 46(1), 157–178 (1993)MathSciNetCrossRefMATHGoogle Scholar
 20.Luo, Z.Q., Tseng, P.: Error bound and convergence analysis of matrix splitting algorithms for the affine variational inequality problem. SIAM J. Optim. 2(1), 43–54 (1992)MathSciNetCrossRefMATHGoogle Scholar
 21.Nesterov, Y.: Introductory Lectures on Convex Optimization. Springer, Berlin (2004)CrossRefMATHGoogle Scholar
 22.Nikol’skii, M.S.: Convergence of the gradient projection method in optimal control problems. Comput. Math. Model. 18(2), 148–156 (2007)MathSciNetCrossRefMATHGoogle Scholar
 23.Peypouquet, J.: Convex Optimization in Normed Spaces: Theory, Methods and Examples. Springer, Berlin (2015)CrossRefMATHGoogle Scholar
 24.Pietrus, A., Scarinci, T., Veliov, V.M.: High order discrete approximations to Mayer’s problems for linear systems. SIAM J. Control Optim. 56(1), 102–119 (2018)MathSciNetCrossRefMATHGoogle Scholar
 25.Preininger, J., Scarinci, T., Veliov, V.M.: Metric regularity properties in bang–bang type linear–quadratic optimal control problems. In: Research Report 201707, ORCOS, TU Wien http://orcos.tuwien.ac.at/fileadmin/t/orcos/Research_Reports/201707.pdf (2017)
 26.Quincampoix, M., Veliov, V.M.: Metric regularity and stability of optimal control problems for linear systems. SIAM J. Control Optim. 51(5), 4118–4137 (2013)MathSciNetCrossRefMATHGoogle Scholar
 27.Scarinci, T., Veliov, V.M.: Higherorder numerical schemes for linear quadratic problems with bangbang controls. Comput. Optim. Appl. (2017). https://doi.org/10.1007/s105890179948z
 28.Scheiber, E.: On the gradient method applied to optimal control problem. Bull. Transilv. Univ. Brasov 7(56), 139–148 (2014)MathSciNetGoogle Scholar
 29.Seydenschwanz, M.: Convergence results for the discrete regularization of linear–quadratic control problems with bang–bang solutions. Comput. Optim. Appl. 61(3), 731–760 (2015)MathSciNetCrossRefMATHGoogle Scholar
 30.Tuan, H.N.: Linear convergence of a type of iterative sequences in nonconvex quadratic programming. J. Math. Anal. Appl. 423(2), 1311–1319 (2015)MathSciNetCrossRefMATHGoogle Scholar
 31.Vasil’ev, F.P.: Optimization Methods. Factorial Press, Moscow (2002). (in Russian)Google Scholar
 32.Veliov, V.M.: Error analysis of discrete approximation to bang–bang optimal control problems: the linear case. Control Cybern. 34(3), 967–982 (2005)MathSciNetMATHGoogle Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.