Advertisement

Applied Mathematics & Optimization

, Volume 78, Issue 1, pp 145–183 | Cite as

Stochastic Linear Quadratic Optimal Control Problems in Infinite Horizon

  • Jingrui Sun
  • Jiongmin Yong
Article

Abstract

This paper is concerned with stochastic linear quadratic (LQ, for short) optimal control problems in an infinite horizon with constant coefficients. It is proved that the non-emptiness of the admissible control set for all initial state is equivaleznt to the \(L^{2}\)-stabilizability of the control system, which in turn is equivalent to the existence of a positive solution to an algebraic Riccati equation (ARE, for short). Different from the finite horizon case, it is shown that both the open-loop and closed-loop solvabilities of the LQ problem are equivalent to the existence of a static stabilizing solution to the associated generalized ARE. Moreover, any open-loop optimal control admits a closed-loop representation. Finally, the one-dimensional case is worked out completely to illustrate the developed theory.

Keywords

Stochastic linear quadratic optimal control Stabilizability Open-loop solvability Closed-loop solvability Algebraic Riccati equation Static stabilizing solution Closed-loop representation 

AMS Subject Classifications

49N10 49N35 93D15 93E20 

1 Introduction

Let \((\Omega ,\mathcal{F},{\mathbb {F}},\mathbb {P})\) be a complete filtered probability space on which a standard one-dimensional Brownian motion \(W=\{W(t); 0\le t<\infty \}\) is defined, where \({\mathbb {F}}=\{\mathcal{F}_t\}_{t\ge 0}\) is the natural filtration of W augmented by all the \(\mathbb {P}\)-null sets in \(\mathcal{F}\). For a Euclidean space \(\mathbb {H}\), let \(L_{\mathbb {F}}^2(\mathbb {H})\) denote the space of \({\mathbb {F}}\)-progressively measurable processes \(\varphi :[0,\infty )\times \Omega \rightarrow \mathbb {H}\) with \(\mathbb {E}\int _0^\infty |\varphi (t)|^2dt<\infty \).

Consider the following controlled linear stochastic differential equation (SDE, for short) on the infinite time horizon \([0,\infty )\):
$$\begin{aligned} \left\{ \begin{array}{ll} dX(t)=\left[ AX(t)+Bu(t)+b(t)\right] dt+\left[ CX(t)+Du(t)+\sigma (t)\right] dW(t),\qquad t\ge 0, \\ X(0)= x,\end{array}\right. \end{aligned}$$
(1.1)
with quadratic cost functional
$$\begin{aligned} J(x;u(\cdot ))\triangleq \mathbb {E}\int _0^\infty \left[ \left\langle \begin{pmatrix}Q&{}\quad S^\top \\ S&{}\quad R\end{pmatrix} \begin{pmatrix}X(t)\\ u(t)\end{pmatrix}, \begin{pmatrix}X(t)\\ u(t)\end{pmatrix}\right\rangle +2\left\langle \begin{pmatrix}q(t)\\ \rho (t)\end{pmatrix}, \begin{pmatrix}X(t)\\ u(t) \end{pmatrix}\right\rangle \right] dt. \end{aligned}$$
(1.2)
Here and throughout the paper, \(A,C,Q\in \mathbb {R}^{n\times n}\), \(B,D,S^\top \in \mathbb {R}^{n\times m}\), and \(R\in \mathbb {R}^{m\times m}\) are given constant matrices, with Q and R being symmetric; the superscript \(\top \) denotes the transpose of matrices and vectors; and \(b(\cdot ), \sigma (\cdot ), q(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^n)\), \(\rho (\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m)\). In (1.1), \(X(\cdot )\), valued in \(\mathbb {R}^n\), is called the state process with initial state \(x\in \mathbb {R}^n\), and \(u(\cdot )\), which belongs to \(L_{\mathbb {F}}^2(\mathbb {R}^m)\), is called the control process. Note that for \((x,u(\cdot ))\in \mathbb {R}^n\times L_{\mathbb {F}}^2(\mathbb {R}^m)\), the solution \(X(\cdot )\equiv X(\cdot \,;x,u(\cdot ))\) of (1.1) might merely be locally square-integrable, i.e.,
$$\begin{aligned} \mathbb {E}\int _0^T|X(t)|^2dt<\infty ,\quad \forall \, 0<T<\infty ;\quad \mathbb {E}\int _0^\infty |X(t)|^2dt=\infty , \end{aligned}$$
and the above cost functional \(J(x; u(\cdot ))\) might not be defined. Therefore, we introduce the following:
$$\begin{aligned} \mathcal{U}_{ad}(x)\triangleq \left\{ u(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m)~\left| ~\mathbb {E}\int _0^\infty |X(t;x,u(\cdot ))|^2dt<\infty \right. \right\} , \quad x\in \mathbb {R}^n. \end{aligned}$$
Any element \(u(\cdot )\in \mathcal{U}_{ad}(x)\) is called an admissible control associated with x. Our linear quadratic (LQ, for short) optimal control problem can now be stated as follows.
Problem (LQ). For any given \(x\in \mathbb {R}^n\), find a \(u^*(\cdot )\in \mathcal{U}_{ad}(x)\) such that
$$\begin{aligned} J(x;u^*(\cdot ))=\inf _{u(\cdot )\in \mathcal{U}_{ad}(x)}J(x;u(\cdot ))\triangleq V(x). \end{aligned}$$
(1.3)
Any \(u^*(\cdot )\in \mathcal{U}_{ad}(x)\) satisfying (1.3) is called an open-loop optimal control of Problem (LQ) for the initial state x; the corresponding \(X^*(\cdot )\equiv X(\cdot \,;x,u^*(\cdot ))\) is called an optimal state process; and the function \(V(\cdot )\) is called the value function of Problem (LQ). In the special case that \(b(\cdot ), \sigma (\cdot ), q(\cdot ), \rho (\cdot )=0\), we denote the corresponding LQ problem, cost functional and value function by Problem (LQ)\(^0\), \(J^0(x;u(\cdot ))\) and \(V^0(x)\), respectively.

The study of LQ problems (both in finite and infinite horizons, deterministic and stochastic) has a long history that can be traced back to the pioneering works of Kalman [12] and Wonham [21] (see also [4, 5, 9, 25] and the references therein). In the literature, take the LQ problem in a finite horizon as an example, it is typically assumed that the cost functional has a (uniformly) positive definite weighting matrix for the control term and positive semidefinite weighting matrices for the state terms. In such a case, the cost functional is convex and coercive in the control variable, and thus the LQ problem (in a finite horizon) has a unique open-loop optimal control which further admits a state feedback representation via the solution to a differential Riccati equation. The problem in infinite horizon is similar, with some additional stabilizability conditions.

In 1983, while studying LQ problems on Hilbert space, You [26] found that the weighting matrices of state in the cost functional do not have to be positive semidefinite. In 1998, Chen–Li–Zhou [6] further found that for stochastic LQ problems, even the control weighting matrix does not have to be positve semidefinite. Since then, extensive research efforts have been devoted to the indefinite LQ problems and to the solvability of indefinite Riccati equations (see, for examples, [1, 7, 8, 10, 14, 16]).

Recently, Sun–Li–Yong [18] found that an indefinite LQ problem with finite horizon might admit an open-loop optimal control (which could even be continuous) for any initial pair, while the Riccati equation is not solvable. This leads to the introduction of the open-loop and closed-loop solvabilities which are essentially different. It was shown in [18] that the closed-loop solvability is actually equivalent to the existence of a regular solution to the Riccati equation (such a fact was firstly revealed by Sun–Yong [19] in 2014 for two-person zero-sum differential games), and that the strongly regular solvability of the Riccati equation is equivalent to the uniform convexity of the cost functional. See Yong [24], Sun [17], and Li–Sun–Yong [13] for relevant works of LQ problems involving mean-field.

Stochastic LQ problems in an infinite horizon and the associated algebraic Riccati equations (AREs, for short) were treated in [2, 3] via the linear matrix inequality and semidefinite programming techniques. This approach has been further developed extensively in [23] by virtue of a duality analysis of semidefinite programming. Along another line, Wu–Zhou [22] introduced a new frequency characteristic and applied the classical frequency domain approach to the LQ problems. Recently, based on the work of Yong [24], Huang–Li–Yong [11] studied a mean-field LQ optimal control problem on \([0,\infty )\).

In the research on LQ problems with infinite horizon, however, there are some important issues that have not been addressed:
  1. (a)

    The precise relation between the solvability of the problem and that of the ARE. In [3], it was shown that the solvability of an LQ problem is equivalent to the existence of a stabilizing solution to the ARE, but it is hard to verify whether a solution is stabilizing since certain selection of additional processes is involved.

     
  2. (b)

    The closed-loop solvability. Because closed-loop controls are independent of the initial state and the future information, it is more meaningful and convenient to use closed-loop controls rather than open-loop controls. On the other hand, closed-loop solvability trivially implies open-loop solvability.

     
  3. (c)

    The structure of admissible control sets. Unlike the finite horizon case, the structure of \(\mathcal{U}_{ad}(x)\) seems to be very complicated since it involves the state equation. In general, \(\mathcal{U}_{ad}(x)\) depends on x, and may even be empty for some x. Figuring out the structure of admissible control sets will give us a better understanding of the LQ problem.

     
This paper is to address the above issues. An interesting fact we find is that for infinite-horizon LQ problems, open-loop and closed-loop solvabilities coincide. Such a fact, as we mentioned earlier, does not hold in the finite horizon case. We shall show that the solvability of the problem can be characterized by the existence of a static stabilizing solution to the ARE, and that every open-loop optimal control admits a closed-loop representation. It is shown that the \(L^2\)-stabilizability is not only sufficient, but also necessary, for the non-emptiness of all admissible control sets. Moreover, we show that the \(L^2\)-stabilizability can be verified by solving an ARE.

The rest of this paper is organized as follows. In Sect. 2, we present some preliminary results that shall be needed later. Section 3 aims to describe the structure of admissible control sets. In Sect. 4, we introduce the notions of open-loop and closed-loop solvabilities as well as the algebraic Riccati equation, and state the main result of the paper. Section 5 is devoted to a special case of Problem (LQ), where system (1.1) is \(L^2\)-stable. In Sect. 6 we prove the main result for the general case. Finally, to illustrate the results obtained, we completely solve the one-dimensional case in Sect. 7.

2 Preliminaries

2.1 Notation

Let us first introduce/recall the following notation that will be used below:
$$\begin{aligned}&\mathbb {R}^{n\times m} \, \text{ is } \text{ the } \text{ Euclidean } \text{ space } \text{ of } \text{ all }\, n\times m \, \text{ real } \text{ matrices; } \,\mathbb {R}^n=\mathbb {R}^{n\times 1} \,\text{ and } \,\mathbb {R}=\mathbb {R}^1.\\&\mathbb {S}^n: \,\text{ the } \text{ space } \text{ of } \text{ all } \text{ symmetric }\, n\times n \,\text{ real } \text{ matrices }.\\&\mathbb {S}^n_+: \text{ the } \text{ subset } \text{ of }\, \mathbb {S}^n \, \text{ which } \text{ consists } \text{ of } \text{ positive } \text{ definite } \text{ matrices }.\\&\overline{\mathbb {S}^n_+}: \text{ the } \text{ subset } \text{ of }\, \mathbb {S}^n \, \text{ which } \text{ consists } \text{ of } \text{ positive } \text{ semidefinite } \text{ matrices }.\\&M^\top : \text{ the } \text{ transpose } \text{ of } \text{ a } \text{ matrix }\, M.\\&M^\dag : \text{ the } \text{ Moore--Penrose } \text{ pseudoinverse } \text{ of } \text{ a } \text{ matrix }\, M.\\&\mathrm{tr}(M): \text{ the } \text{ sum } \text{ of } \text{ diagonal } \text{ elements } \text{ of } \text{ a } \text{ square } \text{ matrix } \, M.\\&|M|\triangleq \sqrt{\mathrm{tr}(MM^\top )}: \text{ the } \text{ Frobenius } \text{ norm } \text{ of } \text{ a } \text{ matrix } \, M.\\&{\mathscr {R}}(M): \text{ the } \text{ range } \text{ of } \text{ a } \text{ matrix } \text{ or } \text{ an } \text{ operator } \, M. \end{aligned}$$
For \(M,N\in \mathbb {S}^n\), we use the notation \(M\ge N\) (respectively, \(M>N\)) to indicate that \(M-N\) is positive semidefinite (respectively, positive definite). Recall that the inner product \(\langle \cdot \,,\cdot \rangle \) on a Euclidean space is given by \(\langle M,N\rangle \mapsto \mathrm{tr}(M^\top N)\). When there is no confusion, we shall use \(\langle \cdot \,,\cdot \rangle \) for inner products in possibly different Hilbert spaces, and denote by \(|\cdot |\) the induced norm. Let \(T>0\) and \(\mathbb {H}\) be a Euclidean space. We denote
$$\begin{aligned} L_{\mathbb {F}}^2(0,T;\mathbb {H})= & {} \bigg \{\varphi :[0,T]\times \Omega \rightarrow \mathbb {H}~\big |~\varphi (\cdot ) \text{ is }\, {\mathbb {F}}\text{-progressively } \text{ measurable, } \\&\mathbb {E}\int ^T_0|\varphi (t)|^2dt<\infty \bigg \},\\ L_{\mathbb {F}}^2(\mathbb {H})= & {} \left\{ \varphi :[0,\infty )\times \Omega \rightarrow \mathbb {H}~\big |~\varphi (\cdot ) \text{ is }\, {\mathbb {F}}\text{-progressively } \text{ measurable, } \right. \\&\left. \mathbb {E}\int ^\infty _0|\varphi (t)|^2dt<\infty \right\} ,\\ \mathcal{X}[0,T]= & {} \left\{ X:[0,\infty )\times \Omega \rightarrow \mathbb {R}^n~\big |~X(\cdot ) \text{ is } {\mathbb {F}}\text{-adapted, } \text{ continuous, } \right. \\&\left. \mathbb {E}\left(\sup _{0\le t\le T}|X(t)|^2\right)<\infty \right\} , \\ \mathcal{X}_{loc}[0,\infty )= & {} \bigcap _{T>0}\mathcal{X}[0,T],\\ \mathcal{X}[0,\infty )= & {} \left\rbrace X(\cdot )\in \mathcal{X}_{loc}[0,\infty )~\bigg |~\mathbb {E}\int _0^\infty |X(t)|^2dt<\infty \right\lbrace . \end{aligned}$$
We define the inner product on \(L_{\mathbb {F}}^2(\mathbb {H})\) by
$$\begin{aligned} \langle \varphi ,\psi \rangle =\mathbb {E}\int ^\infty _0\langle \varphi (t),\psi (t)\rangle dt \end{aligned}$$
so that \(L_{\mathbb {F}}^2(\mathbb {H})\) becomes a Hilbert space.

2.2 \(L^2\)-stability

For given matrices \(A,C\in \mathbb {R}^{n\times n}\), we denote by [AC] the following uncontrolled linear system:
$$\begin{aligned} \left\{ \begin{array}{ll} dX(t)=AX(t)dt+CX(t)dW(t),\quad t\ge 0, \\ X(0)=x.\end{array}\right. \end{aligned}$$
(2.1)

Definition 2.1

System [AC] is said to be \(L^2\)-stable if for any \(x\in \mathbb {R}^n\), the solution \(X(\cdot \,;x)\) of (2.1) is in \(\mathcal{X}[0,\infty )\).

The following result, which will be used frequently in this paper, provides a characterization of the \(L^2\)-stability of [AC]. For a proof, see [2, 11].

Lemma 2.2

The system [AC] is \(L^2\)-stable if and only if there exists a \(P\in \mathbb {S}^n_+\) such that
$$\begin{aligned} PA+A^\top P+C^\top PC<0. \end{aligned}$$
In this case, for any \(\Lambda \in \mathbb {S}^n\), the Lyapunov equation
$$\begin{aligned} PA+A^\top P+C^\top PC+\Lambda =0 \end{aligned}$$
admits a unique solution \(P\in \mathbb {S}^n\) given by
$$\begin{aligned} P=\mathbb {E}\int _0^\infty \Phi (t)^\top \Lambda \Phi (t)dt, \end{aligned}$$
where \(\Phi (\cdot )\) is the solution to the following SDE for \(\mathbb {R}^{n\times n}\)-valued processes:
$$\begin{aligned} \left\{ \begin{array}{ll} d\Phi (t)=A\Phi (t)dt+C\Phi (t)dW(t),\quad t\ge 0, \\ \Phi (0)=I.\end{array}\right. \end{aligned}$$
(2.2)
Next, we present a result concerning the square-integrability of the solution to the system
$$\begin{aligned} \left\{ \begin{array}{ll} dX(t)=\left[ AX(t)+b(t)\right] dt+\left[ CX(t)+\sigma (t)\right] dW(t),\qquad t\ge 0, \\ X(0)=x.\end{array}\right. \end{aligned}$$
(2.3)
For the proof the reader is referred to Proposition 2.4 in Sun–Yong–Zhang [20].

Lemma 2.3

Suppose that [AC] is \(L^2\)-stable. Then for any \(b(\cdot ),\sigma (\cdot )\in L^2_{\mathbb {F}}(\mathbb {R}^n)\) and any \(x\in \mathbb {R}^n\), the solution \(X(\cdot \,;x,b(\cdot ),\sigma (\cdot ))\) of (2.3) is in \(\mathcal{X}[0,\infty )\). Moreover, there exists a constant \(K>0\), independent of x, \(b(\cdot )\) and \(\sigma (\cdot )\) such that
$$\begin{aligned} \mathbb {E}\int _0^\infty |X(t;x,b(\cdot ),\sigma (\cdot ))|^2dt\le K\left[|x|^2+\mathbb {E}\int _0^\infty \Big (|b(t)|^2+|\sigma (t)|^2\Big )dt\right]. \end{aligned}$$
We now consider the following backward stochastic differential equation (BSDE, for short) on \([0,\infty )\):
$$\begin{aligned} dY(t)=-\left[ A^\top Y(t)+C^\top Z(t)+\varphi (t)\right] dt+Z(t)dW(t),\quad t\in [0,\infty ). \end{aligned}$$
(2.4)

Definition 2.4

An \(L^2\)-stable adapted solution of (2.4) is a pair \((Y(\cdot ),Z(\cdot ))\in \mathcal{X}[0,\infty )\times L_{\mathbb {F}}^2(\mathbb {R}^n)\) which satisfies the integral version of (2.4):
$$\begin{aligned} Y(t)= & {} Y(0)-\int _0^t\left[ A^\top Y(s)+C^\top Z(s)+\varphi (s)\right] ds\\&+\int _0^tZ(s)dW(s), \quad \forall t\in [0,\infty ),\quad \mathrm{a.s.}\end{aligned}$$

The following result, found in [20], establishes the existence and uniqueness of solutions to (2.4).

Lemma 2.5

Suppose that [AC] is \(L^2\)-stable. Then for any \(\varphi (\cdot )\in L^2_{\mathbb {F}}(\mathbb {R}^n)\), BSDE (2.4) admits a unique \(L^2\)-stable adapted solution \((Y(\cdot ),Z(\cdot ))\).

2.3 Pseudoinverse

We recall some properties of the pseudoinverse [15].

Lemma 2.6

  1. (i)
    For any \(M\in \mathbb {R}^{m\times n}\), there exists a unique matrix \(M^\dag \in \mathbb {R}^{n\times m}\) such that
    $$\begin{aligned} MM^\dag M=M,\quad M^\dag MM^\dag =M^\dag ,\quad (MM^\dag )^\top =MM^\dag ,\quad (M^\dag M)^\top =M^\dag M. \end{aligned}$$
    In addition, if \(M\in \mathbb {S}^n\), then \(M^\dag \in \mathbb {S}^n\), and
    $$\begin{aligned} MM^\dag =M^\dag M;\quad M\ge 0\iff M^\dag \ge 0. \end{aligned}$$
     
  2. (ii)
    Let \(L\in \mathbb {R}^{n\times k}\) and \(N\in \mathbb {R}^{n\times m}\). The matrix equation \(NX=L\) has a solution if and only if
    $$\begin{aligned} NN^\dag L=L, \end{aligned}$$
    (2.5)
    in which case the general solution is given by
    $$\begin{aligned} X=N^\dag L+(I-N^\dag N)Y, \end{aligned}$$
    where \(Y\in \mathbb {R}^{m\times k}\) is arbitrary.
     

The matrix \(M^\dag \) above is called the Moore–Penrose pseudoinverse of M.

Remark 2.7

  1. (i)

    Clearly, condition (2.5) is equivalent to \({\mathscr {R}}(L)\subseteq {\mathscr {R}}(N)\).

     
  2. (ii)

    It can be easily seen from Lemma 2.6 that if \(N\in \mathbb {S}^n\) and \(NX=L\), then \(X^\top NX=L^\top N^\dag L\).

     

2.4 A Useful Lemma

We conclude this section with a lemma that will be used frequently in what follows.

Lemma 2.8

Let \(\widehat{G}, \widehat{Q}\in \mathbb {S}^n\), \(\widehat{R}\in \mathbb {S}^m\), and \(\widehat{S}\in \mathbb {R}^{m\times n}\) be given. Suppose that for each \(T>0\) the differential Riccati equation
$$\begin{aligned} \left\{ \begin{array}{ll} \dot{P}(t;T)+P(t;T)A+A^\top P(t;T)+C^\top P(t;T)C+\widehat{Q}\\ \quad -\left[P(t;T)B+C^\top P(t;T)D+\widehat{S}^\top \right]\left[\widehat{R}+D^\top P(t;T)D\right]^{-1}\\ \quad ~\cdot \left[B^\top P(t;T)+D^\top P(t;T)C+\widehat{S}\right]=0,\quad t\in [0,T],\\ P(T;T)=\widehat{G}, \end{array}\right. \end{aligned}$$
admits a solution \(P(\cdot \,;T)\in C([0,T];\mathbb {S}^n)\) such that
$$\begin{aligned} \widehat{R}+D^\top P(t;T)D>0,\quad \forall t\in [0,T]. \end{aligned}$$
If P(0; T) converges to P as \(T\rightarrow \infty \) and \(\widehat{R}+D^\top PD\) is invertible, then
$$\begin{aligned}&PA+A^\top P+C^\top PC+\widehat{Q}\\&\quad -\left(PB+C^\top PD+\widehat{S}^\top \right)\left(\widehat{R}+D^\top PD\right)^{-1}\left(B^\top P+D^\top PC+\widehat{S}\right)=0. \end{aligned}$$

Proof

For fixed but arbitrary \(0<T_1<T_2<\infty \), we define
$$\begin{aligned} \left\{ \begin{array}{llll} P_1(t) &{}= &{} P(T_1-t;T_1),\quad &{} 0\le t\le T_1,\\ P_2(t) &{}= &{} P(T_2-t;T_2),\quad &{} 0\le t\le T_2. \end{array}\right. \end{aligned}$$
On the interval \([0,T_1]\), both \(P_1(\cdot )\) and \(P_2(\cdot )\) satisfy the following equation for \(\Sigma (\cdot )\):
$$\begin{aligned} \left\{ \begin{array}{ll} \dot{\Sigma }(t)-\Sigma (t)A-A^\top \Sigma (t)-C^\top \Sigma (t)C-\widehat{Q}\\ +\,\Big [\Sigma (t)B+C^\top \Sigma (t)D+\widehat{S}^\top \Big ]\Big [\widehat{R}+D^\top \Sigma (t)D\Big ]^{-1}\Big [B^\top \Sigma (t)+D^\top \Sigma (t)C+\widehat{S}\Big ]=0,\\ \Sigma (0)=\widehat{G}.\end{array}\right. \end{aligned}$$
(2.6)
A standard argument using the Gronwall inequality shows
$$\begin{aligned} P_1(\cdot )=P_2(\cdot ),\quad \text{ on } ~[0,T_1]. \end{aligned}$$
Therefore, we may define a function \(\Sigma (\cdot ):[0,\infty )\rightarrow \mathbb {S}^n\) by the following:
$$\begin{aligned} \Sigma (t)=P(T-t;T),\quad \text{ if } ~0\le t\le T. \end{aligned}$$
Noting that \(\Sigma (\cdot )\) satisfies (2.6) on the whole interval \([0,\infty )\), we have for any \(T>0\),
$$\begin{aligned} \Sigma (T+1)-\Sigma (T)= & {} \int _T^{T+1}\Big \{\Sigma (t)A+A^\top \Sigma (t)+C^\top \Sigma (t)C+\widehat{Q}\\&-\,\Big [\Sigma (t)B+C^\top \Sigma (t)D+\widehat{S}^\top \Big ]\Big [\widehat{R}+D^\top \Sigma (t)D\Big ]^{-1}\\&\times \Big [B^\top \Sigma (t)+D^\top \Sigma (t)C+\widehat{S}\Big ]\Big \}dt. \end{aligned}$$
The desired result follows by letting \(T\rightarrow \infty \) in the above. \(\square \)

3 Admissible Control Sets and Stabilizability

In this section, we will look into the admissible control sets.

3.1 General Cases

Since the state equation is linear, for any given \(x\in \mathbb {R}^n\), the set \(\mathcal{U}_{ad}(x)\) of admissible controls is either empty or a convex set in \(L^2_{\mathbb {F}}(\mathbb {R}^m)\). To investigate Problem (LQ), we should find conditions for the system so that the set \(\mathcal{U}_{ad}(x)\) is at least non-empty and hopefully it admits an accessible characterization. To this end, we denote by [ACBD] the following controlled system:
$$\begin{aligned} \left\{ \begin{array}{ll} dX(t)=\left[ AX(t)+Bu(t)\right] dt+\left[ CX(t)+Du(t)\right] dW(t),\quad t\ge 0, \\ X(0)=x,\end{array}\right. \end{aligned}$$
and introduce the following definition.

Definition 3.1

System [ACBD] is said to be \(L^2\) -stabilizable if there exists a \(\Theta \in \mathbb {R}^{m\times n}\) such that \([A+B\Theta ,C+D\Theta ]\) is \(L^2\)-stable. In this case, \(\Theta \) is called a stabilizer of [ACBD]. The set of all stabilizers of [ACBD] is denoted by \(\mathscr {S}\equiv \mathscr {S}[A,C;B,D]\).

One observes that the \(L^2\)-stabilizability of [ACBD] implies the non-emptiness of the admissible control set \(\mathcal{U}_{ad}(x)\) for all x. Indeed, if \(\Theta \) is a stabilizer of [ACBD], by Lemma 2.3, the solution \(X(\cdot )\) of the following SDE is in \(\mathcal{X}[0,\infty )\):
$$\begin{aligned} \left\{ \begin{array}{ll} dX(t)=\left[ (A+B\Theta )X(t)+b(t)\right] dt+\left[ (C+D\Theta )X(t)+\sigma (t)\right] dW(t),\quad t\ge 0,\\ X(0)= x.\end{array}\right. \end{aligned}$$
Hence, \(u(\cdot )\triangleq \Theta X(\cdot )\in \mathcal{U}_{ad}(x)\).

The following result shows that the \(L^2\)-stabilizability is not only sufficient, but also necessary, for the non-emptiness of all admissible control sets.

Theorem 3.2

The following statements are equivalent:
  1. (i)

    \(\mathcal{U}_{ad}(x)\ne \emptyset \) for all \(x\in \mathbb {R}^n\);

     
  2. (ii)

    \(\mathscr {S}[A,C;B,D]\ne \emptyset \);

     
  3. (iii)
    The following ARE admits a positive solution \(P\in \mathbb {S}^n_+\):
    $$\begin{aligned}&PA+A^\top P+C^\top PC+I\nonumber \\&\quad -\left( PB+C^\top PD\right) \left( I+D^\top PD\right) ^{-1}\left( B^\top P+D^\top PC\right) =0. \end{aligned}$$
    (3.1)
    If the above are satisfied and P is a positive solution of (3.1), then
    $$\begin{aligned} \Gamma \triangleq -\left( I+D^\top PD\right) ^{-1}\left( B^\top P+D^\top PC\right) \in \mathscr {S}[A,C;B,D]. \end{aligned}$$
    (3.2)
     

Proof

We have proved the implication (ii) \(\mathop {\Rightarrow }\) (i) above. For the implication (iii) \(\mathop {\Rightarrow }\) (ii), we observe that if P is a positive solution of (3.1) and \(\Gamma \) is defined by (3.2), then
$$\begin{aligned} P(A+B\Gamma )+(A+B\Gamma )^\top P+(C+D\Gamma )^\top P(C+D\Gamma )=-I-\Gamma ^\top \Gamma <0. \end{aligned}$$
Hence, by Lemma 2.2 and Definition 3.1, \(\Gamma \) is stabilizer of [ACBD].
We next show that (i) \(\mathop {\Rightarrow }\) (iii). By subtracting solutions of (1.1) corresponding to x and 0, we may assume without loss of generality that \(b(\cdot )=\sigma (\cdot )=0\). Let \(e_1,\cdots ,e_n\) be the standard basis for \(\mathbb {R}^n\). Take \(u_i(\cdot )\in \mathcal{U}_{ad}(e_i)\), \(i=1,\cdots ,n\), and set
$$\begin{aligned} U(\cdot )=(u_1(\cdot ),\cdots ,u_n(\cdot )). \end{aligned}$$
Then \(U(\cdot )x\in \mathcal{U}_{ad}(x)\) for all \(x\in \mathbb {R}^n\). Consider the following cost functional:
$$\begin{aligned} \bar{J}(x;u(\cdot ))=\mathbb {E}\int _0^\infty \Big [|X(t)|^2+|u(t)|^2\Big ]dt. \end{aligned}$$
Let \(\mathbb {X}(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^{n\times n})\) be the solution to the following SDE for \(\mathbb {R}^{n\times n}\)-valued processes:
$$\begin{aligned} \left\{ \begin{array}{ll} d\mathbb {X}(t)=\left[ A\mathbb {X}(t)+BU(t)\right] dt+\left[ C\mathbb {X}(t)+DU(t)\right] dW(t),\quad t\ge 0,\\ \mathbb {X}(0)=I.\end{array}\right. \end{aligned}$$
We have
$$\begin{aligned}&\inf _{u(\cdot )\in \mathcal{U}_{ad}(x)}\bar{J}(x;u(\cdot ))\le \mathbb {E}\int _0^\infty \Big [|\mathbb {X}(t)x|^2+|U(t)x|^2\Big ]dt\\&\quad =\Big \langle \Big (\mathbb {E}\int _0^\infty \left[ \mathbb {X}(t)^\top \mathbb {X}(t)+U(t)^\top U(t)\right] dt\Big )x,x\Big \rangle \equiv \langle \Lambda x,x\rangle , \quad \forall x\in \mathbb {R}^n. \end{aligned}$$
Now for any fixed \(T>0\), let us consider the state equation
$$\begin{aligned} \left\{ \begin{array}{ll} dX_T(t)=\left[ AX_T(t)+Bu(t)\right] dt+\left[ CX_T(t)+Du(t)\right] dW(t),\qquad t\in [0,T],\\ X_T(0)=x,\end{array}\right. \end{aligned}$$
and the cost functional
$$\begin{aligned} \bar{J}_T(x;u(\cdot ))=\mathbb {E}\int _0^T\Big [|X_T(t)|^2+|u(t)|^2\Big ]dt. \end{aligned}$$
It is standard that the following differential Riccati equation
$$\begin{aligned} \left\{ \begin{array}{ll} \dot{P}(t;T)+P(t;T)A+A^\top P(t;T)+C^\top P(t;T)C+I\\ ~-\left[ P(t;T)B+C^\top P(t;T)D\right] \left[ I+D^\top P(t;T)D\right] ^{-1}\\ ~\times \left[ B^\top P(t;T)+D^\top P(t;T)C\right] =0,\quad t\in [0,T],\\ P(T;T)=0, \end{array}\right. \end{aligned}$$
admits a unique solution \(P(\cdot \,;T)\in C([0,T];\mathbb {S}^n_+)\) such that
$$\begin{aligned} \inf _{u(\cdot )\in L_{\mathbb {F}}^2(0,T;\mathbb {R}^m)}\bar{J}_T(x;u(\cdot )) =\langle P(0;T)x,x\rangle ,\quad \forall x\in \mathbb {R}^n. \end{aligned}$$
It is clear that
$$\begin{aligned} \begin{array}{llll} \bar{J}_T(x;u(\cdot ))\le \bar{J}(x;u(\cdot )), &{}\quad \forall x\in \mathbb {R}^n,&{} \forall u(\cdot )\in \mathcal{U}_{ad}(x),&{} \forall \, 0<T<\infty ,\\ \bar{J}_T(x;u(\cdot ))\le \bar{J}_{T^\prime }(x;u(\cdot )), &{}\quad \forall x\in \mathbb {R}^n,&{}\forall u(\cdot )\in L_{\mathbb {F}}^2(0,T^\prime ;\mathbb {R}^m),&{}\forall \, 0<T<T^\prime <\infty . \end{array} \end{aligned}$$
Thus, one has
$$\begin{aligned} 0<P(0;T)\le P(0;T^\prime )\le \Lambda ,\quad 0<T<T^\prime <\infty . \end{aligned}$$
This implies that P(0; T) converges increasingly to some \(P\in \mathbb {S}^n_+\) as \(T\nearrow \infty \). By Lemma 2.8, P solves the ARE (3.1). \(\square \)

Theorem 3.2 provides a characterization of the non-emptiness of all admissible control sets. The following result further gives an explicit description of the admissible controls.

Proposition 3.3

Suppose that \(\Theta \in \mathscr {S}[A,C;B,D]\). Then for any \(x\in \mathbb {R}^n\),
$$\begin{aligned} \mathcal{U}_{ad}(x)=\big \{\Theta X_\Theta (\cdot \,;x,v(\cdot ))+v(\cdot )\,|\,v(\cdot )\in L^2_{\mathbb {F}}(\mathbb {R}^m)\big \}, \end{aligned}$$
where \(X_\Theta (\cdot \,;x,v(\cdot ))\) is the solution to the following SDE:
$$\begin{aligned} \left\{ \begin{array}{ll} dX_\Theta (t)=\left[ (A+B\Theta )X_\Theta (t)+Bv(t)+b(t)\right] dt\\ \qquad \qquad \qquad +\left[ (C+D\Theta )X_\Theta (t)+Dv(t)+\sigma (t)\right] dW(t),\quad t\ge 0, \\ X_\Theta (0)= x.\end{array}\right. \end{aligned}$$
(3.3)

Proof

Let \(v(\cdot )\in L^2_{\mathbb {F}}(\mathbb {R}^m)\) and \(X_\Theta (\cdot )\equiv X_\Theta (\cdot \,;x,v(\cdot ))\) be the solution of (3.3). Since \([A+B\Theta ,C+D\Theta ]\) is \(L^2\)-stable, by Lemma 2.3, \(X_\Theta (\cdot )\in \mathcal{X}[0,\infty )\). Set
$$\begin{aligned} u(\cdot )\triangleq \Theta X_\Theta (\cdot )+v(\cdot )\in L^2_{\mathbb {F}}(\mathbb {R}^m), \end{aligned}$$
and let \(X(\cdot )\in \mathcal{X}_{loc}[0,\infty )\) be the solution to
$$\begin{aligned} \left\{ \begin{array}{ll} dX(t)=\left[ AX(t)+Bu(t)+b(t)\right] dt\\ +\left[ CX(t)+Du(t)+\sigma (t)\right] dW(t),\quad t\ge 0,\\ X(0)=x.\end{array}\right. \end{aligned}$$
(3.4)
By uniqueness of solutions, we have \(X(\cdot )=X_\Theta (\cdot )\in \mathcal{X}[0,\infty )\), and therefore \(u(\cdot )\in \mathcal{U}_{ad}(x)\).
On the other hand, suppose \(u(\cdot )\in \mathcal{U}_{ad}(x)\). Let \(X(\cdot )\in \mathcal{X}[0,\infty )\) be the solution of (3.4) and set
$$\begin{aligned} v(\cdot )\triangleq u(\cdot )-\Theta X(\cdot )\in L^2_{\mathbb {F}}(\mathbb {R}^m). \end{aligned}$$
Again by uniqueness of solutions, we see that \(X(\cdot )\) coincides with the solution \(X_\Theta (\cdot )\) of (3.3). Thus, \(u(\cdot )\) admits a representation of the form \(\Theta X_\Theta (\cdot \,;x,v(\cdot ))+v(\cdot )\). \(\square \)

3.2 The Case \(n=1\)

In this subsection, we look at the case \(n=1\), i.e., the state variable is one-dimensional. However, the control is allowed to be multi-dimensional. First, we present the following lemma.

Lemma 3.4

Let \(n=1\). If the system [ACBD] is not \(L^2\)-stabilizable, then
$$\begin{aligned} \begin{pmatrix}2A+C^2&{}\quad B+CD\\ B^\top +CD^\top &{}\quad D^\top D\end{pmatrix}\ge 0. \end{aligned}$$
(3.5)

Proof

If [ACBD] is not \(L^2\)-stabilizable, then by Definition 3.1 and Lemma 2.2, we have
$$\begin{aligned} 2(A+B\Theta )+(C+D\Theta )^2\ge 0,\quad \forall \Theta \in \mathbb {R}^m. \end{aligned}$$
Since for any nonzero \(x\in \mathbb {R}\) and any \(y\in \mathbb {R}^m\) one can find a \(\Theta \in \mathbb {R}^m\) such that \(y=\Theta x\), we have
$$\begin{aligned}&\begin{pmatrix}x&y^\top \end{pmatrix} \begin{pmatrix}2A+C^2&{}\quad B+CD\\ B^\top +CD^\top &{}\quad D^\top D\end{pmatrix} \begin{pmatrix}x\\ y\end{pmatrix}\\&=x\begin{pmatrix}1&\Theta ^\top \end{pmatrix} \begin{pmatrix}2A+C^2&{}\quad B+CD\\ B^\top +CD^\top &{}\quad D^\top D\end{pmatrix} \begin{pmatrix}1\\ \Theta \end{pmatrix}x\\&=\left[ 2(A+B\Theta )+(C+D\Theta )^2\right] x^2\ge 0,\quad \forall x\ne 0,~y\in \mathbb {R}^m, \end{aligned}$$
and the result follows. \(\square \)
For the moment let us assume that \(b(\cdot )=\sigma (\cdot )=0\) and introduce the following space:
$$\begin{aligned} \mathcal{V}=\bigg \{u(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m)\,|\,Bu(\cdot )=Du(\cdot )=0,~\mathrm{a.e.}\,\, \mathrm{a.s.}\bigg \}. \end{aligned}$$
Obviously, \(0\in \mathcal{V}\subseteq \mathcal{U}_{ad}(0)\), and hence \(\mathcal{U}_{ad}(0)\) is non-empty. In fact, when \(\mathscr {S}[A,C;B,D]=\emptyset \), \(\mathcal{U}_{ad}(0)\) coincides with \(\mathcal{V}\), and is the only non-empty admissible control set. More precisely, we have the following result.

Theorem 3.5

Let \(n=1\), and suppose that \(b(\cdot )=\sigma (\cdot )=0\). Then exactly one of the following holds:
  1. (i)

    \(\mathcal{U}_{ad}(0)=\mathcal{V}\) and \(\mathcal{U}_{ad}(x)=\emptyset \) for all \(x\ne 0\).

     
  2. (ii)

    The system [ACBD] is \(L^2\)-stabilizable.

     

Proof

We prove it by contradiction. From Theorem 3.2 we see that (i) and (ii) cannot hold simultaneously. Now suppose that neither (i) nor (ii) holds. Then either \(\mathcal{U}_{ad}(0)\backslash \mathcal{V}\ne \emptyset \) or else \(\mathcal{U}_{ad}(x)\ne \emptyset \) for some \(x\ne 0\), and (3.5) holds by Lemma 3.4. If there exists a \(u(\cdot )\in \mathcal{U}_{ad}(0)\backslash \mathcal{V}\), then with \(X_0(\cdot )\) denoting the solution of (1.1) corresponding to the initial state \(x=0\) and the admissible control \(u(\cdot )\), we have
$$\begin{aligned} \mathbb {E}|X_0(t)|^2= & {} \mathbb {E}\int _0^t\Big \{2[AX_0(s)+Bu(s)]X_0(s)+|CX_0(s)+Du(s)|^2\Big \}ds\\= & {} \mathbb {E}\int _0^t\left\langle \begin{pmatrix}2A+C^2 &{} B+CD\\ B^\top +CD^\top &{} D^\top D\end{pmatrix} \begin{pmatrix}X_0(s)\\ u(s)\end{pmatrix}, \begin{pmatrix}X_0(s)\\ u(s)\end{pmatrix}\right\rangle ds\ge 0. \end{aligned}$$
Since (3.5) holds and \(X_0(\cdot )\in \mathcal{X}[0,\infty )\) (and hence \(\lim _{t\rightarrow \infty }\mathbb {E}|X_0(t)|^2=0\)), the integrand in the above must vanish for all \(s\ge 0\). It turns out that \(X_0(\cdot )\equiv 0\), and hence
$$\begin{aligned} Bu(\cdot )=0, \quad Du(\cdot )=0,\quad \mathrm{a.e.}\,\, \mathrm{a.s.} \end{aligned}$$
which is a contradiction. Now if \(\mathcal{U}_{ad}(x)\ne \emptyset \) for some \(x\ne 0\), take \(v(\cdot )\in \mathcal{U}_{ad}(x)\) and let \(X(\cdot )\) be the solution of (1.1) corresponding to x and \(v(\cdot )\). Then, using (3.5), we have for any \(t\ge 0\),
$$\begin{aligned} \mathbb {E}|X(t)|^2-|x|^2=\mathbb {E}\int _0^t\left\langle \begin{pmatrix}2A+C^2 &{} B+CD\\ B^\top +CD^\top &{} D^\top D\end{pmatrix} \begin{pmatrix}X(s)\\ v(s)\end{pmatrix}, \begin{pmatrix}X(s)\\ v(s)\end{pmatrix}\right\rangle ds\ge 0, \end{aligned}$$
which is impossible since \(\lim _{t\rightarrow \infty }\mathbb {E}|X(t)|^2=0\). This completes the proof. \(\square \)

For the case \(b(\cdot )\ne 0\) or \(\sigma (\cdot )\ne 0\), we have the following result, which is a consequence of Theorem 3.5.

Theorem 3.6

Let \(n=1\), and suppose that \(b(\cdot )\ne 0\) or \(\sigma (\cdot )\ne 0\). Then exactly one of the following holds:
  1. (i)

    \(\mathcal{U}_{ad}(x)=\emptyset \) for all \(x\in \mathbb {R}^n\).

     
  2. (ii)
    There is only one \(x\in \mathbb {R}^n\) for which the admissible control set \(\mathcal{U}_{ad}(x)\ne \emptyset \). In this case,
    $$\begin{aligned} u(\cdot )-v(\cdot )\in \mathcal{V},\quad \forall u(\cdot ),v(\cdot )\in \mathcal{U}_{ad}(x). \end{aligned}$$
     
  3. (iii)

    The system [ACBD] is \(L^2\)-stabilizable.

     

Proof

Clearly, any two of the statements (i)–(iii) cannot hold simultaneously. To show the above, let us assume that neither (i) nor (ii) holds. Then
$$\begin{aligned} \mathcal{U}_{ad}(x_1)\ne \emptyset ,\quad \mathcal{U}_{ad}(x_2)\ne \emptyset \end{aligned}$$
for some \(x_1\ne x_2\). Take \(u_i(\cdot )\in \mathcal{U}_{ad}(x_i)\), \(i=1,2\), and let \(X_i(\cdot )\) be the solution of (1.1) corresponding to the initial state \(x_i\) and the admissible control \(u_i(\cdot )\). Then with \(x=x_1-x_2\) and \(u(\cdot )=u_1(\cdot )-u_2(\cdot )\), the process \(X(\cdot )\triangleq X_1(\cdot )-X_2(\cdot )\) is in \(\mathcal{X}[0,\infty )\) and solves
$$\begin{aligned} \left\{ \begin{array}{ll} dX(t)=\left[ AX(t)+Bu(t)\right] dt+\left[ CX(t)+Du(t)\right] dW(t),\quad t\ge 0, \\ X(0)=x.\end{array}\right. \end{aligned}$$
Thus, by Theorem 3.5, the system [ACBD] is \(L^2\)-stabilizable.
Now suppose that there is only one \(x\in \mathbb {R}^n\) such that \(\mathcal{U}_{ad}(x)\ne \emptyset \). The same argument as before shows that for any \(u(\cdot ),v(\cdot )\in \mathcal{U}_{ad}(x)\), the solution \(X_0(\cdot )\) of
$$\begin{aligned} \left\{ \begin{array}{ll} dX_0(t)=\big \{AX_0(t)+B[u(t)-v(t)]\big \}dt+\big \{CX_0(t)+D[u(t)-v(t)]\big \}dW(t),\quad t\ge 0,\\ X_0(0)=0,\end{array}\right. \end{aligned}$$
is in \(\mathcal{X}[0,\infty )\). Since \(\mathscr {S}[A,C;B,D]=\emptyset \) in this situation, we have \(u(\cdot )-v(\cdot )\in \mathcal{V}\) by Theorem 3.5. \(\square \)

4 Solvabilities of Problem (LQ) and Generalized AREs

Let us return to Problem (LQ). According to Theorem 3.2, when [ACBD] is not \(L^2\)-stabilizable, Problem (LQ) becomes ill-posed. Because of this, we shall impose the following assumption in the rest of the paper:

(H1) System [ACBD] is \(L^2\)-stabilizable, i.e., \(\mathscr {S}[A,C;B,D]\ne \emptyset \).

Now, we introduce the following definitions.

Definition 4.1

An element \(u^*(\cdot )\in \mathcal{U}_{ad}(x)\) is called an open-loop optimal control of Problem (LQ) for the initial state \(x\in \mathbb {R}^n\) if
$$\begin{aligned} J(x;u^*(\cdot ))\le J(x;u(\cdot )),\quad \forall u(\cdot )\in \mathcal{U}_{ad}(x). \end{aligned}$$
(4.1)
If an open-loop optimal control (uniquely) exists for \(x\in \mathbb {R}^n\), Problem (LQ) is said to be (uniquely) open-loop solvable at x. Problem (LQ) is said to be (uniquely) open-loop solvable if it is (uniquely) open-loop solvable at all \(x\in \mathbb {R}^n\).

Definition 4.2

  1. (i)
    A pair \((\Theta ,v(\cdot ))\in \mathscr {S}[A,C;B,D]\times L^2_{\mathbb {F}}(\mathbb {R}^m)\) is called a closed-loop strategy of Problem (LQ). The outcome
    $$\begin{aligned} u(\cdot )\equiv \Theta X(\cdot )+v(\cdot ) \end{aligned}$$
    of a closed-loop strategy \((\Theta ,v(\cdot ))\) is called a closed-loop control for the initial state x, where \(X(\cdot )\) is the closed-loop state process corresponding to \((x,\Theta ,v(\cdot ))\):
    $$\begin{aligned} \left\{ \begin{array}{ll} dX(t)=\left[ (A+B\Theta )X(t)+Bv(t)+b(t)\right] dt\\ \qquad \qquad \quad +\left[ (C+D\Theta )X(t)+Dv(t)+\sigma (t)\right] dW(t), \quad t\ge 0,\\ X(0)=x.\end{array}\right. \end{aligned}$$
     
  2. (ii)
    A closed-loop strategy \((\Theta ^*,v^*(\cdot ))\) is said to be optimal if
    $$\begin{aligned}&J(x;\Theta ^*X^*(\cdot )+v^*(\cdot ))\le J(x;\Theta X(\cdot )+v(\cdot )), \nonumber \\&\quad \forall (x,\Theta ,v(\cdot ))\in \mathbb {R}^n\times \mathscr {S}[A,C;B,D]\times L^2_{\mathbb {F}}(\mathbb {R}^m), \end{aligned}$$
    (4.2)
    where \(X^*(\cdot )\) and \(X(\cdot )\) are the closed-loop state processes corresponding to \((x,\Theta ^*,v^*(\cdot ))\) and \((x,\Theta ,v(\cdot ))\), respectively. If a closed-loop optimal strategy (uniquely) exists, Problem (LQ) is said to be (uniquely) closed-loop solvable.
     

Remark 4.3

From Proposition 3.3 we see that when system [ACBD] is \(L^2\)-stabilizable, the set \(\mathcal{U}_{ad}(x)\) of admissible controls is made of closed-loop controls for all x, and that (4.2) is equivalent to the following condition:
$$\begin{aligned} J(x;\Theta ^*X^*(\cdot )+v^*(\cdot ))\le J(x;u(\cdot )), \quad \forall (x,u(\cdot ))\in \mathbb {R}^n\times \mathcal{U}_{ad}(x). \end{aligned}$$
(4.3)

It is worth pointing out that, in general, the admissible control sets \(\mathcal{U}_{ad}(x)\) are different for different x, and an open-loop optimal control depends on the initial state \(x\in \mathbb {R}^n\), whereas a closed-loop optimal strategy is required to be independent of x. From (4.2), one sees that the outcome \(u^*(\cdot )\equiv \Theta ^*X^*(\cdot )+v^*(\cdot )\) of a closed-loop optimal strategy \((\Theta ^*,v^*(\cdot ))\) is an open-loop optimal control for the initial state \(X^*(0)\). Hence, closed-loop solvability implies open-loop solvability. For LQ optimal control problems in finite horizon, the same is true, and open-loop solvability does not necessarily imply closed-loop solvability (see [18]). However, for our Problem (LQ) (in an infinite horizon), as we shall prove later, the open-loop and closed-loop solvabilities are equivalent, and both are equivalent to the existence of a static stabilizing solution to a generalized algebraic Riccati equation which we are going to introduce below.

Definition 4.4

The following constrained nonlinear algebraic equation
$$\begin{aligned} \left\{ \begin{array}{ll} PA+A^{\top } P+C^{\top } PC+Q\\ -\left( PB+C^{\top } PD+S^{\top }\right) \left( R+D^{\top } PD\right) ^{\dag } \left( B^\top P+D^\top PC+S\right) =0,\\ {\mathscr {R}}\left( B^\top P+D^\top PC+S\right) \subseteq {\mathscr {R}}\left( R+D^\top PD\right) ,\\ R+D^\top PD\ge {0} \end{array}\right. \end{aligned}$$
(4.4)
with the unknown \(P\in {\mathbb {S}}^{n}\), is called a generalized algebraic Riccati equation. A solution P of (4.4) is said to be static stabilizing if there exists a \({\Pi }\in {\mathbb {R}}^{m\times n}\) such that
$$\begin{aligned}&-\left( R+D^{\top } PD\right) ^\dag \left( B^{\top } P+D^{\top } PC+S\right) \\&\quad \quad \quad +\left[ I-\left( R+D^{\top } PD\right) ^{\dag }\left( R+D^{\top } PD\right) \right] \Pi \in \mathscr {S}[A,C;B,D]. \end{aligned}$$
For notational simplicity, we shall write hereafter
$$\begin{aligned} \mathcal{M}(P)= & {} PA+A^\top P+C^\top PC+Q, \quad ~\mathcal{L}(P)=PB+C^\top PD+S^\top ,\\ \mathcal{N}(P)= & {} R+D^\top PD. \end{aligned}$$
Now we state the main result of this paper.

Theorem 4.5

Let (H1) hold. Then the following statements are equivalent:
  1. (i)

    Problem (LQ) is open-loop solvable;

     
  2. (ii)

    Problem (LQ) is closed-loop solvable;

     
  3. (iii)
    ARE (4.4) admits a static stabilizing solution \(P\in \mathbb {S}^n\), and the BSDE
    $$\begin{aligned} \begin{array}{ll} d\eta =-\Big \{\left[ A^\top -\mathcal{L}(P)\mathcal{N}(P)^\dag B^\top \right] \eta +\left[ C^\top -\mathcal{L}(P)\mathcal{N}(P)^\dag D^\top \right] \zeta \\ \quad \qquad +\,\left[ C^\top -\mathcal{L}(P)\mathcal{N}(P)^\dag D^\top \right] P\sigma -\mathcal{L}(P)\mathcal{N}(P)^\dag \rho +Pb+q\Big \}dt+\zeta dW, \quad t\ge 0, \end{array}\nonumber \\ \end{aligned}$$
    (4.5)
    admits an \(L^2\)-stable adapted solution \((\eta (\cdot ),\zeta (\cdot ))\) such that
    $$\begin{aligned} B^\top \eta (t)+D^\top \zeta (t)+D^\top P\sigma (t)+\rho (t)\in {\mathscr {R}}\left( \mathcal{N}(P)\right) ,\quad \mathrm{a.e.}\,\, t\in [0,\infty ),~~\mathrm{a.s.}\end{aligned}$$
    (4.6)
    In the above case, any closed-loop optimal strategy \((\Theta ^*,v^*(\cdot ))\) is given by
    $$\begin{aligned} \left\{ \begin{array}{ll} \Theta ^*=-\mathcal{N}(P)^\dag \mathcal{L}(P)^\top +\left[ I-\mathcal{N}(P)^\dag \mathcal{N}(P)\right] \Pi ,\\ v^*(\cdot )=-\mathcal{N}(P)^\dag \left[ B^\top \eta (\cdot )+D^\top \zeta (\cdot )+D^\top P\sigma (\cdot )+\rho (\cdot )\right] \\ \qquad \qquad +\left[ I-\mathcal{N}(P)^\dag \mathcal{N}(P)\right] \nu (\cdot ), \end{array}\right. \end{aligned}$$
    (4.7)
    where \(\Pi \in \mathbb {R}^{m\times n}\) is chosen so that \(\Theta ^*\in \mathscr {S}[A,C;B,D]\), and \(\nu (\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m)\); every open-loop optimal control \(u^*(\cdot )\) for the initial state x admits a closed-loop representation:
    $$\begin{aligned} u^*(\cdot )=\Theta ^*X^*(\cdot )+v^*(\cdot ), \end{aligned}$$
    (4.8)
    where \((\Theta ^*,v^*(\cdot ))\) is a closed-loop optimal strategy of Problem (LQ) and \(X^*(\cdot )\) is the corresponding closed-loop state process. Further, the value function admits the following representation:
    $$\begin{aligned} \begin{array}{ll} V(x)=\langle Px,x\rangle +\mathbb {E}\bigg \{2\langle \eta (0),x\rangle +\displaystyle \int \nolimits _0^\infty \Big [\langle P\sigma ,\sigma \rangle +2\langle \eta ,b\rangle +2\langle \zeta ,\sigma \rangle \\ \quad \qquad \qquad -\,\big \langle \mathcal{N}(P)^\dag \left( B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \right) , B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \big \rangle \Big ]dt\bigg \}.\end{array}\nonumber \\ \end{aligned}$$
    (4.9)
     
The proof will be given in the subsequent sections. We make some observations here. Suppose that ARE (4.4) admits a static stabilizing solution \(P\in \mathbb {S}^n\). Then one can choose a matrix \(\Pi \in \mathbb {R}^{m\times n}\) such that
$$\begin{aligned} \Theta =-\mathcal{N}(P)^\dag \mathcal{L}(P)^\top +\left[ I-\mathcal{N}(P)^\dag \mathcal{N}(P)\right] \Pi \in \mathscr {S}[A,C;B,D]. \end{aligned}$$
If \((\eta (\cdot ),\zeta (\cdot ))\) is an \(L^2\)-stable adapted solution of (4.5) satisfying (4.6), then it follows easily from Lemma 2.6 that
$$\begin{aligned} \left[ \Theta ^\top +\mathcal{L}(P)\mathcal{N}(P)^\dag \right] \left[ B^\top \eta (t)+D^\top \zeta (t)+D^\top P\sigma (t)+\rho (t)\right] =0. \end{aligned}$$
Thus,
$$\begin{aligned} d\eta= & {} -\,\Big \{A^\top \eta +C^\top \zeta +C^\top P\sigma +Pb+q\\&\quad \quad -\mathcal{L}(P)\mathcal{N}(P)^\dag \left[ B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \right] \Big \}dt+\zeta dW\\= & {} -\,\Big \{A^\top \eta +C^\top \zeta +C^\top P\sigma +Pb+q\\&\quad \quad +\Theta ^\top \left[ B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \right] \Big \}dt+\zeta dW\\= & {} -\,\Big \{(A+B\Theta )^\top \eta +(C+D\Theta )^\top \zeta +(C+D\Theta )^\top P\sigma \\&\quad \quad +Pb+q+\Theta ^\top \rho \Big \}dt+\zeta dW. \end{aligned}$$
Since \([A+B\Theta ,C+D\Theta ]\) is \(L^2\)-stable, we conclude from Lemma 2.5 that the \(L^2\)-stable adapted solution of (4.5) satisfying (4.6) is unique. In particular, for Problem (LQ)\(^0\) where the coefficients \(b(\cdot )\), \(\sigma (\cdot )\), \(q(\cdot )\), and \(\rho (\cdot )\) are all identically zero, \((\eta (\cdot ),\zeta (\cdot ))\equiv (0,0)\) is the unique solution of (4.5) such that (4.6) holds. This leads to the following result.

Corollary 4.6

Let (H1) hold. Then the following statements are equivalent:
  1. (i)

    Problem (LQ)\(^0\) is open-loop solvable;

     
  2. (ii)

    Problem (LQ)\(^0\) is closed-loop solvable;

     
  3. (iii)

    ARE (4.4) admits a static stabilizing solution \(P\in \mathbb {S}^n\).

     
In the above case, all closed-loop optimal strategies \((\Theta ^*,v^*(\cdot ))\) are given by
$$\begin{aligned} \Theta ^*= & {} -\mathcal{N}(P)^\dag \mathcal{L}(P)^\top +\left[ I-\mathcal{N}(P)^\dag \mathcal{N}(P)\right] \Pi ,\quad \nonumber \\&v^*(\cdot )=\left[ I-\mathcal{N}(P)^\dag \mathcal{N}(P)\right] \nu (\cdot ), \end{aligned}$$
(4.10)
where \(\Pi \in \mathbb {R}^{m\times n}\) is chosen so that \(\Theta ^*\in \mathscr {S}[A,C;B,D]\), and \(\nu (\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m)\); the value function is given by
$$\begin{aligned} V(x)=\langle Px,x\rangle ,\quad x\in \mathbb {R}^n. \end{aligned}$$
(4.11)

Remark 4.7

From (4.11), it is easily seen that ARE (4.4) admits at most one static stabilizing solution.

To conclude this section, we give an informal explanation why the open-loop solvability and the closed-loop solvability coincide for LQ optimal control problems in infinite horizon. We take Problem (LQ)\(^0\) for example. In this case the state equation and the cost functional respectively become
$$\begin{aligned}&\left\{ \begin{array}{ll} dX(t)=\left[ AX(t)+Bu(t)\right] dt+\left[ CX(t)+Du(t)\right] dW(t),\quad t\ge 0,\\ X(0)= x,\end{array}\right.&\end{aligned}$$
(4.12)
$$\begin{aligned}&\displaystyle J(x;u(\cdot ))=\mathbb {E}\int _0^\infty \left\langle \begin{pmatrix}Q&{}S^\top \\ S&{}R\end{pmatrix} \begin{pmatrix}X(t)\\ u(t)\end{pmatrix}, \begin{pmatrix}X(t)\\ u(t)\end{pmatrix}\right\rangle dt.&\end{aligned}$$
(4.13)
Suppose Problem (LQ)\(^0\) is open-loop solvable. Then there exists an \(\mathbb {R}^{m\times n}\)-valued process \(U^*(\cdot )\) such that for any initial distribution \(\xi \), \(U^*(\cdot )\xi \) is the open-loop optimal control corresponding to the linear quadratic structure of Problem (LQ)\(^0\). Let \(X^*(\cdot )\) be the solution of (4.12 4.13) corresponding to the initial state x and the open-loop optimal control \(u^*(\cdot )\equiv U^*(\cdot )x\). For any \(s\ge 0\), we may consider Problem (LQ)\(^0\) with initial time being s. That is to say, the state equation reads
$$\begin{aligned} \left\{ \begin{array}{ll} dX(t)=\left[ AX(t)+Bu(t)\right] dt+\left[ CX(t)+Du(t)\right] dW(t),\quad t\ge s, \\ X(s)=x,\end{array}\right. \end{aligned}$$
and the cost functional reads
$$\begin{aligned} J(s,x;u(\cdot ))=\mathbb {E}\int _s^\infty \left\langle \begin{pmatrix}Q&{}S^\top \\ S&{}R\end{pmatrix} \begin{pmatrix}X(t)\\ u(t)\end{pmatrix}, \begin{pmatrix}X(t)\\ u(t)\end{pmatrix}\right\rangle dt. \end{aligned}$$
We denote this problem by Problem (LQ)\(_s^0\). Since the matrices in the state equation and in the cost functional are time-invariant and the time horizon is infinite, Problem (LQ)\(^0\) and Problem (LQ)\(_s^0\) can be regarded as the same problem. Thus, \(U^*(\cdot )X^*(s)\) is the open-loop optimal control of Problem (LQ)\(_s^0\) for the initial distribution \(X^*(s)\). On the other hand, by the dynamic programming principle, \(U^*(s+\cdot )x\) is also an open-loop optimal control of Problem (LQ)\(_s^0\) for the initial distribution \(X^*(s)\). Therefore, one should have
$$\begin{aligned} U^*(s+t)x=U^*(t)X^*(s)\quad \forall s,t\ge 0. \end{aligned}$$
In particular, taking \(t=0\), we have
$$\begin{aligned} u^*(s)=U^*(s)x=U^*(0)X^*(s),\quad s\ge 0, \end{aligned}$$
which should imply the closed-loop solvability of Problem (LQ)\(^0\). The key fact employed here is that, for any \(s\ge 0\), Problem (LQ)\(_s^0\) is the same as Problem (LQ)\(^0\). We call such a property the stationarity of Problem (LQ)\(^0\).

5 The \(L^2\)-Stable Case

In this section, we shall prove Theorem 4.5 for the case \(0\in \mathscr {S}[A,C;B,D]\), that is, [AC] is \(L^2\)-stable. Recall from Proposition 3.3 that in this case,
$$\begin{aligned} \mathcal{U}_{ad}(x)=L_{\mathbb {F}}^2(\mathbb {R}^m),\quad \forall x\in \mathbb {R}^n. \end{aligned}$$
This allows us to represent \(J(x;u(\cdot ))\) as a quadratic functional on the Hilbert space \(L_{\mathbb {F}}^2(\mathbb {R}^m)\).

Proposition 5.1

Suppose that [AC] is \(L^2\)-stable. Then there exist a bounded self-adjoint linear operator \(M_2:L_{\mathbb {F}}^2(\mathbb {R}^m)\rightarrow L_{\mathbb {F}}^2(\mathbb {R}^m)\), a bounded linear operator \(M_1:\mathbb {R}^n\rightarrow L_{\mathbb {F}}^2(\mathbb {R}^m)\), an \(M_0\in \mathbb {S}^n\), and \(\widehat{u}(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m)\), \(\widehat{x}\in \mathbb {R}^n\), \(c\in \mathbb {R}\) such that
$$\begin{aligned} \begin{array}{rll} J(x;u(\cdot )) &{}= &{}\langle M_2u,u\rangle +2\langle M_1x,u\rangle +\langle M_0x,x\rangle +2\langle u,\widehat{u}\rangle +2\langle x,\widehat{x}\rangle +c,\\ J^0(x;u(\cdot )) &{}= &{}\langle M_2u,u\rangle +2\langle M_1x,u\rangle +\langle M_0x,x\rangle ,\quad \forall (x,u(\cdot ))\in {\mathbb {R}}^{n} \times L_{\mathbb {F}}^{2}({\mathbb {R}}^{m}). \end{array} \end{aligned}$$
(5.1)

Proof

It is similar to the finite horizon case. We omit the proof here and refer the interested reader to [25, Chapter 6] for details. \(\square \)

The representation (5.1) of the cost functional has several consequences, which we summarize as follows.

Proposition 5.2

Suppose that [AC] is \(L^2\)-stable. We have the following results:
  1. (i)

    Problem (LQ) is open-loop solvable at x if and only if \(M_2\ge 0\) and \(M_1x+\widehat{u}\in {\mathscr {R}}( M_2)\). In this case, \(u^*(\cdot )\) is an open-loop optimal control for the initial state x if and only if \(M_2u^{*}+M_1x+\widehat{u}=0\).

     
  2. (ii)

    If Problem (LQ) is open-loop solvable, then so is Problem (LQ)\(^0\).

     
  3. (iii)

    If Problem (LQ)\(^0\) is open-loop solvable, then there exists a \(U^*(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^{m\times n})\) such that for any \(x\in \mathbb {R}^n\), \(U^*(\cdot )x\) is an open-loop optimal control for the initial state x.

     

Proof

  1. (i)
    By definition, a \(u^*(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m)\) is an open-loop optimal control for the initial state x if and only if
    $$\begin{aligned} J(x;u^*(\cdot )+\lambda v(\cdot ))-J(x;u^*(\cdot ))\ge 0, \quad \forall v(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m),~\forall \lambda \in \mathbb {R}^n. \end{aligned}$$
    (5.2)
    From (5.1) we have
    $$\begin{aligned}&J(x;u^*(\cdot )+\lambda v(\cdot ))\\&\quad =\langle M_2(u^*+\lambda v),u^*+\lambda v\rangle +2\langle M_1x,u^*+\lambda v\rangle +\langle M_0x,x\rangle \\&\qquad +2\langle u^*+\lambda v,\widehat{u}\rangle +2\langle x,\widehat{x}\rangle +c\\&\quad =\langle M_2u^*,u^*\rangle +2\lambda \langle M_2u^*,v\rangle +\lambda ^2\langle M_2v,v\rangle +2\langle M_1x,u^*\rangle +2\lambda \langle M_1x,v\rangle \\&\qquad +\langle M_0x,x\rangle +2\langle u^*,\widehat{u}\rangle +2\lambda \langle v,\widehat{u}\rangle +2\langle x,\widehat{x}\rangle +c\\&\quad =J(x;u^*(\cdot ))+\lambda ^2\langle M_2v,v\rangle +2\lambda \langle M_2u^*+M_1x+\widehat{u},v\rangle . \end{aligned}$$
    Thus, (5.2) is equivalent to
    $$\begin{aligned} \lambda ^2\langle M_2v,v\rangle +2\lambda \langle M_2u^*+M_1x+\widehat{u},v\rangle \ge 0, \quad \forall v(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m),~\forall \lambda \in \mathbb {R}^n, \end{aligned}$$
    which in turn is equivalent to
    $$\begin{aligned} \langle M_2v,v\rangle \ge 0,\quad \forall v(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m)\quad \text{ and }\quad M_2u^*+M_1x+\widehat{u}=0. \end{aligned}$$
    The conclusions follow readily.
     
  2. (ii)
    If Problem (LQ) is open-loop solvable, then we have by (i): \(M_2\ge 0\), and
    $$\begin{aligned} M_1x+\widehat{u}\in {\mathscr {R}}( M_2),\quad \forall x\in \mathbb {R}^n. \end{aligned}$$
    In particular, by taking \(x=0\), we see that \(\widehat{u}\in {\mathscr {R}}( M_2)\), and hence \(M_1x\in {\mathscr {R}}( M_2)\) for all \(x\in \mathbb {R}^n\). Using (i) again, we obtain the open-loop solvability of Problem (LQ)\(^0\).
     
  3. (iii)

    Let \(e_1,\cdots ,e_n\) be the standard basis for \(\mathbb {R}^n\), and let \(u^*_i(\cdot )\) be an open-loop optimal control for the initial state \(e_i\). Then \(U^*(\cdot )\triangleq (u^*_1(\cdot ),\cdots ,u^*_n(\cdot ))\) has the desired properties.

     
\(\square \)
Let us observe that \(M_2\ge 0\) if and only if
$$\begin{aligned} J^0(0;u(\cdot ))\ge 0,\quad \forall u(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m). \end{aligned}$$
(5.3)
Further, if there exists a constant \(\delta >0\) such that \(M_2\ge \delta I\), or equivalently,
$$\begin{aligned} J^0(0;u(\cdot ))\ge \delta \,\mathbb {E}\int _0^\infty |u(t)|^2dt,\quad \forall u(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m), \end{aligned}$$
(5.4)
then, by Proposition 5.2 (i), Problem (LQ)\(^0\) is uniquely solvable, with the unique optimal control for the initial state x given by
$$\begin{aligned} u_x^*(\cdot )=-M_2^{-1}M_1x. \end{aligned}$$
Note that the value function of Problem (LQ)\(^0\) is now given by
$$\begin{aligned} V^0(x)=\big \langle \left( M_0-M_1^*M_2^{-1}M_1\right) x,x\big \rangle \equiv \langle Px,x\rangle ,\quad x\in \mathbb {R}^n. \end{aligned}$$
(5.5)
The following result shows that P defined in (5.5) is a static stabilizing solution of ARE (4.4).

Theorem 5.3

Suppose that [AC] is \(L^2\)-stable and that (5.4) holds for some \(\delta >0\). Then the matrix P defined in (5.5) solves the ARE
$$\begin{aligned} \left\{ \begin{array}{ll} PA+A^\top P+C^\top PC+Q\\ -\left( PB+C^\top PD+S^\top \right) \left( R+D^\top PD\right) ^{-1}\left( B^\top P+D^\top PC+S\right) =0,\\ R+D^\top PD>0,\end{array}\right. \end{aligned}$$
(5.6)
and
$$\begin{aligned} \Theta \triangleq -\left( R+D^\top PD\right) ^{-1}\left( B^\top P+D^\top PC+S\right) \end{aligned}$$
(5.7)
is a stabilizer of [ACBD]. Moreover, the unique open-loop optimal control of Problem (LQ)\(^0\) for the initial state x is given by
$$\begin{aligned} u_x^*(\cdot )=\Theta X_\Theta (\cdot \,;x), \end{aligned}$$
where \(X_\Theta (\cdot \,;x)\) is the solution to the following closed-loop system:
$$\begin{aligned} \left\{ \begin{array}{ll} dX(t)=(A+B\Theta )X(t)dt+(C+D\Theta )X(t)dW(t),\quad t\ge 0, \\ X(0)= x.\end{array}\right. \end{aligned}$$

Proof

Let \(\Phi (\cdot )\) be the solution of (2.2). Since [AC] is \(L^2\)-stable, the following is well-defined:
$$\begin{aligned} G\triangleq \mathbb {E}\int _0^\infty \Phi (t)^\top Q\Phi (t)dt. \end{aligned}$$
For \(T>0\), let us consider the state equation
$$\begin{aligned} \left\{ \begin{array}{ll} dX_T(t)=\left[ AX_T(t)+Bu(t)\right] dt\\ \qquad \qquad \qquad +\left[ CX_T(t)+Du(t)\right] dW(t),\quad t\in [0,T],\\ X_T(0)=x,\end{array}\right. \end{aligned}$$
(5.8)
and the cost functional
$$\begin{aligned} J_T(x;u(\cdot ))\triangleq \mathbb {E}\left\{ \langle GX_T(T),X_T(T)\rangle +\int _0^T\left\langle \begin{pmatrix}Q&{}S^\top \\ S&{}R\\ \end{pmatrix} \begin{pmatrix}X_T(t)\\ u(t)\end{pmatrix}, \begin{pmatrix}X_T(t)\\ u(t)\end{pmatrix}\right\rangle dt\right\} . \end{aligned}$$
We claim that
$$\begin{aligned} J_T(0;u(\cdot ))\ge \delta \,\mathbb {E}\int _0^T|u(t)|^2dt,\quad \forall u(\cdot )\in L_{\mathbb {F}}^2(0,T;\mathbb {R}^m). \end{aligned}$$
(5.9)
To show this, take any \(u(\cdot )\in L_{\mathbb {F}}^2(0,T;\mathbb {R}^m)\) and let \(X_T(\cdot )\) be the corresponding solution to (5.8) with initial state x. Define the zero-extension of \(u(\cdot )\) as follows:
$$\begin{aligned}{}[u(\cdot )\oplus 0\mathbf{1}_{(T,\infty )}](s)=\left\{ \begin{array}{ll} u(t),\quad &{} t\in [0,T],\\ 0,\quad &{} t\in (T,\infty ). \end{array}\right. \end{aligned}$$
Then \(v(\cdot )\equiv [u(\cdot )\oplus 0\mathbf{1}_{(T,\infty )}](\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m)\), and the solution \(X(\cdot )\) of
$$\begin{aligned} \left\{ \begin{array}{ll} dX(t)=\left[ AX(t)+Bv(t)\right] dt+\left[ CX(t)+Dv(t)\right] dW(t),\quad t\ge 0,\\ X(0)=x, \end{array}\right. \end{aligned}$$
satisfies
$$\begin{aligned} X(t)=\left\{ \begin{array}{ll} X_T(t), \quad &{} t\in [0,T],\\ \Phi (t)\Phi (T)^{-1}X_T(T),\quad &{} t\in (T,\infty ). \end{array}\right. \end{aligned}$$
Note that for \(t\ge T\), \(\Phi (t)\Phi (T)^{-1}\) has the same distribution as \(\Phi (t-T)\) and is independent of \(\mathcal{F}_T\). Thus,
$$\begin{aligned} J_T(x;u(\cdot ))= & {} \mathbb {E}\bigg \{\bigg \langle \bigg (\mathbb {E}\int _0^\infty \Phi (t)^\top Q\Phi (t)dt\bigg )X_T(T),X_T(T)\bigg \rangle \nonumber \\&\quad +\int _0^T\left\langle \begin{pmatrix}Q&{}S^\top \\ S&{}R\end{pmatrix} \begin{pmatrix}X_T(t)\\ u(t)\end{pmatrix}, \begin{pmatrix}X_T(t)\\ u(t)\end{pmatrix}\right\rangle dt\bigg \}\nonumber \\= & {} \mathbb {E}\bigg \{\bigg \langle \bigg (\mathbb {E}\int _T^\infty \Big [\Phi (t)\Phi (T)^{-1}\Big ]^\top Q\Big [\Phi (t)\Phi (T)^{-1}\Big ]dt\bigg ) X_T(T),X_T(T)\bigg \rangle \nonumber \\&\quad +\int _0^T\left\langle \begin{pmatrix}Q&{}S^\top \\ S&{}R\\ \end{pmatrix} \begin{pmatrix}X_T(t)\\ u(t)\end{pmatrix}, \begin{pmatrix}X_T(t)\\ u(t)\end{pmatrix}\right\rangle dt\bigg \}\nonumber \\= & {} \mathbb {E}\bigg \{\int _T^\infty \big \langle Q\Phi (t)\Phi (T)^{-1}X_T(T),\Phi (t)\Phi (T)^{-1}X_T(T)\big \rangle dt\nonumber \\&\quad +\int _0^T\left\langle \begin{pmatrix}Q&{}S^\top \\ S&{}R\\ \end{pmatrix} \begin{pmatrix}X_T(t)\\ u(t)\end{pmatrix}, \begin{pmatrix}X_T(t)\\ u(t)\end{pmatrix}\right\rangle dt\bigg \}\nonumber \\= & {} \mathbb {E}\bigg \{\int _T^\infty \langle QX(t),X(t)\rangle dt +\int _0^T\left\langle \begin{pmatrix}Q&{}S^\top \\ S&{}R\end{pmatrix} \begin{pmatrix}X(t)\\ v(t)\end{pmatrix}, \begin{pmatrix}X(t)\\ v(t)\end{pmatrix}\right\rangle dt\bigg \}\nonumber \\= & {} J^0(x;[u(\cdot )\oplus 0\mathbf{1}_{(T,\infty )}](\cdot )). \end{aligned}$$
(5.10)
In particular, taking \(x=0\), we obtain
$$\begin{aligned} J_T(0;u(\cdot ))= & {} J^0(0;[u(\cdot )\oplus 0\mathbf{1}_{(T,\infty )}](\cdot ))\\\ge & {} \delta \,\mathbb {E}\int _0^\infty \big |[u(\cdot )\oplus 0\mathbf{1}_{(T,\infty )}](t)\big |^2dt=\delta \,\mathbb {E}\int _0^T|u(t)|^2dt. \end{aligned}$$
This proves our claim.
The fact (5.9) allows us to use [18], Theorem 4.6] to conclude that for any \(T>0\), the differential Riccati equation
$$\begin{aligned} \left\{ \begin{array}{ll} \dot{P}(t;T)+P(t;T)A+A^\top P(t;T)+C^\top P(t;T)C+Q\\ \quad -\left[P(t;T)B+C^\top P(t;T)D+S^\top \right]\left[R+D^\top P(t;T)D\right]^{-1}\\ \quad ~\cdot \left[B^\top P(t;T)+D^\top P(t;T)C+S\right]=0,\quad t\in [0,T],\\ P(T;T)=G \end{array}\right. \end{aligned}$$
admits a unique solution \(P(\cdot \,;T)\in C([0,T];\mathbb {S}^n)\) such that
$$\begin{aligned}&R+D^\top P(t;T)D\ge \delta I,\quad \forall t\in [0,T],&\end{aligned}$$
(5.11)
$$\begin{aligned}&\inf _{u(\cdot )\in L_{\mathbb {F}}^2(0,T;\mathbb {R}^m)}J_T(x;u(\cdot ))=\langle P(0;T)x,x\rangle ,\quad \forall x\in \mathbb {R}^n.&\end{aligned}$$
(5.12)
We are going to show that \(\{P(0;T)\}_{T>0}\) converges to P as \(T\rightarrow \infty \). To this end, one observes that (5.10) implies
$$\begin{aligned} \langle Px,x\rangle \le J^0(x;[u(\cdot )\oplus 0\mathbf{1}_{(T,\infty )}](\cdot ))=J_T(x;u(\cdot )), \quad \forall x\in \mathbb {R}^n,~\forall u(\cdot )\in L_{\mathbb {F}}^2(0,T;\mathbb {R}^m). \end{aligned}$$
Taking infimum over \(u(\cdot )\in L_{\mathbb {F}}^2(0,T;\mathbb {R}^m)\), we obtain
$$\begin{aligned} \langle Px,x\rangle \le \langle P(0;T)x,x\rangle ,\quad \forall x\in \mathbb {R}^n,~\forall \, T>0. \end{aligned}$$
(5.13)
On the other hand, for any given \(\varepsilon >0\), one can find a \(u^\varepsilon (\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m)\) such that
$$\begin{aligned} \mathbb {E}\int _0^\infty \left\langle \begin{pmatrix}Q&{}S^\top \\ S&{}R\\ \end{pmatrix} \begin{pmatrix}X^\varepsilon (t)\\ u^\varepsilon (t)\end{pmatrix}, \begin{pmatrix}X^\varepsilon (t)\\ u^\varepsilon (t)\end{pmatrix}\right\rangle dt =J^0(x;u^\varepsilon (\cdot ))\le \langle Px,x\rangle +\varepsilon , \end{aligned}$$
(5.14)
where \(X^\varepsilon (\cdot )\) is the solution of
$$\begin{aligned} \left\{ \begin{array}{ll} dX^\varepsilon (t)=\left[ AX^\varepsilon (t)+Bu^\varepsilon (t)\right] dt+\left[ CX^\varepsilon (t)+Du^\varepsilon (t)\right] dW(t),\quad t\ge 0, \\ X^\varepsilon (0)= x.\end{array}\right. \end{aligned}$$
Since by Lemma 2.3 \(X^\varepsilon (\cdot )\in \mathcal{X}[0,\infty )\), we have for large \(T>0\),
$$\begin{aligned} \left|\mathbb {E}\langle GX^\varepsilon (T),X^\varepsilon (T)\rangle \right|\le \varepsilon ,\quad \left|\mathbb {E}\int _T^\infty \left\langle \begin{pmatrix}Q&{}S^\top \\ S&{}R\end{pmatrix} \begin{pmatrix}X^\varepsilon (t) \\ u^\varepsilon (t)\end{pmatrix}, \begin{pmatrix}X^\varepsilon (t) \\ u^\varepsilon (t)\end{pmatrix}\right\rangle dt\right|\le \varepsilon . \end{aligned}$$
Take \(u^\varepsilon _T(\cdot )=u^\varepsilon (\cdot )|_{[0,T]}\). Then
$$\begin{aligned} J^0(x;u^\varepsilon (\cdot ))= & {} J_T(x;u^\varepsilon _T(\cdot ))-\mathbb {E}\langle GX^\varepsilon (T),X^\varepsilon (T)\rangle \nonumber \\&+\,\mathbb {E}\displaystyle \int _T^\infty \left\langle \begin{pmatrix}Q&{}S^\top \\ S&{}R\\ \end{pmatrix} \begin{pmatrix}X^\varepsilon (t)\\ u^\varepsilon (t)\end{pmatrix}, \begin{pmatrix}X^\varepsilon (t)\\ u^\varepsilon (t)\end{pmatrix}\right\rangle dt\nonumber \\\ge & {} \langle P(0;T)x,x\rangle -2\varepsilon . \end{aligned}$$
(5.15)
Combining (5.14) and (5.15), we see that for large \(T>0\),
$$\begin{aligned} \langle P(0;T)x,x\rangle \le \langle Px,x\rangle +3\varepsilon , \end{aligned}$$
which, together with (5.13), implies that \(P(0;T)\rightarrow P\) as \(T\rightarrow \infty \). Now it follows from (5.11) that \(R+D^\top PD>0\). Thus, by Lemma 2.8, P solves ARE (5.6).
Finally, let \(\Theta \) be as in (5.7), and for any initial state x, let \((X^*_x(\cdot ),u_x^*(\cdot ))\) be the corresponding optimal pair of Problem (LQ)\(^0\). By applying Itô’s formula to \(t\rightarrow \langle PX^*_x(t),X^*_x(t)\rangle \), we have
$$\begin{aligned} \langle Px,x\rangle= & {} J^0(x;u_x^*(\cdot ))\\= & {} \langle Px,x\rangle +\mathbb {E}\int _0^\infty \Big [\big \langle \left( PA+A^\top P+C^\top PC+Q\right) X^*_x(t),X^*_x(t)\big \rangle \\&~+\;2\big \langle \left( B^\top P + D^\top PC + S\right) X^*_x(t),u_x^*(t)\big \rangle + \big \langle \left( R + D^\top PD\right) u_x^*(t),u_x^*(t)\big \rangle \Big ]dt\\= & {} \langle Px,x\rangle +\mathbb {E}\int _0^\infty \big \langle \left( R+D^\top PD\right) \left[ u_x^*(t)-\Theta X^*_x(t)\right] ,u_x^*(t)-\Theta X^*_x(t)\big \rangle dt. \end{aligned}$$
Since \(R+D^\top PD>0\), we must have
$$\begin{aligned} u_x^*(\cdot )=\Theta X^*_x(\cdot ),\quad \forall x\in \mathbb {R}^n, \end{aligned}$$
and hence \(X^*_x(\cdot )\) satisfies
$$\begin{aligned} \left\{ \begin{array}{ll} dX^*_x(t)=(A+B\Theta )X^*_x(t)dt+(C+D\Theta )X^*_x(t)dW(t),\quad t\ge 0, \\ X^*_x(0)= x.\end{array}\right. \end{aligned}$$
Since \(X^*_x(\cdot )\in \mathcal{X}[0,\infty )\) for all \(x\in \mathbb {R}^n\), we conclude that \(\Theta \) is a stabilizer of [ACBD]. The rest of the proof is clear. \(\square \)

We are now ready to prove Theorem 4.5 for the case when [AC] is \(L^2\)-stable.

Proof of Theorem 4.5

under the assumption that [AC] is \(L^2\) -stable. (ii) \(\mathop {\Rightarrow }\) (i) is obvious.

(i) \(\mathop {\Rightarrow }\) (iii): First, by Proposition 5.2, \(M_2\ge 0\) and Problem (LQ)\(^0\) is also open-loop solvable. For any \(\varepsilon >0\), let us consider the state equation
$$\begin{aligned} \left\{ \begin{array}{ll} dX(t)=\left[ AX(t)+Bu(t)\right] dt+\left[ CX(t)+Du(t)\right] dW(t),\quad t\ge 0, \\ X(0)= x,\end{array}\right. \end{aligned}$$
and the cost functional
$$\begin{aligned} J^0_\varepsilon (x; u(\cdot ))= & {} J^0(x; u(\cdot ))+\varepsilon \mathbb {E}\int _0^\infty |u(t)|^2dt\\= & {} \mathbb {E}\int _0^\infty \left\langle \begin{pmatrix}Q&{}S^\top \\ S&{}R+\varepsilon I\end{pmatrix} \begin{pmatrix}X(t)\\ u(t)\end{pmatrix}, \begin{pmatrix}X(t)\\ u(t)\end{pmatrix}\right\rangle dt. \end{aligned}$$
Denote by Problem (LQ)\(_\varepsilon ^0\) the above problem, and by \(V_\varepsilon ^0(\cdot )\) the corresponding value function. Since
$$\begin{aligned} J^0_\varepsilon (0; u(\cdot ))=\langle (M_2+\varepsilon I)u,u\rangle \ge \varepsilon \mathbb {E}\int _0^\infty |u(t)|^2dt, \quad \forall u(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m), \end{aligned}$$
by Theorem 5.3, the following ARE
$$\begin{aligned} \left\{ \begin{array}{ll} P_\varepsilon A+A^\top P_\varepsilon +C^\top P_\varepsilon C+Q~-\left( P_\varepsilon B+C^\top P_\varepsilon D+S^\top \right) \\ \quad \times \left( R+\varepsilon I+D^\top P_\varepsilon D\right) ^{-1} \left( B^\top P_\varepsilon +D^\top P_\varepsilon C+S\right) =0,\\ R+\varepsilon I+D^\top P_\varepsilon D>0, \end{array}\right. \end{aligned}$$
(5.16)
admits a unique solution \(P_\varepsilon \in \mathbb {S}^n\) such that \(V_\varepsilon ^0(x)=\langle P_\varepsilon x,x\rangle \) for all \(x\in \mathbb {R}^n\). Moreover,
$$\begin{aligned} \Theta _\varepsilon \triangleq -\left( R+\varepsilon I+D^\top P_\varepsilon D\right) ^{-1}\left( B^\top P_\varepsilon +D^\top P_\varepsilon C+S\right) \end{aligned}$$
(5.17)
is a stabilizer of [ACBD], and the unique open-loop optimal control \(u^*_\varepsilon (\cdot \,;x)\) of Problem (LQ)\(_\varepsilon ^0\) for the initial state x is given by
$$\begin{aligned} u^*_\varepsilon (t;x)=\Theta _\varepsilon \Psi _\varepsilon (t)x,\quad t\ge 0, \end{aligned}$$
where \(\Psi _\varepsilon (\cdot )\) is the solution to the following SDE for \(\mathbb {R}^{n\times n}\)-valued processes:
$$\begin{aligned} \left\{ \begin{array}{ll} d\Psi _\varepsilon (t)=(A+B\Theta _\varepsilon )\Psi _\varepsilon (t)dt+(C+D\Theta _\varepsilon )\Psi _\varepsilon (t)dW(t),\quad t\ge 0, \\ \Psi _\varepsilon (0)=I.\end{array}\right. \end{aligned}$$
Now let \(U^*(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^{m\times n})\) be a process with the property in Proposition 5.2 (iii). By the definition of value function, we have for any \(x\in \mathbb {R}^n\) and \(\varepsilon >0\),
$$\begin{aligned}&V^0(x)+\varepsilon \mathbb {E}\int _0^\infty |\Theta _\varepsilon \Psi _\varepsilon (t)x|^2dt\nonumber \\&\quad \le J^0(x;\Theta _\varepsilon \Psi _\varepsilon (\cdot )x)+\varepsilon \mathbb {E}\int _0^\infty |\Theta _\varepsilon \Psi _\varepsilon (t)x|^2dt\nonumber \\&\quad = J_\varepsilon ^0(x;\Theta _\varepsilon \Psi _\varepsilon (\cdot )x)=V^0_\varepsilon (x)\le J_\varepsilon ^0(x;U^*(\cdot )x)\nonumber \\&\quad = V^0(x)+\varepsilon \mathbb {E}\int _0^\infty |U^*(t)x|^2dt, \end{aligned}$$
(5.18)
which implies the following:
$$\begin{aligned} V^0(x)\le & {} V^0_\varepsilon (x)=\langle P_\varepsilon x,x\rangle \le V^0(x)+\varepsilon \mathbb {E}\int _0^\infty |U^*(t)x|^2dt, \forall x\in \mathbb {R}^n,~\forall \varepsilon >0, \nonumber \\\end{aligned}$$
(5.19)
$$\begin{aligned} 0\le & {} \mathbb {E}\int _0^\infty \Psi _\varepsilon (t)^\top \Theta _\varepsilon ^\top \Theta _\varepsilon \Psi _\varepsilon (t)dt \le \mathbb {E}\int _0^\infty U^*(t)^\top U^*(t)dt,\quad \forall \varepsilon >0.\nonumber \\ \end{aligned}$$
(5.20)
From (5.19) we see that \(P\equiv \lim _{\varepsilon \rightarrow 0}P_\varepsilon \) exists and \(V^0(x)=\langle Px,x\rangle \) for all \(x\in \mathbb {R}^n\). Denote
$$\begin{aligned} \Pi _\varepsilon =\mathbb {E}\int _0^\infty \Psi _\varepsilon (t)^\top \Theta _\varepsilon ^\top \Theta _\varepsilon \Psi _\varepsilon (t)dt. \end{aligned}$$
It follows from (5.20) that \(\{\Pi _\varepsilon \}_{\varepsilon >0}\) is bounded. Moreover, noting that \([A+B\Theta _\varepsilon ,C+D\Theta _\varepsilon ] \) is \(L^2\)-stable, we have by Lemma 2.2 that
$$\begin{aligned} \Pi _\varepsilon (A+B\Theta _\varepsilon )+(A+B\Theta _\varepsilon )^\top \Pi _\varepsilon +(C+D\Theta _\varepsilon )^\top \Pi _\varepsilon (C+D\Theta _\varepsilon )+\Theta _\varepsilon ^\top \Theta _\varepsilon =0. \end{aligned}$$
Thus,
$$\begin{aligned} 0\le \Theta _\varepsilon ^\top \Theta _\varepsilon= & {} -\,\left[ \Pi _\varepsilon (A+B\Theta _\varepsilon )+(A+B\Theta _\varepsilon )^\top \Pi _\varepsilon +(C+D\Theta _\varepsilon )^\top \Pi _\varepsilon (C+D\Theta _\varepsilon )\right] \\\le & {} -\,\left[ \Pi _\varepsilon (A+B\Theta _\varepsilon )+(A+B\Theta _\varepsilon )^\top \Pi _\varepsilon \right] ,\quad \forall \varepsilon >0. \end{aligned}$$
The above, together with the boundedness of \(\{\Pi _\varepsilon \}_{\varepsilon >0}\), shows that
$$\begin{aligned} | \Theta _\varepsilon |^2\le K(1+|\Theta _\varepsilon |),\quad \forall \varepsilon >0, \end{aligned}$$
(5.21)
for some constant \(K>0\). Noting that (5.21) implies the boundedness of \(\{\Theta _\varepsilon \}_{\varepsilon >0}\), we may choose a sequence \(\{\varepsilon _k\}^\infty _{k=1}\subseteq (0,\infty )\) with \(\lim _{k\rightarrow \infty }\varepsilon _k=0\) such that \(\Theta \equiv \lim _{k\rightarrow \infty }\Theta _{\varepsilon _k}\) exists. Then
$$\begin{aligned} \left( R+D^\top P D\right) \Theta= & {} \lim _{k\rightarrow \infty }\left( R+\varepsilon _k I+D^\top P_{\varepsilon _k} D\right) \Theta _{\varepsilon _k}\nonumber \\= & {} -\left( B^\top P+D^\top PC+S\right) . \end{aligned}$$
(5.22)
It follows from Lemma 2.6 that
$$\begin{aligned} \left\{ \begin{array}{ll} {\mathscr {R}}\left( B^\top P+D^\top PC+S\right) \subseteq {\mathscr {R}}\left( R+D^\top PD\right) ,\\ \Theta =-\mathcal{N}(P)^\dag \mathcal{L}(P)^\top +\left[ I-\mathcal{N}(P)^\dag \mathcal{N}(P)\right] \Pi \quad \text{ for } \text{ some }~\Pi \in \mathbb {R}^{m\times n}. \end{array}\right. \end{aligned}$$
(5.23)
Note that by (5.17),
$$\begin{aligned} \left( P_\varepsilon B+C^\top P_\varepsilon D+S^\top \right) =-\Theta _\varepsilon ^\top \left( R+\varepsilon I+D^\top P_\varepsilon D\right) . \end{aligned}$$
Thus (5.16) can be written as
$$\begin{aligned} \left\{ \begin{array}{ll} P_\varepsilon A+A^\top P_\varepsilon +C^\top P_\varepsilon C+Q-\Theta _\varepsilon ^\top \left( R+\varepsilon I+D^\top P_\varepsilon D\right) \Theta _\varepsilon =0,\\ R+\varepsilon I+D^\top P_\varepsilon D>0. \end{array}\right. \end{aligned}$$
Now a passage to the limit along \(\{\varepsilon _k\}^\infty _{k=1}\) in the above yields
$$\begin{aligned} \left\{ \begin{array}{ll} PA+A^\top P+C^\top PC+Q-\Theta ^\top \left( R+D^\top PD\right) \Theta =0,\\ R+D^\top PD\ge 0, \end{array}\right. \end{aligned}$$
(5.24)
which, together with (5.23), implies that P solves ARE (4.4). To see that P is a static stabilizing solution, we need only show that \(\Theta \in \mathscr {S}[A,C;B,D]\). For this, let \(\Psi (\cdot )\) be the solution of
$$\begin{aligned} \left\{ \begin{array}{ll} d\Psi (t)=(A+B\Theta )\Psi (t)dt+(C+D\Theta )\Psi (t)dW(t),\quad t\ge 0, \\ \Psi (0)=I.\end{array}\right. \end{aligned}$$
Since \(\Theta _{\varepsilon _k}\rightarrow \Theta \) as \(k\rightarrow \infty \), we have \(\Psi _{\varepsilon _k}(t)\rightarrow \Psi (t),~\mathrm{a.s.}\) for all \(t\ge 0\). Using Fatou’s lemma and (5.18), we obtain
$$\begin{aligned} \mathbb {E}\int _0^\infty |\Theta \Psi (t)x|^2dt\le & {} \liminf _{k\rightarrow \infty }\mathbb {E}\int _0^\infty |\Theta _{\varepsilon _k}\Psi _{\varepsilon _k}(t)x|^2dt\\\le & {} \mathbb {E}\int _0^\infty |U^*(t)x|^2dt<\infty ,\quad \forall x\in \mathbb {R}^n. \end{aligned}$$
This implies \(\Theta \Psi (\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^{m\times n})\), and thus by Lemma 2.3 \(\Psi (\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^{n\times n})\). Therefore, \(\Theta \) is a stabilizer of [ACBD].
Next, consider the following BSDE on \([0,\infty )\):
$$\begin{aligned} d\eta (t)= & {} -\left[ (A+B\Theta )^\top \eta +(C+D\Theta )^\top \zeta +(C+D\Theta )^\top P\sigma \right. \nonumber \\&\quad \quad \left. +\Theta ^\top \rho +Pb+q\right] dt+\zeta dW(t). \end{aligned}$$
(5.25)
Since \([A+B\Theta ,C+D\Theta ]\) is \(L^2\)-stable, it follows form Lemma 2.5 that (5.25) admits a unique \(L^2\)-stable adapted solution \((\eta (\cdot ),\zeta (\cdot ))\). For any initial state x and any control process \(u(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m)\), let \(X(\cdot )\equiv X(\cdot \,;x,u(\cdot ))\) be the corresponding solution of (1.1). Applying Itô’s formula to \(t\mapsto \langle PX(t),X(t)\rangle \), we have
$$\begin{aligned} -\langle Px,x\rangle= & {} \mathbb {E}\displaystyle \int _0^\infty \Big [\big \langle \big (PA+A^\top P+C^\top PC\big )X,X\big \rangle \nonumber \\&+\,2\big \langle \big (B^\top P+D^\top PC\big )X,u\big \rangle +\big \langle D^\top PDu,u\big \rangle \nonumber \\&+\, 2\big \langle C^\top P\sigma +Pb,X\big \rangle +2\big \langle D^\top P\sigma ,u\big \rangle +\langle P\sigma ,\sigma \rangle \Big ]dt. \end{aligned}$$
(5.26)
Applying Itô’s formula to \(t\mapsto \langle \eta (t),X(t)\rangle \), we have
$$\begin{aligned} \mathbb {E}\langle \eta (0),x\rangle= & {} \mathbb {E}\displaystyle \int _0^\infty \Big [\big \langle \Theta ^\top \big (B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \big ), X\big \rangle +\, \big \langle C^\top P\sigma +Pb+q,X\big \rangle \nonumber \\&\qquad \quad \qquad -\big \langle B^\top \eta +D^\top \zeta ,u\big \rangle -\langle \eta ,b\rangle -\langle \zeta ,\sigma \rangle \Big ]dt. \end{aligned}$$
(5.27)
Then using (5.22) and (5.24), we get
$$\begin{aligned}&J(x;u(\cdot ))-\langle Px,x\rangle -2\mathbb {E}\langle \eta (0),x\rangle \nonumber \\&\quad =\mathbb {E}\int _0^\infty \Big [\big \langle \big (R+D^\top PD\big )(u-\Theta X),(u-\Theta X)\big \rangle \nonumber \\&\qquad \quad \qquad \quad \quad +2\big \langle B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho ,u-\Theta X\big \rangle \nonumber \\&\quad \qquad \qquad \quad \quad +\langle P\sigma ,\sigma \rangle +2\langle \eta ,b\rangle +2\langle \zeta ,\sigma \rangle \Big ]dt. \end{aligned}$$
(5.28)
Let \(u^*(\cdot )\) be an open-loop optimal control of Problem (LQ) for the initial state x, and denote by \(X_\Theta (\cdot \,;x,v(\cdot ))\) the solution to the following SDE:
$$\begin{aligned} \left\{ \begin{array}{ll} dX_\Theta (t)=\left[ (A+B\Theta )X_\Theta (t)+Bv(t)+b(t)\right] dt\\ \qquad \qquad \qquad +\left[ (C+D\Theta )X_\Theta (t)+Dv(t)+\sigma (t)\right] dW(t),\quad t\ge 0, \\ X_\Theta (0)= x.\end{array}\right. \end{aligned}$$
By Proposition 3.3,
$$\begin{aligned} u^*(\cdot )=\Theta X_\Theta (\cdot \,;x,v^*(\cdot ))+v^*(\cdot ), \end{aligned}$$
for some \(v^*(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m)\), and hence
$$\begin{aligned}&J(x;\Theta X_\Theta (\cdot \,;x,v^*(\cdot ))+v^*(\cdot ))\nonumber \\&\quad = J(x;u^*(\cdot )) \le J(x;\Theta X_\Theta (\cdot \,;x,v(\cdot ))+v(\cdot )), \quad \forall v(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m). \end{aligned}$$
(5.29)
Taking \(u(\cdot )=\Theta X_\Theta (\cdot \,;x,v(\cdot ))+v(\cdot )\) and noting that
$$\begin{aligned} X(\cdot \,;x,u^*(\cdot ))=X_\Theta (\cdot \,;x,v^*(\cdot )),\quad X(\cdot \,;x,u(\cdot ))=X_\Theta (\cdot \,;x,v(\cdot )), \end{aligned}$$
we have from (5.28) and (5.29) that for any \(v(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m)\),
$$\begin{aligned} \begin{array}{lll} \langle Px,x\rangle +2\mathbb {E}\langle \eta (0),x\rangle +\mathbb {E}\displaystyle \int _0^\infty \Big [\langle P\sigma ,\sigma \rangle +2\langle \eta ,b\rangle +2\langle \zeta ,\sigma \rangle \Big ]dt\\ \qquad +\mathbb {E}\displaystyle \int _0^\infty \Big [\big \langle \left( R+D^\top PD\right) v^*,v^*\big \rangle +2\big \langle B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho ,v^*\big \rangle \Big ]dt\\ \quad =J(x;u^*(\cdot ))\le J(x;u(\cdot ))\\ \quad \le \langle Px,x\rangle +2\mathbb {E}\langle \eta (0),x\rangle +\mathbb {E}\displaystyle \int _0^\infty \Big [\langle P\sigma ,\sigma \rangle +2\langle \eta ,b\rangle +2\langle \zeta ,\sigma \rangle \Big ]dt\\ \qquad +\mathbb {E}\displaystyle \int _0^\infty \Big [\big \langle \left( R+D^\top PD\right) v,v\big \rangle +2\big \langle B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho ,v\big \rangle \Big ]dt, \end{array} \end{aligned}$$
(5.30)
which shows that \(v^*(\cdot )\) is a minimizer of the functional
$$\begin{aligned}&F(v(\cdot ))\triangleq \mathbb {E}\int _0^\infty \Big [\big \langle \left( R+D^\top PD\right) v,v\big \rangle +2\big \langle B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho ,v\big \rangle \Big ]dt,\quad ~\\&\quad v(\cdot )\in L_{\mathbb {F}}^2(\mathbb {R}^m). \end{aligned}$$
Therefore, we must have
$$\begin{aligned} \left( R+D^\top PD\right) v^*+B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho =0. \end{aligned}$$
It follows from Lemma 2.6 that
$$\begin{aligned} \left\{ \begin{array}{ll} B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \in {\mathscr {R}}\left( R+D^\top PD\right) ,\\ v^*=-\mathcal{N}(P)^\dag \left( B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \right) \\ \qquad \quad +\left[ I-\mathcal{N}(P)^\dag \mathcal{N}(P)\right] \nu \quad \text{ for } \text{ some }~\nu \in L_{\mathbb {F}}^2(\mathbb {R}^m). \end{array}\right. \end{aligned}$$
(5.31)
Recall (5.23) and observe that \(\left[ \Theta ^\top +\mathcal{L}(P)\mathcal{N}(P)^\dag \right] \left( B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \right) =0\). Thus,
$$\begin{aligned}&(A+B\Theta )^\top \eta +(C+D\Theta )^\top \zeta +(C+D\Theta )^\top P\sigma +\Theta ^\top \rho +Pb+q\\&\quad =A^\top \eta +C^\top \zeta +C^\top P\sigma +Pb+q +\Theta ^\top \left( B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \right) \\&\quad =A^\top \eta +C^\top \zeta +C^\top P\sigma +Pb+q -\mathcal{L}(P)\mathcal{N}(P)^\dag \left( B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \right) \\&\quad =\left[ A^\top -\mathcal{L}(P)\mathcal{N}(P)^\dag B^\top \right] \eta +\left[ C^\top -\mathcal{L}(P)\mathcal{N}(P)^\dag D^\top \right] \zeta \\&\qquad +\;\left[ C^\top -\mathcal{L}(P)\mathcal{N}(P)^\dag D^\top \right] P\sigma -\mathcal{L}(P)\mathcal{N}(P)^\dag \rho +Pb+q. \end{aligned}$$
We see then \((\eta (\cdot ),\zeta (\cdot ))\) is an \(L^2\)-stable adapted solution of (4.5). Further, combining (5.30) and (5.31), we have
$$\begin{aligned} V(x)= & {} J(x;u^*(\cdot ))\\= & {} \langle Px,x\rangle +2\mathbb {E}\langle \eta (0),x\rangle +\mathbb {E}\int _0^\infty \Big [\langle P\sigma ,\sigma \rangle +2\langle \eta ,b\rangle +2\langle \zeta ,\sigma \rangle \Big ]dt\\&+\,\mathbb {E}\int _0^\infty \Big [\big \langle \left( R+D^\top PD\right) v^*,v^*\big \rangle +2\big \langle B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho ,v^*\big \rangle \Big ]dt\\= & {} \langle Px,x\rangle +2\mathbb {E}\langle \eta (0),x\rangle +\mathbb {E}\int _0^\infty \Big [\langle P\sigma ,\sigma \rangle +2\langle \eta ,b\rangle +2\langle \zeta ,\sigma \rangle \Big ]dt\\&-\,\mathbb {E}\int _0^\infty \big \langle \mathcal{N}(P)^\dag \left( B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \right) , B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \big \rangle dt. \end{aligned}$$
(iii) \(\mathop {\Rightarrow }\) (ii): We take any \((x,u(\cdot ))\in \mathbb {R}^n\times L_{\mathbb {F}}^2(\mathbb {R}^m)\), and let \(X(\cdot )\equiv X(\cdot \,;x,u(\cdot ))\) be the corresponding state process. Proceeding by analogy with (5.26)–(5.28), we obtain
$$\begin{aligned} J(x;u(\cdot ))&= \langle Px,x\rangle +2\mathbb {E}\langle \eta (0),x\rangle +\mathbb {E}\int _0^\infty \Big [\langle P\sigma ,\sigma \rangle +2\langle \eta ,b\rangle +2\langle \zeta ,\sigma \rangle \Big ]dt\\&\quad +\,\mathbb {E}\int _0^\infty \Big [\big \langle \left( PA+A^\top P+C^\top PC+Q\right) X,X\big \rangle \\&\quad +2\big \langle \left( B^\top P+D^\top PC+S\right) X,u\big \rangle +\big \langle \left( R+D^\top PD\right) u,u\big \rangle \\&\quad +2\big \langle B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho ,u+\mathcal{N}(P)^\dag \mathcal{L}(P)^\top X\big \rangle \Big ]dt. \end{aligned}$$
Let \((\Theta ^*,v^*(\cdot ))\) be defined by (4.7). We have
$$\begin{aligned} \left\rbrace \begin{array}{ll} B^\top P+D^\top PC+S=-(R+D^\top PD)\Theta ^*=-\mathcal{N}(P)\Theta ^*,\\ B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho =-(R+D^\top PD)v^*=-\mathcal{N}(P)v^*,\\ \mathcal{N}(P)\mathcal{N}(P)^\dag \mathcal{L}(P)^\top =-\mathcal{N}(P)\Theta ^*. \end{array}\right. \end{aligned}$$
Thus,
$$\begin{aligned} \begin{array}{lll} J(x;u(\cdot )) &{}= &{}\langle Px,x\rangle +2\mathbb {E}\langle \eta (0),x\rangle +\mathbb {E}\displaystyle \int _0^\infty \Big [\langle P\sigma ,\sigma \rangle +2\langle \eta ,b\rangle +2\langle \zeta ,\sigma \rangle \Big ]dt\\ &{}~ &{}+\;\mathbb {E}\displaystyle \int _0^\infty \Big [\big \langle (\Theta ^*)^\top \mathcal{N}(P)\Theta ^*X,X\big \rangle -2\langle \mathcal{N}(P)\Theta ^*X,u\rangle +\langle \mathcal{N}(P)u,u\rangle \\ &{}~ &{} -\;2\big \langle \mathcal{N}(P)v^*,u+\mathcal{N}(P)^\dag \mathcal{L}(P)^\top X\big \rangle \Big ]dt\\ &{}= &{}\langle Px,x\rangle +2\mathbb {E}\langle \eta (0),x\rangle +\mathbb {E}\displaystyle \int _0^\infty \Big [\langle P\sigma ,\sigma \rangle +2\langle \eta ,b\rangle +2\langle \zeta ,\sigma \rangle \Big ]dt\\ &{}~ &{}+\;\mathbb {E}\displaystyle \int _0^\infty \Big [\langle \mathcal{N}(P)(u-\Theta ^*X),u-\Theta ^*X\rangle -2\langle v^*,\mathcal{N}(P)u-\mathcal{N}(P)\Theta ^*X\rangle \Big ]dt\\ &{}= &{}\langle Px,x\rangle +2\mathbb {E}\langle \eta (0),x\rangle +\mathbb {E}\displaystyle \int _0^\infty \Big [\langle P\sigma ,\sigma \rangle +2\langle \eta ,b\rangle +2\langle \zeta ,\sigma \rangle -\langle \mathcal{N}(P)v^*,v^*\rangle \Big ]dt\\ &{}~ &{}+\;\mathbb {E}\displaystyle \int _0^\infty \langle \mathcal{N}(P)(u-\Theta ^*X-v^*),u-\Theta ^*X-v^*\rangle dt\\ &{}= &{}\langle Px,x\rangle +2\mathbb {E}\langle \eta (0),x\rangle +\mathbb {E}\displaystyle \int _0^\infty \Big [\langle P\sigma ,\sigma \rangle +2\langle \eta ,b\rangle +2\langle \zeta ,\sigma \rangle \\ &{}~ &{}-\;\big \langle \mathcal{N}(P)^\dag \left( B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \right) , B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \big \rangle \Big ]dt\\ &{}~ &{}+\;\mathbb {E}\displaystyle \int _0^\infty \langle \mathcal{N}(P)(u-\Theta ^*X-v^*),u-\Theta ^*X-v^*\rangle dt. \end{array} \end{aligned}$$
(5.32)
Since \(\mathcal{N}(P)\ge 0\) and \(\Theta ^*\) is a stabilizer of [ACBD], we see that
$$\begin{aligned} J(x;u(\cdot ))\ge & {} \langle Px,x\rangle +2\mathbb {E}\langle \eta (0),x\rangle +\mathbb {E}\int _0^\infty \Big [\langle P\sigma ,\sigma \rangle +2\langle \eta ,b\rangle +2\langle \zeta ,\sigma \rangle \\&-\;\big \langle \mathcal{N}(P)^\dag \left( B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \right) , B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \big \rangle \Big ]dt\\= & {} J(x;\Theta ^*X^*(\cdot )+v^*(\cdot )),\quad \forall x\in \mathbb {R}^n,~\forall u(\cdot )\in L^2_{\mathbb {F}}(\mathbb {R}^m). \end{aligned}$$
That is, \((\Theta ^*,v^*(\cdot ))\) is a closed-loop optimal strategy of Problem (LQ).
Finally, if \((\Theta ,v(\cdot ))\) is a closed-loop optimal strategy, then with \(X(\cdot )\) denoting the solution of
$$\begin{aligned} \left\{ \begin{array}{ll} dX(t)=\left[ (A+B\Theta )X(t)+Bv(t)+b(t)\right] dt\\ \qquad \qquad +\left[ (C+D\Theta )X(t)+Dv(t)+\sigma (t)\right] dW(t),\quad t\ge 0,\\ X(0)= x,\end{array}\right. \end{aligned}$$
and \(u(\cdot )\equiv \Theta X(\cdot )+v(\cdot )\) denoting the outcome of \((\Theta ,v(\cdot ))\), (5.32) implies that
$$\begin{aligned} V(x)&= J(x;u(\cdot ))\\&= \langle Px,x\rangle +2\mathbb {E}\langle \eta (0),x\rangle +\mathbb {E}\int _0^\infty \Big [\langle P\sigma ,\sigma \rangle +2\langle \eta ,b\rangle +2\langle \zeta ,\sigma \rangle \\&\quad -\;\big \langle \mathcal{N}(P)^\dag \left( B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \right) , B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \big \rangle \Big ]dt\\&\quad +\;\mathbb {E}\int _0^\infty \langle \mathcal{N}(P)(u-\Theta ^*X-v^*),u-\Theta ^*X-v^*\rangle dt\\&= V(x)+\mathbb {E}\int _0^\infty \langle \mathcal{N}(P)(\Theta X+v-\Theta ^*X-v^*),\Theta X+v-\Theta ^*X-v^*\rangle dt. \end{aligned}$$
Thus,
$$\begin{aligned}&\mathbb {E}\int _0^\infty \big |\mathcal{N}(P)^{1\over 2}(\Theta X+v-\Theta ^*X-v^*)\big |^2 dt\\&\quad =\mathbb {E}\int _0^\infty \langle \mathcal{N}(P)(\Theta X+v-\Theta ^*X-v^*),\Theta X+v-\Theta ^*X-v^*\rangle dt=0,\quad \forall x\in \mathbb {R}^n. \end{aligned}$$
It follows that \(\mathcal{N}(P)^{1\over 2}(\Theta X+v-\Theta ^*X-v^*)=0\) for all \(x\in \mathbb {R}^n\), and hence
$$\begin{aligned} \mathcal{N}(P)(\Theta -\Theta ^*)X+\mathcal{N}(P)(v-v^*)=0,\quad \forall x\in \mathbb {R}^n. \end{aligned}$$
Since the above holds for each \(x\in \mathbb {R}^n\), and \(\Theta \), \(\Theta ^*\), \(v(\cdot )\), and \(v^*(\cdot )\) are independent of x, by subtracting solutions corresponding x and 0, the latter from the former, we see that for any \(x\in \mathbb {R}^n\), the solution \(X_0(\cdot )\) of
$$\begin{aligned} \left\{ \begin{array}{ll} dX_0(t)=(A+B\Theta )X_0(t)dt+(C+D\Theta )X_0(t)dW(t),\quad t\ge 0,\\ X_0(0)=x,\end{array}\right. \end{aligned}$$
satisfies \(\mathcal{N}(P)(\Theta -\Theta ^*)X_0=0\), from which we conclude that \(\mathcal{N}(P)(\Theta -\Theta ^*)=0\) and \(\mathcal{N}(P)(v-v^*)=0\). Hence,
$$\begin{aligned} \begin{array}{ll} \mathcal{N}(P)\Theta =\mathcal{N}(P)\Theta ^*=-\mathcal{L}(P)^\top ,\\ \mathcal{N}(P)v=\mathcal{N}(P)v^*=-\left( B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \right) .\end{array} \end{aligned}$$
It follows from Lemma 2.6 that \((\Theta ,v(\cdot ))\) is of the form (4.7). Similarly, if \(u^*(\cdot )\) is an open-loop optimal control for the initial state x, then with \(X^*(\cdot )\) denoting the corresponding optimal state process, we have
$$\begin{aligned} \mathcal{N}(P)(u^*-\Theta ^*X^*-v^*)=0, \end{aligned}$$
or equivalently,
$$\begin{aligned} \mathcal{N}(P)u^*=\mathcal{N}(P)\Theta ^*X^*+\mathcal{N}(P)v^*=-\mathcal{L}(P)^\top X^*-\left( B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \right) . \end{aligned}$$
By Lemma 2.6, there exists a \(\nu (\cdot )\in L^2_{\mathbb {F}}(\mathbb {R}^m)\) such that
$$\begin{aligned} u^*= & {} -\,\mathcal{N}(P)^\dag \mathcal{L}(P)^\top X^*-\mathcal{N}(P)^\dag \left( B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \right) \\&+\left[ I-\mathcal{N}(P)^\dag \mathcal{N}(P)\right] \nu \\= & {} \left\rbrace -\mathcal{N}(P)^\dag \mathcal{L}(P)^\top +\left[ I-\mathcal{N}(P)^\dag \mathcal{N}(P)\right] \Pi \right\lbrace X^*\\&-\;\mathcal{N}(P)^\dag \left( B^\top \eta +D^\top \zeta +D^\top P\sigma +\rho \right) +\left[ I-\mathcal{N}(P)^\dag \mathcal{N}(P)\right] (\nu -\Pi X^*). \end{aligned}$$
In the above, \(\Pi \in \mathbb {R}^{m\times n}\) is chosen such that \(-\mathcal{N}(P)^\dag \mathcal{L}(P)^\top +\left[ I-\mathcal{N}(P)^\dag \mathcal{N}(P)\right] \Pi \in \mathscr {S}[A,C;B,D]\). This shows that \(u^*(\cdot )\) has the closed-loop representation (4.8). \(\square \)

6 The Proof of Theorem 4.5: The General Case

We turn to the proof of Theorem 4.5 for the general case \(\mathscr {S}[A,C;B,D]\ne \emptyset \). The idea is to apply Proposition 3.3, thus converting Problem (LQ) into an equivalent one, in which the corresponding uncontrolled system is \(L^2\)-stable.

More precisely, take any \(\Sigma \in \mathscr {S}[A,C;B,D]\), and consider the state equation
$$\begin{aligned} \left\{ \begin{array}{ll} d\widetilde{X}(t)=\left[ \widetilde{A}\widetilde{X}(t)+Bv(t)+b(t)\right] dt\\ \qquad \quad +\left[ \widetilde{C}\widetilde{X}(t)+Dv(t)+\sigma (t)\right] dW(t),\quad t\ge 0,\\ \widetilde{X}(0)= x,\end{array}\right. \end{aligned}$$
(6.1)
and the cost functional
$$\begin{aligned} \widetilde{J}(x;v(\cdot ))\triangleq & {} J(x;\Sigma \widetilde{X}(\cdot )+v(\cdot ))\nonumber \\= & {} \mathbb {E}\displaystyle \int \nolimits _0^\infty \Bigg [\left\langle \begin{pmatrix}Q&{}S^\top \\ S&{}R\\ \end{pmatrix} \begin{pmatrix}\widetilde{X}(t)\\ \Sigma \widetilde{X}(t)+v(t)\end{pmatrix}, \begin{pmatrix}\widetilde{X}(t)\\ \Sigma \widetilde{X}(t)+v(t)\end{pmatrix}\right\rangle \nonumber \\&\qquad +\;2\left\langle \begin{pmatrix}q(t)\\ \rho (t)\end{pmatrix}, \begin{pmatrix}\widetilde{X}(t)\\ \Sigma \widetilde{X}(t)+v(t)\end{pmatrix}\right\rangle \Bigg ]dt\nonumber \\ \displaystyle= & {} \mathbb {E}\int \nolimits _0^\infty \Bigg [\left\langle \begin{pmatrix}\widetilde{Q}&{}\widetilde{S}^\top \\ \widetilde{S}&{}R\\ \end{pmatrix} \begin{pmatrix}\widetilde{X}(t)\\ v(t)\end{pmatrix}, \begin{pmatrix}\widetilde{X}(t)\\ v(t)\end{pmatrix}\right\rangle \nonumber \\&\qquad +2\left\langle \begin{pmatrix}\widetilde{q}(t)\\ \rho (t)\end{pmatrix}, \begin{pmatrix}\widetilde{X}(t)\\ v(t)\end{pmatrix}\right\rangle \Bigg ]dt, \end{aligned}$$
(6.2)
where
$$\begin{aligned} \left\{ \begin{array}{llr} \widetilde{A} = A+B\Sigma , \quad \widetilde{C}=C+D\Sigma ,\\ \widetilde{Q} = Q+S^\top \Sigma +\Sigma ^\top S+\Sigma ^\top R\Sigma ,\\ \widetilde{S} = S+R\Sigma , \quad ~\widetilde{q}=q+\Sigma ^\top \rho . \end{array}\right. \end{aligned}$$
Note that the system \([\widetilde{A},\widetilde{C}]\) is \(L^2\)-stable. We denote by \(\widetilde{X}(\cdot \,;x,v(\cdot ))\) the solution of (6.1) corresponding to x and \(v(\cdot )\), and by Problem (\(\widetilde{\text{ LQ }}\)) the above problem. The following lists several basic facts about Problem (\(\widetilde{\text{ LQ }}\)), whose proofs are straightforward consequences of Proposition 3.3.
  1. (i)

    Problem (\(\widetilde{\text{ LQ }}\)) is open-loop solvable at \(x\in \mathbb {R}^n\) if and only if Problem (LQ) is. In this case, \(v^*(\cdot )\) is an open-loop optimal control of Problem (\(\widetilde{\text{ LQ }}\)) if and only if \(u^*(\cdot )\triangleq v^*(\cdot )+\Sigma \widetilde{X}(\cdot \,;x,v^*(\cdot ))\) is an open-loop optimal control of Problem (LQ).

     
  2. (ii)

    Problem (\(\widetilde{\text{ LQ }}\)) is closed-loop solvable if and only if Problem (LQ) is. In this case, \((\Sigma ^*,v^*(\cdot ))\) is a closed-loop optimal strategy of Problem (\(\widetilde{\text{ LQ }}\)) if and only if \((\Sigma ^*+\Sigma ,v^*(\cdot ))\) is a closed-loop optimal strategy of Problem (LQ).

     

Proof of Theorem 4.5

general case. (ii) trivially implies (i), and the implication (iii) \(\mathop {\Rightarrow }\) (ii) can be proved in the same way as in the case when [AC] is \(L^2\)-stable. For the implication (i) \(\mathop {\Rightarrow }\) (iii), we consider Problem (\(\widetilde{\text{ LQ }}\)). Since the open-loop solvabilities of Problem (LQ) and Problem (\(\widetilde{\text{ LQ }}\)) are equivalent, and \([\widetilde{A},\widetilde{C}]\) is \(L^2\)-stable, by the result for the \(L^2\)-stable case, the following ARE
$$\begin{aligned} \left\{ \begin{array}{ll} P\widetilde{A}+\widetilde{A}^\top P+\widetilde{C}^\top P\widetilde{C}+\widetilde{Q}\\ -\left( PB+\widetilde{C}^\top PD+\widetilde{S}^\top \right) \left( R+D^\top PD\right) ^\dag \left( B^\top P+D^\top P\widetilde{C}+\widetilde{S}\right) =0,\\ {\mathscr {R}}\left( B^\top P+D^\top P\widetilde{C}+\widetilde{S}\right) \subseteq {\mathscr {R}}\left( R+D^\top PD\right) ,\\ R+D^\top PD\ge 0 \end{array}\right. \end{aligned}$$
admits a (unique) static stabilizing solution \(P\in \mathbb {S}^n\), and the following BSDE
$$\begin{aligned} d\eta (t)= & {} -\Big \{\left[ \widetilde{A}^\top -\left( PB+\widetilde{C}^\top PD+\widetilde{S}^\top \right) \left( R+D^\top PD\right) ^\dag B^\top \right] \eta \nonumber \\&+\,\left[ \widetilde{C}^\top -\left( PB+\widetilde{C}^\top PD+\widetilde{S}^\top \right) \left( R+D^\top PD\right) ^\dag D^\top \right] \zeta \nonumber \\&+\,\left[ \widetilde{C}^\top -\left( PB+\widetilde{C}^\top PD+\widetilde{S}^\top \right) \left( R+D^\top PD\right) ^\dag D^\top \right] P\sigma \nonumber \\&-\,\left( PB+\widetilde{C}^\top PD+\widetilde{S}^\top \right) \left( R+D^\top PD\right) ^\dag \rho +Pb+\widetilde{q}\Big \}dt+\zeta dW(t), \quad t\ge 0\nonumber \\ \end{aligned}$$
(6.3)
admits an \(L^2\)-stable adapted solution \((\eta (\cdot ),\zeta (\cdot ))\) such that
$$\begin{aligned}&B^\top \eta (t)+D^\top \zeta (t)+D^\top P\sigma (t)+\rho (t)\in {\mathscr {R}}\left( R+D^\top PD\right) ,\nonumber \\&\quad \mathrm{a.e. }~t\in [0,\infty ),~~\mathrm{a.s.}\end{aligned}$$
(6.4)
Using (6.4) and the equality
$$\begin{aligned} \left( PB+\widetilde{C}^\top PD+\widetilde{S}^\top \right) \left( R+D^\top PD\right) ^\dag =\mathcal{L}(P)\mathcal{N}(P)^\dag +\Sigma ^\top \mathcal{N}(P)\mathcal{N}(P)^\dag , \end{aligned}$$
it is straightforward to show that (6.3) is equivalent to
$$\begin{aligned} d\eta (t)&=-\Big \{\left[ A^\top -\mathcal{L}(P)\mathcal{N}(P)^\dag B^\top \right] \eta +\left[ C^\top -\mathcal{L}(P)\mathcal{N}(P)^\dag D^\top \right] \zeta \\&\quad +\,\left[ C^\top -\mathcal{L}(P)\mathcal{N}(P)^\dag D^\top \right] P\sigma -\mathcal{L}(P)\mathcal{N}(P)^\dag \rho +Pb+q\Big \}dt+\zeta dW(t), \quad t\ge 0. \end{aligned}$$
Thus, we need only show that P is a static stabilizing solution of ARE (4.4). To this end, choose \(\Lambda \in \mathbb {R}^{m\times n}\) such that
$$\begin{aligned} \Sigma ^*\triangleq & {} -\left( R+D^\top PD\right) ^\dag \left( B^\top P+D^\top P\widetilde{C}+\widetilde{S}\right) \\&+\left[ I-\left( R+D^\top PD\right) ^\dag \left( R+D^\top PD\right) \right] \Lambda \end{aligned}$$
is a stabilizer of \([\widetilde{A},\widetilde{C};B,D]\). We have
$$\begin{aligned} \begin{array}{lll} \left( R+D^\top PD\right) (\Sigma ^*+\Sigma ) &{}= &{} -\,\left( B^\top P+D^\top P\widetilde{C}+\widetilde{S}\right) +\left( R+D^\top PD\right) \Sigma \\ &{}= &{} -\,\left( B^\top P+D^\top PC+S\right) . \end{array} \end{aligned}$$
(6.5)
It follows that \({\mathscr {R}}\left( B^\top P+D^\top PC+S\right) \subseteq {\mathscr {R}}\left( R+D^\top PD\right) \). Moreover,
$$\begin{aligned} 0= & {} P\widetilde{A}+\widetilde{A}^\top P+\widetilde{C}^\top P\widetilde{C}+\widetilde{Q} -\left( PB+\widetilde{C}^\top PD+\widetilde{S}^\top \right) \left( R+D^\top PD\right) ^\dag \left( B^\top P+D^\top P\widetilde{C}+\widetilde{S}\right) \\= & {} P(A+B\Sigma )+(A+B\Sigma )^\top P+(C+D\Sigma )^\top P(C+D\Sigma )+Q+S^\top \Sigma +\Sigma ^\top S+\Sigma ^\top R\Sigma \\&-\;\left[ PB+(C+D\Sigma )^\top PD+(S+R\Sigma )^\top \right] \left( R+D^\top PD\right) ^\dag \left[ B^\top P+D^\top P(C+D\Sigma )+S+R\Sigma \right] \\= & {} PA+A^\top P+C^\top PC+Q +\left( PB+C^\top PD+S^\top \right) \Sigma +\Sigma ^\top \left( B^\top P+D^\top PC+S\right) \\&-\,\left( PB+C^\top PD+S^\top \right) \left( R+D^\top PD\right) ^\dag \left( B^\top P+D^\top PC+S\right) \\&-\,\left( PB+C^\top PD+S^\top \right) \left( R+D^\top PD\right) ^\dag \left( R+D^\top PD\right) \Sigma \\&-\,\Sigma ^\top \left( R+D^\top PD\right) \left( R+D^\top PD\right) ^\dag \left( B^\top P+D^\top PC+S\right) \\= & {} PA+A^\top P+C^\top PC+Q -\left( PB+C^\top PD+S^\top \right) \left( R+D^\top PD\right) ^\dag \left( B^\top P+D^\top PC+S\right) \\&+\,\left( PB+C^\top PD+S^\top \right) \left[ I-\left( R+D^\top PD\right) ^\dag \left( R+D^\top PD\right) \right] \Sigma \\&+\,\Sigma ^\top \left[ I-\left( R+D^\top PD\right) \left( R+D^\top PD\right) ^\dag \right] \left( B^\top P+D^\top PC+S\right) \\= & {} PA+A^\top P+C^\top PC+Q -\left( PB+C^\top PD+S^\top \right) \left( R+D^\top PD\right) ^\dag \left( B^\top P+D^\top PC+S\right) \\&-\,(\Sigma ^*+\Sigma )^\top \left( R+D^\top PD\right) \left[ I-\left( R+D^\top PD\right) ^\dag \left( R+D^\top PD\right) \right] \Sigma \\&-\,\Sigma ^\top \left[ I-\left( R+D^\top PD\right) \left( R+D^\top PD\right) ^\dag \right] \left( R+D^\top PD\right) (\Sigma ^*+\Sigma )\\= & {} PA+A^\top P+C^\top PC+Q -\left( PB+C^\top PD+S^\top \right) \left( R+D^\top PD\right) ^\dag \left( B^\top P+D^\top PC+S\right) . \end{aligned}$$
Therefore, P solves ARE (4.4). Noting that \(\Sigma ^*+\Sigma \) is a stabilizer of [ACBD], it is further clear from (6.5) and Lemma 2.6 that P is static stabilizing. \(\square \)

7 The One-Dimensional Case

In this section, we look at the case where both the state and the control variables are one-dimensional. For such a case, we can solve Problem (LQ)\(^0\) completely. To avoid trivial exceptions we assume that
$$\begin{aligned} \left\{ \begin{array}{ll} B\ne 0~~\,\text{ or }\,~~ D\ne 0,\\ \mathscr {S}[A,C;B,D]\ne \emptyset .\end{array}\right. \end{aligned}$$
(7.1)
By Lemma 2.2, the second condition in (7.1) is equivalent to the solvability of \(2(A+B \Theta )+(C+D \Theta )^2<0\) with the unknown \(\Theta \). It is then easy to verify that (7.1) holds if and only if
$$\begin{aligned} (2A+C^2)D^2<(B+CD)^2. \end{aligned}$$
(7.2)
Let us first look at the case \(D=0\). By scaling, we may assume without loss of generality that \(B=1\). Then ARE (4.4) becomes
$$\begin{aligned} \left\{ \begin{array}{ll} P(2A+C^2)+Q-R^\dag (P+S)^2=0,\\ P+S=0\quad \text{ if }~R=0,\\ R\ge 0.\end{array}\right. \end{aligned}$$
(7.3)
Also, we note that, by Lemma 2.2, \(\Theta \) is a stabilizer of [AC; 1, 0] if and only if \(\Theta <-(2A+C^2)/2\).

Theorem 7.1

Suppose that \(D=0\) and \(B=1\). We have the following:
  1. (i)

    If \(R<0\), then Problem (LQ)\(^0\) is not solvable.

     
  2. (ii)
    If \(R=0\), then Problem (LQ)\(^0\) is solvable if and only if \(Q=S(2A+C^2)\). In this case,
    $$\begin{aligned} (\Theta , v(\cdot ))\quad \text{ with }\quad \Theta <-(2A+C^2)/2,\quad v(\cdot )\in L^2_{\mathbb {F}}(\mathbb {R}) \end{aligned}$$
    are all the closed-loop optimal strategies of Problem (LQ)\(^0\).
     
  3. (iii)
    If \(R>0\), then Problem (LQ)\(^0\) is solvable if and only if \(\Sigma \equiv R(2A+C^2)^2-4S(2A+C^2)+4Q>0\). In this case,
    $$\begin{aligned} \left(-\,{2A+C^2+\sqrt{\Sigma /R}\over 2},~0\right) \end{aligned}$$
    is the unique closed-loop optimal strategy of Problem (LQ)\(^0\).
     

Proof

  1. (i)

    It is obvious because ARE (7.3) is not solvable in this case.

     
  2. (ii)
    When \(R=0\), ARE (7.3) further reduces to
    $$\begin{aligned} \left\{ \begin{array}{ll} P(2A+C^2)+Q=0,\\ P+S=0,\end{array}\right. \end{aligned}$$
    which is solvable if and only if \(Q=S(2A+C^2)\). In this case, \(\mathcal{N}(P)=R=0\), and the second assertion follows immediately from Corollary 4.6.
     
  3. (iii)
    When \(R>0\), ARE (7.3) can be written as
    $$\begin{aligned} P^2+\left[ 2S-(2A+C^2)R\right] P+S^2-QR=0, \end{aligned}$$
    (7.4)
    which is solvable if and only if
    $$\begin{aligned} 0\le & {} \Delta \triangleq \left[ 2S-(2A+C^2)R\right] ^2-4(S^2-QR)\\= & {} R\left[ R(2A+C^2)^2-4S(2A+C^2)+4Q\right] . \end{aligned}$$
    In the case of \(\Delta \ge 0\), (7.4) has two solutions:
    $$\begin{aligned} P_1={(2A+C^2)R-2S-\sqrt{\Delta }\over 2},\quad P_2={(2A+C^2)R-2S+\sqrt{\Delta }\over 2}, \end{aligned}$$
    and \(P_i\) is static stabilizing if and only if
    $$\begin{aligned} -{2A+C^2\over 2}>-{P_i+S\over R}=-{2A+C^2\over 2}-{(-1)^i\sqrt{\Delta }\over 2R}. \end{aligned}$$
    Clearly, \(P_1\) cannot be static stabilizing, and \(P_2\) is static stabilizing if and only if \(\Delta >0\), or equivalently, \(R(2A+C^2)^2-4S(2A+C^2)+4Q>0\). The second assertion follows easily. \(\square \)
     
We now look at the case \(D\ne 0\). As before, we may assume, without loss of generality (by scaling, if necessary), that \(D=1\). Denote
$$\begin{aligned} \left\{ \begin{array}{ll} \alpha =(B+C)^2-(2A+C^2),\\ \beta =Q-(2A+C^2)R+2(B+C)[(B+C)R-S],\\ \gamma =[(B+C)R-S]^2.\end{array}\right. \end{aligned}$$
(7.5)
Then (7.2) is equivalent to \(\alpha >0\), and \(\Theta \) is a stabilizer of [ACB, 1] if and only if
$$\begin{aligned} |\Theta +B+C|<\sqrt{\alpha }. \end{aligned}$$
(7.6)

Theorem 7.2

Suppose that \(D=1\) and \(\alpha >0\). Then Problem (LQ)\(^0\) is solvable if and only if one of the following conditions holds:
  1. (i)
    \(Q=(2A+C^2)R\) and \(S=(B+C)R\). In this case,
    $$\begin{aligned} (\Theta , v(\cdot ))\quad \text{ with }\quad |\Theta +B+C|<\sqrt{\alpha },\quad v(\cdot )\in L^2_{\mathbb {F}}(\mathbb {R}) \end{aligned}$$
    (7.7)
    are all the closed-loop optimal strategies of Problem (LQ)\(^0\).
     
  2. (ii)
    \(2A+C^2\ne 0\), \((2A+C^2)S\ge (B+C)Q\), and
    $$\begin{aligned} R>{2(B+C-\sqrt{\alpha })S-Q\over (B+C-\sqrt{\alpha })^2}. \end{aligned}$$
     
  3. (iii)
    \(2A+C^2\ne 0\), \((2A+C^2)S<(B+C)Q\), and
    $$\begin{aligned} R>{2(B+C+\sqrt{\alpha })S-Q\over (B+C+\sqrt{\alpha })^2}. \end{aligned}$$
     
  4. (iv)
    \(2A+C^2=0\), \(Q>0\), and
    $$\begin{aligned} R>{4(B+C)S-Q\over 4(B+C)^2}. \end{aligned}$$
    In the cases (ii), (iii), and (iv),
    $$\begin{aligned} \left({2\alpha [(B+C)R-S]\over \beta +\sqrt{\beta ^2-4\alpha \gamma }}-(B+C),~0\right) \end{aligned}$$
    is the unique closed-loop optimal strategy of Problem (LQ)\(^0\).
     

Proof

We rewrite ARE (4.4) as follows:
$$\begin{aligned} \left\{ \begin{array}{ll} P(2A+C^2)+Q-(R+P)^\dag [P(B+C)+S]^2=0,\\ P(B+C)+S=0\quad \text{ if }~R+P=0,\\ R+P\ge 0.\end{array}\right. \end{aligned}$$
(7.8)
By Corollary 4.6, Problem (LQ)\(^0\) is solvable if and only if (7.8) admits a static stabilizing solution. So we need only discuss the solvability of (7.8).
Clearly, \(P=-R\) is a solution of (7.8) if and only if
$$\begin{aligned} Q=(2A+C^2)R,\quad S=(B+C)R. \end{aligned}$$
(7.9)
In this case, \(P=-R\) is also static stabilizing, and \(\mathcal{N}(P)=R+P=0\). By Corollary 4.6 and (7.6), we see all closed-loop optimal strategies of Problem (LQ)\(^0\) are given by (7.7).
If (7.9) does not hold, by the change of variable \(y=R+P\), equation (7.8) further reduces to the following:
$$\begin{aligned} \left\{ \begin{array}{ll} \alpha y^2-\beta y+\gamma =0,\\ y>0,\end{array}\right. \end{aligned}$$
(7.10)
which is solvable if and only if
$$\begin{aligned} \Delta =\beta ^2-4\alpha \gamma \ge 0,\quad \beta +\sqrt{\Delta }>0, \end{aligned}$$
or equivalently (noting that \(\alpha >0\) and \(\gamma \ge 0\)),
$$\begin{aligned} \Delta =\beta ^2-4\alpha \gamma \ge 0,\quad \beta >0. \end{aligned}$$
(7.11)
In this case, if \(\gamma >0\), then (7.10) has two solutions:
$$\begin{aligned} y_1=R+P_1={\beta -\sqrt{\Delta }\over 2\alpha },\quad y_2=R+P_2={\beta +\sqrt{\Delta }\over 2\alpha }. \end{aligned}$$
(7.12)
For \(i=1,2\), let
$$\begin{aligned} \Theta _i=-{P_i(B+C)+S\over R+P_i}={(B+C)R-S\over y_i}-(B+C). \end{aligned}$$
Note that \(\Theta _i\) is a stabilizer of [ACB, 1] if and only if \(|\Theta _i+B+C|<\sqrt{\alpha }\), or equivalently,
$$\begin{aligned} \gamma =[(B+C)R-S]^2<\alpha y_i^2=\beta y_i-\gamma . \end{aligned}$$
(7.13)
Upon substitution of (7.12) into (7.13), the latter is in turn equivalent to
$$\begin{aligned} \Delta +(-1)^i\beta \sqrt{\Delta }>0. \end{aligned}$$
(7.14)
Obviously, (7.14) cannot hold for \(i=1\), and it holds for \(i=2\) if and only if \(\Delta >0\). Likewise, if \(\gamma =0\), then \(P_2\) is the unique solution of (7.10), and \(\Theta _2\) is a a stabilizer of [ACB, 1] if and only if \(\Delta >0\). Therefore, ARE (7.8) admits a static stabilizing solution \(P\ne R\) if and only if
$$\begin{aligned} \beta>0,\quad \beta ^2-4\alpha \gamma >0. \end{aligned}$$
(7.15)
Noting that \(\beta =[(B+C)^2+\alpha ]R+Q-2(B+C)S\), we have by a straightforward computation:
$$\begin{aligned} \beta ^2-4\alpha \gamma= & {} [(B+C)^2-\alpha ]^2R^2-\Big (4[(B+C)^2-\alpha ](B+C)S-2[(B+C)^2+\alpha ]Q\Big )R\\&+\;Q^2-4(B+C)SQ+4[(B+C)^2-\alpha ]S^2\\\equiv & {} aR^2-bR+c. \end{aligned}$$
Also, we have
$$\begin{aligned} b^2-4ac=16\alpha \Big ([(B+C)^2-\alpha ]S-(B+C)Q\Big )^2\ge 0. \end{aligned}$$
If \(a=[(B+C)^2-\alpha ]^2=(2A+C^2)^2\ne 0\), then \(\beta ^2-4\alpha \gamma >0\) if and only if
$$\begin{aligned} R>{b+\sqrt{b^2-4ac}\over 2a},\quad \text{ or }\quad R<{b-\sqrt{b^2-4ac}\over 2a}. \end{aligned}$$
Because \(R={2(B+C)S-Q\over (B+C)^2+\alpha }\) implies \(\beta ^2-4\alpha \gamma =-4\alpha \gamma \le 0\), we have
$$\begin{aligned} {b-\sqrt{b^2-4ac}\over 2a}\le {2(B+C)S-Q\over (B+C)^2+\alpha }\le {b+\sqrt{b^2-4ac}\over 2a}. \end{aligned}$$
Therefore, (7.15) holds if and only if
$$\begin{aligned} R>{b+\sqrt{b^2-4ac}\over 2a}=\left\{ \begin{array}{ll} {2(B+C-\sqrt{\alpha })S-Q\over (B+C-\sqrt{\alpha })^2},\quad \text{ if }\quad (2A+C^2)S\ge (B+C)Q,\\ {2(B+C+\sqrt{\alpha })S-Q\over (B+C+\sqrt{\alpha })^2},\quad \text{ if }\quad (2A+C^2)S<(B+C)Q. \end{array}\right. \end{aligned}$$
(7.16)
If \(a=[(B+C)^2-\alpha ]^2=(2A+C^2)^2=0\), then
$$\begin{aligned} \beta= & {} 2(B+C)^2R-2(B+C)S+Q,\\ \beta ^2-4\alpha \gamma= & {} Q\left[ 4(B+C)^2R-4(B+C)S+Q\right] , \end{aligned}$$
and it is not hard to see that (7.15) holds if and only if
$$\begin{aligned} Q>0,\quad R>{4(B+C)S-Q\over 4(B+C)^2}. \end{aligned}$$
(7.17)
Finally, in the cases (ii), (iii), and (iv), we see from the preceding argument that ARE (7.8) has a unique static stabilizing solution
$$\begin{aligned} P={\beta +\sqrt{\Delta }\over 2\alpha }-R. \end{aligned}$$
Note that \(\mathcal{N}(P)=R+P>0\) and
$$\begin{aligned} -\mathcal{N}(P)^{-1}\mathcal{L}(P)=-{P(B+C)+S\over R+P} ={2\alpha [(B+C)R-S]\over \beta +\sqrt{\beta ^2-4\alpha \gamma }}-(B+C). \end{aligned}$$
The last assertion follows immediately from Corollary 4.6. \(\square \)

Notes

Acknowledgements

The authors would like to thank the anonymous referees for their suggestive comments, which lead to an improvement of the paper. Jiongmin Yong was partially supported by NSF Grant DMS-1406776.

References

  1. 1.
    Ait Rami, M., Moore, J.B., Zhou, X.Y.: Indefinite stochastic linear quadratic control and generalized differential Riccati equation. SIAM J. Control Optim. 40, 1296–1311 (2001)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Ait Rami, M., Zhou, X.Y.: Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls. IEEE Trans. Automat. Control 45, 1131–1143 (2000)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Ait Rami, M., Zhou, X.Y., Moore, J.B.: Well-posedness and attainability of indefnite stochastic linear quadratic control in infnite time horizon. Syst. Control Lett. 41, 123–133 (2000)CrossRefGoogle Scholar
  4. 4.
    Anderson, B.D.O., Moore, J.B.: Optimal Control: Linear Quadratic Methods. Prentice Hall, Englewood Cliffs (1989)Google Scholar
  5. 5.
    Bensoussan, A.: Lectures on Stochastic Control, Part I, in Nonlinear Filtering and Stochastic Control. Lecture Notes in Mathematics, vol. 972. Springer, Berlin (1982)MATHGoogle Scholar
  6. 6.
    Chen, S., Li, X., Zhou, X.Y.: Stochastic linear quadratic regulators with indefinite control weight costs. SIAM J. Control Optim. 36, 1685–1702 (1998)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Chen, S., Yong, J.: Stochastic linear quadratic optimal control problems. Appl. Math. Optim. 43, 21–45 (2001)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Chen, S., Zhou, X.Y.: Stochastic linear quadratic regulators with indefinite control weight costs. II. SIAM J. Control Optim. 39, 1065–1081 (2000)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Davis, M.H.A.: Linear Estimation and Stochastic Control. Chapman and Hall, London (1977)MATHGoogle Scholar
  10. 10.
    Hu, Y., Zhou, X.Y.: Indefinite stochastic Riccati equations. SIAM J. Control Optim. 42, 123–137 (2003)MathSciNetCrossRefMATHGoogle Scholar
  11. 11.
    Huang, J., Li, X., Yong, J.: A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Math. Control & Relat. Fields 5, 97–139 (2015)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Kalman, R.E.: Contributions to the theory of optimal control. Bol. Soc., Mat. Mex. 5, 102–119 (1960)MathSciNetGoogle Scholar
  13. 13.
    Li, X., Sun, J., Yong, J.: Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability. Probab. Uncertain. Quant. Risk 1, 2 (2016). doi: 10.1186/s41546-016-0002-3 MathSciNetCrossRefGoogle Scholar
  14. 14.
    Lim, A.E.B., Zhou, X.Y.: Stochastic optimal LQR control with integral quadratic constraints and indefinite control weights. IEEE Trans. Automat. Control 44, 359–369 (1999)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Penrose, R.: A generalized inverse of matrices. Proc. Camb. Philos. Soc. 52, 17–19 (1955)CrossRefMATHGoogle Scholar
  16. 16.
    Qian, Z., Zhou, X.Y.: Existence of solutions to a class of indefinite stochastic Riccati equations. SIAM J. Control Optim. 51, 221–229 (2013)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Sun, J.: Mean-field stochastic linear quadratic optimal control problems: Open-loop solvabilities. ESAIM: COCV. doi: 10.1051/cocv/2016023 (2016)
  18. 18.
    Sun, J., Li, X., Yong, J.: Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems. SIAM J. Control Optim. 54, 2274–2308 (2016)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Sun, J., Yong, J.: Linear quadratic stochastic differential games: open-loop and closed-loop saddle points. SIAM J. Control Optim. 52, 4082–4121 (2014)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    Sun, J., Yong, J., Zhang, S.: Linear quadratic stochastic two-person zero-sum differential games in an infinite horizon. ESAIM: COCV 22, 743–769 (2016)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Wonham, W.M.: On a matrix Riccati equation of stochastic control. SIAM J. Control 6, 681–697 (1968)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Wu, H., Zhou, X.Y.: Stochastic frequency characteristics. SIAM J. Control Optim. 40, 557–576 (2001)MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Yao, D.D., Zhang, S., Zhou, X.Y.: Stochastic linear-quadratic control via semidefinite programming. SIAM J. Control Optim. 40, 801–823 (2001)MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Yong, J.: Linear-quadratic optimal control problems for mean-field stochastic differential equations. SIAM J. Control Optim. 51, 2809–2838 (2013)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Yong, J., Zhou, X.Y.: Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer, New York (1999)CrossRefMATHGoogle Scholar
  26. 26.
    You, Y.: Optimal control for linear system with quadratic indefinite criterion on Hilbert spaces. Chinese Ann. Math. Ser. B 4, 21–32 (1983)MathSciNetMATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Department of MathematicsNational University of SingaporeSingaporeSingapore
  2. 2.Department of MathematicsUniversity of Central FloridaOrlandoUSA

Personalised recommendations