Some Elements of Linear-Quadratic Optimal Controls

Abstract

This chapter is a brief review on the stochastic linear-quadratic optimal control. Some useful concepts and results, which will be needed throughout this book, are presented in the context of finite and infinite horizon problems. These materials are mainly for beginners and may also serve as a quick reference for knowledgeable readers.

1.3 Appendix: Pseudoinverse and Infinite-Horizon BSDEs

Let M be an $m\times n$ real matrix. The Moore-Penrose pseudoinverse of M, denoted by $M^\dag$, is an $n\times m$ real matrix such that

$$\begin{aligned} MM^\dag M&=M, \quad (MM^\dag )^\top =MM^\dag , \\ M^\dag MM^\dag&=M^\dag , \,\, (M^\dag M)^\top =M^\dag M. \end{aligned}$$

Every matrix has a unique (Moore-Penrose) pseudoinverse. If $M\in \mathbb {S}^n$, then $M^\dag \in \mathbb {S}^n$, $MM^\dag =M^\dag M$, and $M\geqslant 0$ if and only if $M^\dag \geqslant 0$.

Proposition 1.3.1

Let $\mathcal{I}$ be an interval. Let L(t) and N(t) be two Lebesgue measurable functions on $\mathcal{I}$, with values in $\mathbb {R}^{n\times k}$ and $\mathbb {R}^{n\times m}$, respectively. Then the equation $N(t)X(t)=L(t)$ has a solution $X(t)\in L^2(\mathcal{I};\mathbb {R}^{m\times k})$ if and only if

$${(i)}~\mathscr {R}(L(t))\subseteq \mathscr {R}(N(t)), \quad \text{ and }\quad {(ii)}~N(t)^\dag L(t)\in L^2(\mathcal{I};\mathbb {R}^{m\times k}),$$

in which case the general solution is given by

$$ X(t)=N(t)^\dag L(t)+[I_m-N(t)^\dag N(t)]Y(t), $$

where $Y(t)\in L^2(\mathcal{I};\mathbb {R}^{m\times k})$ is arbitrary.

Remark 1.3.2

The following are obvious:

• (i) The condition $\mathscr {R}(L(t))\subseteq \mathscr {R}(N(t))$ is equivalent to

  $$ N(t)N(t)^\dag L(t)=L(t). $$

• (ii) If $N(t)\in \mathbb {S}^n$ and $N(t)X(t)=L(t)$, then

  $$ X(t)^\top N(t)X(t)=L(t)^\top N(t)^\dag L(t). $$

Next, let $\mathcal{X}[0,\infty )$ be the subspace of $L_\mathbb {F}^2(\mathbb {R}^n)$ whose elements are continuous. Consider the following BSDE over the infinite horizon $[0,\infty )$:

$$\begin{aligned} dY(t) = -[A^\top Y(t)+C^\top Z(t)+\varphi (t)]dt + Z(t)dW(t), \quad t\in [0,\infty ),\end{aligned}$$

(1.3.1)

where $A, C\in \mathbb {R}^{n\times n}$ are given constant matrices, and $\{\varphi (t);0\leqslant t<\infty \}$ is a given $\mathbb {F}$-progressively measurable, $\mathbb {R}^n$-valued process.

Definition 1.3.3

An $L^2$-stable adapted solution to the BSDE (1.3.1) is a pair $(Y, Z)\in \mathcal{X}[0,\infty )\times L_\mathbb {F}^2(\mathbb {R}^n)$ that satisfies the integral version of (1.3.1):

$$ Y(t) = Y(0) - \int _0^t\Big [A^\top Y(s)+C^\top Z(s)+\varphi (s)\Big ]ds + \int _0^tZ(s)dW(s), ~ t\geqslant 0. $$

The following theorem establishes the basic existence and uniqueness result for the BSDE (1.3.1).

Theorem 1.3.4

Suppose that the system (1.2.1) is $L^2$-stable. Then for every $\varphi \in L^2_\mathbb {F}(\mathbb {R}^n)$, the BSDE (1.3.1) admits a unique $L^2$-stable adapted solution.