Optimal Power Allocation for Kalman Filtering over Fading Channels

Abstract

Kalman filtering with random packet drops has been studied extensively since the work of Sinopoli (IEEE Trans. Autom. Control 49(9), 1453–1464, 2004, [1]), which showed that for i.i.d. Bernoulli packet drops, there exists a critical threshold such that if the packet arrival rate exceeds this threshold, then the expected error covariance remains bounded but diverges otherwise.

Appendix

2.1.1 Proof of Lemma 2.2

Proof

The proof uses similar ideas to the proof of Proposition 3.1 in [36]. The decreasing property follows from the relation

$$\begin{aligned} \mathbb {E}[P_{k+1}]&= \mathbb {E}[P_{k+1}|P_k, g_k,u_k]\\&= \mathbb {E}[A P_k A^T \!\!+\! Q\! -\! f(g_k u_k) A P_k C^T \!(C P_k C^T\!\!+\!R)^{-1}\! C P_k A^T] \end{aligned}$$

and the assumption that f(.) is an increasing function.

For the proof of convexity, let $u^1$ and $u^2$ be two average transmit powers, where $u^1 \ne u^2$, with $\mathfrak {p}^*(u^1)$ and $\mathfrak {p}^*(u^2)$ the corresponding traces of the expected error covariances. We want to show that

$$\mathfrak {p}^*(\lambda u^1 + (1-\lambda )u^2) < \lambda \mathfrak {p}^*(u^1) + (1-\lambda ) \mathfrak {p}^*(u^2), \forall \lambda \in (0,1).$$

Let $\{u_k^1 (P_k,g_k)\}$ be the optimal power allocation policy that achieves $\mathfrak {p}^*(u^1)$, and $\{u_k^2(P_k,g_k)\}$ be the optimal power allocation policy that achieves $\mathfrak {p}^*(u^2)$. Define a new policy $\{u^\lambda _k(P_k,g_k)\}$ such that

$$u_k^\lambda (P_k,g_k) = \lambda u_k^1(P_k,g_k) + (1-\lambda ) u_k^2(P_k,g_k), \forall P_k,g_k.$$

We will first show that for a given $P_k$, we have:

$$\begin{aligned} \begin{aligned}&(1)\,\, \mathbb {E}[u_k^\lambda | P_k] \le \lambda \mathbb {E}[u_k^1 | P_k] + (1-\lambda ) \mathbb {E}[u_k^2 | P_k], \text { and } \\&(2)\,\, \mathbb {E}[\text {tr}(P_{k+1}^\lambda ) | P_k] < \lambda \mathbb {E}[\text {tr}(P_{k+1}^1) | P_k] + (1-\lambda ) \mathbb {E}[\text {tr}(P_{k+1}^2) | P_k], \end{aligned} \end{aligned}$$

where $P_{k+1}^j$ is the value of $P_{k+1}$ that follows from using policy $\{u_k^j(.)\}$, for $j=1,2,\lambda$, respectively. For (1), this clearly follows from the definition of $u_k^\lambda$. For (2), we have

$$\begin{aligned} \begin{aligned}&\mathbb {E}[\text {tr}(P_{k+1}^\lambda ) | P_k] \\&= \int \Big ( \text {tr}(A P_k A^T+ Q) - f(g_k u_k^\lambda ) \text {tr}(A P_k C^T (C P_k C^T + R)^{-1} C P_k A^T ) \Big ) F(dg_k) \\&< \int \Big (\text {tr}(A P_k A^T + Q) - (\lambda f(g_k u_k^1) + (1-\lambda ) f(g_k u_k^2)) \\&\qquad \qquad \times \text {tr}(A P_k C^T (C P_k C^T + R)^{-1} C P_k A^T ) \Big )F(dg_k)\\&= \lambda \mathbb {E}[\text {tr}(P_{k+1}^1)|P_k] + (1-\lambda ) \mathbb {E}[\text {tr}(P_{k+1}^2)|P_k] \end{aligned} \end{aligned}$$

where the inequality comes from the strict concavity of f(.).

From (1) and (2), we have

$$\begin{aligned} \begin{aligned} \lim _{K\rightarrow \infty } \frac{1}{K} \sum _{k=1}^K \mathbb {E}[u_k^\lambda ]&= \lim _{K\rightarrow \infty } \frac{1}{K} \sum _{k=1}^K \mathbb {E}[\mathbb {E}[u_k^\lambda |P_k]] \\&\le \lim _{K\rightarrow \infty } \frac{1}{K} \sum _{k=1}^K \mathbb {E}[\lambda \mathbb {E}[u_k^1|P_k] + (1-\lambda ) \mathbb {E}[u_k^2|P_k]] \\&= \lambda u^1 + (1-\lambda ) u^2 \end{aligned} \end{aligned}$$

and

$$\begin{aligned} \begin{aligned} \lim _{K\rightarrow \infty } \frac{1}{K} \sum _{k=1}^K \mathbb {E}[\text {tr}(P_{k+1}^\lambda )]&= \lim _{K\rightarrow \infty } \frac{1}{K} \sum _{k=1}^K \mathbb {E}[\mathbb {E}[\text {tr}(P_{k+1}^\lambda )|P_k]] \\&< \lim _{K\rightarrow \infty } \frac{1}{K} \sum _{k=1}^K \mathbb {E}\Big [\lambda \mathbb {E}[\text {tr}(P_{k+1}^1)|P_k] + (1-\lambda ) \mathbb {E}[\text {tr}(P_{k+1}^2)|P_k]\Big ] \\&= \lambda \mathfrak {p}^*(u^1) + (1-\lambda ) \mathfrak {p}^*(u^2). \end{aligned} \end{aligned}$$

By the definition of $\mathfrak {p}^*(u)$ being the minimum expected error covariance such that the average transmit power is less than or equal to u, we then have $\mathfrak {p}^*(\lambda u^1 + (1-\lambda )u^2) \le \frac{1}{K} \sum _{k=1}^K \mathbb {E}[\text {tr}(P_{k+1}^\lambda )] < \lambda \mathfrak {p}^*(u^1) + (1-\lambda ) \mathfrak {p}^*(u^2)$.

2.1.2 Proof of Theorem 2.3

We first establish the inequality

$$\begin{aligned}&\rho + V(P,g,H,B) \ge \min _{0 \le u \le B} \Big \{\mathbb {E} \big [\mathrm {tr}\big (\mathscr {L}(P,\gamma )\big )\big |P,g,u\big ] \nonumber \\& + \mathbb {E} \Big [V\big (\mathscr {L}(P,\gamma ),\tilde{g},\tilde{H},\min \{B-u+\tilde{H}, B_{\text {max}}\}\big ) \big | P,g,H,u\Big ] \Big \} \end{aligned}$$

(2.26)

by verifying conditions (W) and (B) of [48], that guarantee the existence of solutions to (2.26) for MDPs with general state space. Denote the state space by $\mathscr {S}$ and action space by $\mathscr {A}$, i.e. $(P_k,g_k,H_k,B_k) \in \mathscr {S}$ and $u_k \in \mathscr {A}$. Condition (W) of [48] in our notation says that:

(1) The state space $\mathscr {S}$ is locally compact.

(2) Let $U(\cdot )$ be the mapping that assigns to each $(P_k,g_k,H_k,B_k)$ the nonempty set of available actions. Then $U(P_k,g_k,H_k,B_k)$ lies in a compact subset of $\mathscr {A}$ and $U(\cdot )$ is upper semicontinuous.

(3) The transition probabilities are weakly continuous.

(4) $\mathbb {E} \big [\mathrm {tr}\big (\mathscr {L}(P,\gamma )\big )\big |P,g,u\big ]$ is lower semicontinuous.

By our assumption that $u_k \le B_k \le B_{\text {max}}$, (0) and (1) of (W) can be easily verified. The conditions (2) and (3) follow from the definition (2.23).

Define $w_{\delta }(P_0,g_0,H_0,B_0) = v_{\delta } (P_0,g_0,H_0,B_0) - m_{\delta }$, where

$$\begin{aligned}&v_{\delta } (P_0,g_0,H_0,B_0) = \inf _{\{u_k: k \ge 0\}} \mathbb {E}\left[ \sum _{k=0}^\infty \delta ^k \mathbb {E} \big [\mathrm {tr}\big (\mathscr {L}(P_k,\gamma _k)\big ) \big |P_k,g_k,u_k\big ]\big |P_0,g_0,H_0,B_0\right] \end{aligned}$$

and $m_{\delta } = \inf _{(P_0,g_0,H_0,B_0)} v_\delta (P_0,g_0,H_0,B_0)$. Condition (B) of [48] in our notation says that

$$\begin{aligned}&\sup _{\delta<1} w_{\delta } (P_0,g_0,H_0,B_0) < \infty , \qquad \forall ~ (P_0,g_0,H_0,B_0). \end{aligned}$$

Following Sect. 4 of [48], define the stopping time

$$\tau = \inf \{k \ge 0: v_{\delta } (P_k,g_k,H_k,B_k) \le m_{\delta } + \varsigma \}$$

for some $\varsigma \ge 0$. Given $\varsigma > 0$ and an arbitrary $(P_0,g_0,H_0,B_0)$, consider a suboptimal power allocation policy where the sensor transmits based on the same policy as the one that achieves $m_\delta$ (with a different initial condition) until $v_{\delta } (P_N,g_N,H_N,B_N) \le m_{\delta } + \varsigma$ is satisfied at some time N. By the exponential forgetting property of initial conditions for Kalman filtering, we have $N < \infty$ with probability 1 and $\mathbb {E}[N] < \infty$. Since $\tau \le N$, we have $\mathbb {E}[\tau ] < \infty$. Then by Lemma 4.1 of [48],

$$\begin{aligned} w_\delta (P_0,g_0,H_0,B_0)&\le \varsigma + \inf _{\{\gamma _k\}} \mathbb {E}\left[ \sum _{k=0}^{\tau -1} \mathbb {E} \big [\mathrm {tr}\big (\mathscr {L}(P_k,\gamma _k)\big ) \big |P_k,g_k,u_k\big ]\big |P_0,g_0,H_0,B_0\right] \nonumber \\&\le \varsigma + \mathbb {E}[\tau ] \times Z < \infty \end{aligned}$$

(2.27)

where the second inequality uses Wald’s equation, with Z being an upper bound to the expected error covariance, which exists by Theorem 2.2. Hence, condition (B) of [48] is satisfied and a solution to the inequality (2.26) exists.

To show equality in (2.26), we will require a further equicontinuity property of the optimal cost for the related discounted cost MDP to be satisfied. This can be shown by a similar argument as in the proof of Proposition 3.2 of [49]. The assumptions in Sects. 5.4 and 5.5 of [33] may then be verified to conclude the existence of a solution to the average cost optimality equation (2.24).