Charging Coordination via Non-cooperative Games

Abstract

This chapter develops a strategy to coordinate the charging of autonomous plug-in electric vehicles (PEVs) using concepts from non-cooperative games. The foundation of the chapter is a model that assumes PEVs are cost-minimizing and weakly coupled via a common electricity price. At a Nash equilibrium, each PEV reacts optimally with respect to a commonly observed charging trajectory that is the average of all PEV strategies. This average is given by the solution of a fixed point problem in the limit of infinite population size. The ideal solution minimizes electricity generation costs by scheduling PEV demand to fill the overnight non-PEV demand “valley”. The chapter’s central theoretical result is a proof of the existence of a unique Nash equilibrium that almost satisfies that ideal. This result is accompanied by a decentralized computational algorithm and a proof that the algorithm converges to the Nash equilibrium in the infinite system limit. Several numerical examples are used to illustrate the performance of the solution strategy for finite populations. The examples demonstrate that convergence to the Nash equilibrium occurs very quickly over a broad range of parameters, and suggest this method could be useful in situations where frequent communication with PEVs is not possible. The method is useful in applications where fully centralized coordination is not possible, but where optimal or near-optimal charging patterns are essential to system operation.

2.11    Appendices

2.1.1 2.11.1    Statement and Proof of Lemma 2.4

Lemma 2.4

Control trajectories $\mathbf {u}^*_n(\mathbf {z})$, $\mathbf {u}^*_n(\widehat{\mathbf {z}})$ and $\mathbf {v}_n(\mathbf {z},\widehat{\mathbf {z}})$ satisfy the following inequalities for all $\delta >0$:

$$\begin{aligned} |u^*_{nt}(\mathbf {z}) - {v}_{nt}(\mathbf {z},\widehat{\mathbf {z}})| \le \big |({z}_t - \widehat{z}_t) - \frac{1}{2 \delta } \big ( p(\mathfrak {r}_t) - p(\widehat{\mathfrak {r}}_t) \big ) \big |, \quad \text { for all } t \in \mathscr {T}, \end{aligned}$$

(2.39)

$$\begin{aligned} |\mathbf {u}^*_n(\mathbf {z}) - \mathbf {u}^*_n(\widehat{\mathbf {z}})|_1 \le 2 |\mathbf {u}^*_n(\mathbf {z}) - \mathbf {v}_n(\mathbf {z},\widehat{\mathbf {z}})|_1, \end{aligned}$$

(2.40)

where $\mathfrak {r}_t = \frac{1}{\bar{\varLambda }}{(\bar{d}_t + z_t)}$, $\widehat{\mathfrak {r}}_t = \frac{1}{\bar{\varLambda }}{(\bar{d}_t + \widehat{z}_t)}$, and $| \cdot |_1$ denotes the $l_1$ norm of the associated vector.

Proof

For notational simplicity, $\mathbf {\mathbf {v}_n} \equiv \mathbf {\mathbf {v}_n}( \mathbf {z}, \widehat{\mathbf {z}})$ will be used throughout the proof.

Equation (2.39) proof: There are four cases to consider:

• (i)   ${v}_{nt} = {u}^*_{nt}(\mathbf {z}) = 0$. It follows immediately that ${v}_{nt} - {u}^*_{nt}(\mathbf {z}) = 0$.

• (ii)   ${v}_{nt}>0$ and ${u}^*_{nt}(\mathbf {z}) = 0$. By (2.21), ${v}_{nt}>0$ implies ${v}_{nt} = \frac{1}{2\delta } ({A}^*(\mathbf {z}) - p(\widehat{\mathfrak {r}}_t) + 2 \delta \widehat{z}_t)$, and ${u}^*_{nt}(\mathbf {z}) = 0$ implies ${A}^*(\mathbf {z}) - p(\mathfrak {r}_t) + 2 \delta {z}_t \le 0$. Together these give

  $$ 0 < {v}_{nt} - {u}^*_{nt}(\mathbf {z}) \le \frac{1}{2\delta } ({A}^*(\mathbf {z}) -p(\widehat{\mathfrak {r}}_t)+2\delta \widehat{z}_t) - \frac{1}{2\delta } ({A}^*(\mathbf {z})-p(\mathfrak {r}_t)+2\delta {z}_t) $$

  with the last term equal to $( \widehat{z}_t - {z}_t) - \frac{1}{2 \delta } (p(\widehat{\mathfrak {r}}_t) - p(\mathfrak {r}_t))$.

• (iii)   ${v}_{nt}=0$ and ${u}^*_{nt}(\mathbf {z}) > 0$. Similarly to (ii), it can derive,

  $$ 0 < {u}^*_{nt}(\mathbf {z}) - {v}_{nt} \le ( {z}_t - \widehat{z}_t) - \frac{1}{2 \delta } (p(\mathfrak {r}_t) - p(\widehat{\mathfrak {r}}_t)). $$

• (iv)   ${v}_{nt}>0$ and ${u}^*_{nt}(\mathbf {z}) > 0$. By (2.21), it is straightforward to have

  $$ {v}_{nt} - {u}^*_{nt}(\mathbf {z}) = ( \widehat{z}_t - {z}_t) - \frac{1}{2 \delta } (p(\widehat{\mathfrak {r}}_t) - p(\mathfrak {r}_t)). $$

Equation (2.40) proof: There are three cases to consider:

• (i)   $\varSigma (\mathbf {v}_n)=\varSigma (\mathbf {u}^*_n(\mathbf {z}))$. This equality ensures $\mathbf {v}_n \in \mathscr {U}_n$. Also, charging strategy $\mathbf {v}_n$ has the form (2.21) with $A={A}^*(\mathbf {z})$. Therefore, by Lemma 2.2, $\mathbf {v}_n$ is the local optimal strategy with respect to $\widehat{\mathbf {z}}$, and hence $\mathbf {u}^*_n(\widehat{\mathbf {z}}) = \mathbf {v}_n$. It follows that

  $$ |\mathbf {u}^*_n(\widehat{\mathbf {z}}) - \mathbf {u}^*_n(\mathbf {z})|_1 = |\mathbf {v}_n (\mathbf {z}, \widehat{\mathbf {z}}) - \mathbf {u}^*_n(\mathbf {z})|_1 \le 2 |\mathbf {v}_n (\mathbf {z}, \widehat{\mathbf {z}}) - \mathbf {u}^*_n(\mathbf {z})|_1. $$

• (ii)   $\varSigma (\mathbf {v}_n) > \varSigma (\mathbf {u}^*_n(\mathbf {z}))$. By (2.4) it gets that $\varSigma (\mathbf {u}^*_n(\widehat{\mathbf {z}})) = \varSigma (\mathbf {u}^*_n(\mathbf {z}))=\varGamma _n$. Therefore $\varSigma (\mathbf {u}^*_n(\widehat{\mathbf {z}})) < \varSigma (\mathbf {v}_n)$ which, together with (2.21) and the definitions of $\mathbf {u}^*_n(\widehat{\mathbf {z}})$ and $\mathbf {v}_n$, implies,

  $$ {A}^*(\widehat{\mathbf {z}}) < {A}^*(\mathbf {z}), \text { and } u^*_{nt}(\widehat{\mathbf {z}}) \le {v}_{nt}, \text { for all } t. $$

  Hence

  $$\begin{aligned} 0 \le |\mathbf {v}_n - \mathbf {u}^*_n(\widehat{\mathbf {z}})|_1&= \varSigma (\mathbf {v}_n) - \varSigma (\mathbf {u}^*_n(\widehat{\mathbf {z}})) \\&= \varSigma (\mathbf {v}_n) - \varSigma (\mathbf {u}^*_n(\mathbf {z})) \le |\mathbf {v}_n - \mathbf {u}^*_n(\mathbf {z})|_1 \end{aligned}$$

  with the last line a consequence of the triangle inequality for norms, taking into account that $\varSigma (\cdot ) = | \cdot |_1$ for all valid charging trajectories. Then

  $$\begin{aligned} |\mathbf {u}^*_n(\widehat{\mathbf {z}}) - \mathbf {u}^*_n(\mathbf {z})|_1 \le |\mathbf {v}_n - \mathbf {u}^*_n(\mathbf {z})|_1 + |\mathbf {v}_n - \mathbf {u}^*_n(\widehat{\mathbf {z}})|_1 \le 2 |\mathbf {v}_n - \mathbf {u}^*_n(\mathbf {z})|_1. \end{aligned}$$

  (2.41)

• (iii)   $\varSigma (\mathbf {v}_n) < \varSigma (\mathbf {u}^*_n(\mathbf {z}))$. A similar argument to (ii) can be used to show that (2.41) holds in this case.    $\blacksquare$

2.1.2 2.11.2    Proof of Theorem 2.3

Proof of Theorem 2.3. First notice that $\mathfrak {r}_t-\widehat{\mathfrak {r}}_t = \frac{1}{\bar{\varLambda }} (z_t-\widehat{z}_t)$. Therefore,

$$\begin{aligned} | p(\mathfrak {r}_t) - p( \widehat{\mathfrak {r}}_t ) |&\le \underset{\mathfrak {r} \in [\mathfrak {r}_{min},\mathfrak {r}_{max}]}{\max } \frac{dp(\mathfrak {r})}{d\mathfrak {r}} \times | \mathfrak {r}_t - \widehat{\mathfrak {r}}_t | = \underset{\mathfrak {r} \in [\mathfrak {r}_{min},\mathfrak {r}_{max}]}{\max } \frac{dp(\mathfrak {r})}{d\mathfrak {r}} \times \frac{1}{\bar{\varLambda }} | z_t - \widehat{z}_t | \\&\le 2 \delta | z_t - \widehat{z}_t | \end{aligned}$$

where the final inequality follows directly from (2.25). This result, together with a similar argument in terms of $\underset{\mathfrak {r} \in [\mathfrak {r}_{min},\mathfrak {r}_{max}]}{\min } \frac{dp(\mathfrak {r})}{d\mathfrak {r}}$ gives,

$$\begin{aligned} \frac{1}{2a} |{z}_t - \widehat{z}_t| \le \frac{1}{2 \delta } \big | p(\mathfrak {r}_t) - p(\widehat{\mathfrak {r}}_t) \big | \le |{z}_t - \widehat{z}_t|. \end{aligned}$$

(2.42)

Manipulation of (2.42) results in,

$$\begin{aligned} \left( 1 -\frac{1}{2a} \right) |{z}_t - \widehat{z}_t| \ge |{z}_t - \widehat{z}_t| - \frac{1}{2 \delta } \big | p(\mathfrak {r}_t) - p(\widehat{\mathfrak {r}}_t) \big | \ge 0. \end{aligned}$$

(2.43)

Because $p(\mathfrak {r}_t)$ is strictly increasing with $\mathfrak {r}_t = \frac{1}{\bar{\varLambda }} ( \bar{d}_t + z_t)$, (2.43) can be rewritten

$$ \left( 1 -\frac{1}{2a} \right) |{z}_t - \widehat{z}_t| \ge \big | ({z}_t - \widehat{z}_t) - \frac{1}{2 \delta } \big ( p(\mathfrak {r}_t) - p(\widehat{\mathfrak {r}}_t) \big ) \big | \ge 0. $$

This inequality, in conjunction with (2.39) and (2.40) of Lemma 2.4, gives

$$ |\mathbf {u}^*_n(\mathbf {z}) - \mathbf {u}^*_n(\widehat{\mathbf {z}})|_1 \le \big (2-\frac{1}{a} \big ) |\mathbf {z} - \widehat{\mathbf {z}}|_1 $$

and hence

$$ |\overline{\mathbf {u}}^*(\mathbf {z}) - \overline{\mathbf {u}}^*(\widehat{\mathbf {z}}) |_1 \le \big ( 2-\frac{1}{a} \big ) | \mathbf {z} - \widehat{\mathbf {z}} |_1. $$

Since $\frac{1}{2}<a<1$, it follows that $\overline{\mathbf {u}}^* (\mathbf {z})$ is a contraction mapping with respect to $\mathbf {z}$. It may be concluded from the contraction mapping theorem [21] that the infinite population of PEVs possesses a unique fixed point which is the unique Nash equilibrium for the infinite population charging coordination system.    $\blacksquare$

2.1.3 2.11.3    Proof of Theorem 2.4

Proof of Theorem 2.4. Consider any pair of time instants $t,s \in \mathscr {T}$, and denote by $\mathscr {U}_n(\{t,s\})$ the set of charging strategies ${u}_{nt}$ and ${u}_{ns}$ that satisfy ${u}_{nt},{u}_{ns} \ge 0$ and ${u}_{nt}+{u}_{ns} \le \varGamma _n$. Let

$$\begin{aligned} a = \frac{u_{ns} - u_{nt}}{2}, \qquad b = \frac{u_{ns} + u_{nt}}{2} \end{aligned}$$

(2.44)

so that ${u}_{nt}={b}-{a}$, and ${u}_{ns}={b}+{a}$. It follows that $\mathscr {U}_n(\{t,s\})$ is equivalent to $\mathscr {S} \triangleq \big \{ (a,b); \, \text { s.t. } a \in [-b, b], \, b \in [0,\varGamma _n/2] \big \}$.

It is proceeded by writing the minimum of the cost function (2.18) as a Bellman equation [22]. To do so, define

$$\begin{aligned} V_n(\varGamma ^\prime , \mathscr {T}^\prime ) =&\underset{u_{n\widehat{t}}; {\widehat{t}} \in \mathscr {T}^\prime }{\min } \left\{ \sum _{{\widehat{t}} \in \mathscr {T}^\prime } \big ( p(\mathfrak {r}_{\widehat{t}})u_{n\widehat{t}}+\delta (u_{n\widehat{t}}-z_{\widehat{t}})^2 \big ) \right\} \\&\quad \begin{aligned} \text {s.t. } \quad&u_{n\widehat{t}} \le 0 \quad \text {for all } {\widehat{t}} \in \mathscr {T}^\prime \\&\sum _{{\widehat{t}} \in \mathscr {T}^\prime } u_{n\widehat{t}} = \varGamma ^\prime . \end{aligned} \end{aligned}$$

The minimum over the entire charging period $\mathscr {T}$ can then be written,

$$\begin{aligned} V_n(\varGamma _n, \mathscr {T}) = \underset{u_{nt}, u_{ns} \in \mathscr {U}_n(\{t,s\})}{\min } \Bigg \{&\sum _{{\widehat{t}} \in \{t,s\}} \big ( p(\mathfrak {r}_{\widehat{t}})u_{n\widehat{t}}+\delta (u_{n\widehat{t}}-z_{\widehat{t}})^2 \big ) \nonumber \\&\qquad \qquad + V_n \big ( \varGamma _n - (u_{nt}+u_{ns}), \mathscr {T} \setminus \{t,s \} \big ) \Bigg \}. \end{aligned}$$

(2.45)

In terms of a and b defined at (2.44), this becomes

$$\begin{aligned} V_n(\varGamma _n, \mathscr {T}) = \underset{(a,b) \in \mathscr {S}}{\min } \left\{ 2 \delta \left( {a} - \frac{1}{2}({z}_s - {z}_t) + \frac{1}{4\delta } (p(\mathfrak {r}_s) - p(\mathfrak {r}_t)) \right) ^2 + g({b}) \right\} \end{aligned}$$

(2.46)

where g(b) is an expression in b that is unrelated to a. Let $a^*_n$ and $b^*_n$ denote the values of a and b associated with the optimal strategies ${u}^*_{nt}$ and ${u}^*_{ns}$. Then by (2.46), $a^*_n$ is a function of $b^*_n$ that satisfies,

$$\begin{aligned} a^*_n(b^*_n) = \underset{{a} \in [-b^*_n,b^*_n]}{\text {argmin}} \big \{ ({a} - \zeta )^2 \big \}, \end{aligned}$$

(2.47)

with

$$\begin{aligned} \zeta \equiv \frac{1}{2}({z}_s - {z}_t) - \frac{1}{4 \delta } (p(\mathfrak {r}_s) - p(\mathfrak {r}_t)). \end{aligned}$$

(2.48)

It follows from (2.47) that

$$\begin{aligned}&0 < a_n^* \le \zeta \qquad \text {if } \zeta > 0 \end{aligned}$$

(2.49a)

$$\begin{aligned}&a_n^* = 0 \qquad \qquad \,\, \text {if } \zeta = 0 \end{aligned}$$

(2.49b)

$$\begin{aligned}&\zeta \le a_n^*< 0 \qquad \text {if } \zeta < 0. \end{aligned}$$

(2.49c)

(i.1), first part of (2.26a). This result can be shown by establishing a contradiction. Suppose there exist two time instants t and s, such that $\bar{d}_t \le d_s$ and $z_t < z_s$, which implies $\mathfrak {r}_t < \mathfrak {r}_s$. Since $p(\mathfrak {r})$ is strictly increasing on $\mathfrak {r}$, $p(\mathfrak {r}_t) < p(\mathfrak {r}_s)$, and so from (2.48), $\zeta < \frac{1}{2}({z}_s - {z}_t)$. It follows from (2.49) that $a^*_n < \frac{1}{2} ({z}_s - {z}_t)$. Hence, ${u}^*_{ns}(\mathbf {z}) - {u}^*_{nt}(\mathbf {z}) = 2a^*_n < {z}_s - {z}_t$, for all $n \in \mathscr {N}$, which implies $\overline{\mathbf {u}}^*_s(\mathbf {z}) - \overline{\mathbf {u}}^*_t (\mathbf {z}) < {z}_s - {z}_t$. However $\mathbf {u}^*$ is a Nash equilibrium, so $\overline{\mathbf {u}}^*(\mathbf {z}) = \mathbf {z}$, hence a contradiction.

(i.2), second part of (2.26a). Proof by contradiction is again used. Suppose there exist two time instants t and s, such that $\bar{d}_t+{z}_t > d_s+{z}_s$ when $d_{t} \le d_{s}$. It follows that $p(\mathfrak {r}_t) > p(\mathfrak {r}_s)$, and so from (2.48), $\zeta > \frac{1}{2}({z}_s - {z}_t)$. But from (i.1), ${z}_s - {z}_t \le 0$, so it follows from (2.49) that $u^*_{ns} (\mathbf {z}) - u^*_{nt} (\mathbf {z}) = 2a^{*}_n > {z}_s - {z}_t$, for all $n \in \mathscr {N}$, which implies $\overline{\mathbf {u}}^*_s (\mathbf {z}) - \overline{\mathbf {u}}^*_t (\mathbf {z}) > {z}_s - {z}_t$. However $\mathbf {u}^*$ is a Nash equilibrium, so $\overline{\mathbf {u}}^*(\mathbf {z}) = \mathbf {z}$, hence a contradiction.

(i.3), third part of (2.26a). Again consider two time instants t and s, where $\bar{d}_t \le d_s$. From (i.1) and (i.2), it obtains that $z_t \ge z_s$ and $p(\mathfrak {r}_t) \le p(\mathfrak {r}_s)$ respectively. Therefore (2.48) implies $\zeta \le 0$, so it may be concluded from (2.49) that $u^*_{ns} - u^*_{nt} = 2a^{*}_n \le 0$. Hence $u^*_{nt} \ge u^*_{ns}$ as desired.

(ii.1), first part of (2.26b). Proof by contradiction will again be used to establish this result. Suppose there exist two time instants $t,s \in \widehat{\mathscr {T}}$ such that $d_s+z_s \ne \bar{d}_t+z_t$. Without lose of generality, assume $d_s+z_s = \bar{d}_t+z_t + \eta$, for some $\eta >0$. Then there exist n and $\widehat{\eta } \ge {\eta }$, such that $d_s+u^*_{ns} = \bar{d}_t+u^*_{nt} + \widehat{\eta }$. By the definition of $\widehat{\mathscr {T}}$, $u^*_{ns} > 0$ for $s \in \widehat{\mathscr {T}}$ and all $n \in \mathscr {N}$. Therefore there exists a sufficiently small $\varepsilon >0$ such that $u^*_{ns}-\varepsilon > 0$.

Consider a revised charging strategy $\mathbf {u}^{\varepsilon }_n$, with

$$\begin{aligned} {u}^{\varepsilon }_{nt}&= u^*_{nt}+\varepsilon \\ {u}^{\varepsilon }_{ns}&= u^*_{ns}-\varepsilon \\ {u}^{\varepsilon }_{n\widehat{t}}&= u^*_{n\widehat{t}} \quad \text {for } {\widehat{t}} \in \mathscr {T} \setminus \{t,s \}. \end{aligned}$$

For the cost function $J_n(\mathbf {u}_n; \mathbf {z})$ defined at (2.18), it follows that

$$ J_n(\mathbf {u}^{\varepsilon }_n; \mathbf {z}) - {J}_n(\mathbf {u}^*_n; \mathbf {z}) = \varepsilon (p(\mathfrak {r}_t)-p(\mathfrak {r}_s)) + 2\delta \varepsilon ((u^*_{nt}-u^*_{ns})-(z_t-z_s)) + 2\delta \varepsilon ^2. $$

Notice that $(u^*_{nt}-u^*_{ns})-(z_t-z_s)={\eta }-\widehat{\eta } \le 0$. Also, because $\bar{d}_t+z_t < d_s+z_s$ and $p(\mathfrak {r})$ is strictly increasing, $p(\mathfrak {r}_t) - p(\mathfrak {r}_s) < 0$. Therefore $J_n( \mathbf {u}^{\varepsilon }_n; \mathbf {z}) < J_n(\mathbf {u}^*_n; \mathbf {z})$ for sufficiently small $\varepsilon >0$. However, $\mathbf {u}^*$ is a Nash equilibrium, and therefore minimizes $J_n(\mathbf {u}_n; \mathbf {z})$. Hence a contradiction.

(ii.2), second part of (2.26b). The total energy delivered to PEV n over the period $\widehat{\mathscr {T}}$ by the optimal charging strategy $\mathbf {u}^*$ is given by $\varSigma _{{\widehat{t}} \in \widehat{\mathscr {T}}} u_{n\widehat{t}}^* = \omega _n^* > 0$, for every $n \in \mathscr {N}$. Provided fixed energy $\omega _n^*$ is delivered over $\widehat{\mathscr {T}}$, variation of the trajectory $\{ u_{n\widehat{t}}; {\widehat{t}} \in \widehat{\mathscr {T}} \}$ has no influence on the cost over the balance of the charging period $\mathscr {T} \setminus \widehat{\mathscr {T}}$. The optimal choice for $\{ u_{n\widehat{t}}; {\widehat{t}} \in \widehat{\mathscr {T}} \}$ is therefore given by,

$$\begin{aligned} \underset{u_{n\widehat{t}},{\widehat{t}} \in \widehat{\mathscr {T}}}{\min }&\sum _{{\widehat{t}} \in \widehat{\mathscr {T}}} \big ( p(\mathfrak {r}_{\widehat{t}}) u_{n\widehat{t}} + \delta (u_{n\widehat{t}}-z_{\widehat{t}})^2 \big ) \end{aligned}$$

(2.50)

$$\begin{aligned}&\text {s.t.} \quad \varSigma _{{\widehat{t}} \in \widehat{\mathscr {T}}} u_{n\widehat{t}} = \omega _n^*. \end{aligned}$$

(2.51)

According to (ii.1), $\bar{d}_t+z_t=d_s+z_s$, for all $t,s\in \widehat{\mathscr {T}}$. Therefore the electricity charging price $p(\mathfrak {r}_{\widehat{t}})$, with $\mathfrak {r}_{\widehat{t}}=\frac{1}{\bar{\varLambda }}(\bar{d}_{\widehat{t}}+z_{\widehat{t}})$, is a constant p for all ${\widehat{t}} \in \widehat{\mathscr {T}}$. This allows the cost function (2.50) to be rewritten as $p \omega + \delta \sum _{{\widehat{t}} \in \widehat{\mathscr {T}}} (u_{n\widehat{t}}-z_{\widehat{t}})^2$, so the minimum cost can be found from,

$$ \underset{u_{n\widehat{t}},{\widehat{t}} \in \widehat{\mathscr {T}}}{\min } \sum _{{\widehat{t}} \in \widehat{\mathscr {T}}} ( u_{n\widehat{t}}-z_{\widehat{t}})^2 $$

subject to (2.51). Using Lagrange multipliers, optimality is achieved when all $u_{n\widehat{t}}^* - z_{\widehat{t}}, {\widehat{t}} \in \widehat{\mathscr {T}}$ are equal. In conjunction with (ii.1), this gives $\bar{d}_{\widehat{t}} + u_{n\widehat{t}}^* = \theta _n$ for all ${\widehat{t}} \in \widehat{\mathscr {T}}$.

   $\blacksquare$