EPIRK-W and EPIRK-K Time Discretization Methods

Abstract

Exponential integrators are special time discretization methods where the traditional linear system solves used by implicit schemes are replaced with computing the action of matrix exponential-like functions on a vector. A very general formulation of exponential integrators is offered by the Exponential Propagation Iterative methods of Runge–Kutta type (EPIRK) family of schemes. The use of Jacobian approximations is an important strategy to drastically reduce the overall computational costs of implicit schemes while maintaining the quality of their solutions. This paper extends the EPIRK class to allow the use of inexact Jacobians as arguments of the matrix exponential-like functions. Specifically, we develop two new families of methods: EPIRK-W integrators that can accommodate any approximation of the Jacobian, and EPIRK-K integrators that rely on a specific Krylov-subspace projection of the exact Jacobian. Classical order conditions theories are constructed for these families. Practical EPIRK-W methods of order three and EPIRK-K methods of order four are developed. Numerical experiments indicate that the methods proposed herein are computationally favorable when compared to a representative state-of-the-art exponential integrator, and a Rosenbrock–Krylov integrator.

Appendices

A Derivation of K-Methods

The epirkk method is derived from epirkw method, where a specific Krylov-subspace approximation of the Jacobian is used instead of an arbitrary approximate Jacobian as admitted by the W-method. We start the derivation by first stating the general form of the epirkw method:

$$\begin{aligned} \mathbf {Y}_i&= \mathbf {y}_{n} + a_{i,1}\, \varvec{\psi }_{i,1}(g_{i,1}\,\,h\,\mathbf {A}_n)\, h\mathbf {f}(\mathbf {y}_{n}) + \displaystyle \sum _{j = 2}^{i} a_{i,j}\, \varvec{\psi }_{i,j}(g_{i,j}\,h\,\mathbf {A}_{n})\, h\Delta ^{(j-1)}\mathbf {r}(\mathbf {y}_{n}),\nonumber \\&\quad i = 1, \ldots , s - 1, \nonumber \\ \mathbf {y}_{n+1}&= \mathbf {y}_{n}\, + b_{1}\, \varvec{\psi }_{s,1}(g_{s,1}\,h\,\mathbf {A}_{n})\, h\mathbf {f}(\mathbf {y}_{n}) + \displaystyle \sum _{j = 2}^{s} b_{j}\, \varvec{\psi }_{s,j}(g_{s,j}\,h\,\mathbf {A}_{n})\, h\Delta ^{(j-1)}\mathbf {r}(\mathbf {y}_{n}). \end{aligned}$$

In the above equation, the $\varvec{\psi }$ function is as defined in Eq. (8) and the following simplifying assumption is made about it:

$$\begin{aligned} \varvec{\psi }_{i,j}(z)= \varvec{\psi }_{j}(z) = \displaystyle \sum _{k=1}^{j} p_{j,k}\, \varvec{\varphi }_k(z), \end{aligned}$$

where $\varvec{\varphi }$ function is as defined in Eqs. (5) and (6). Additionally, the remainder function ($\mathbf {r}(\mathbf {y})$) and the forward difference operator ($\Delta ^{(j)}\mathbf {r}(\mathbf {Y}_i)$) are defined accordingly below:

$$\begin{aligned} \mathbf {r}(\mathbf {y})= & {} \mathbf {f}(\mathbf {y}) - \mathbf {f}(\mathbf {y}_{n}) - \mathbf {A}_{n} \, (\mathbf {y} - \mathbf {y}_{n}),\nonumber \\ \Delta ^{(j)}\mathbf {r}(\mathbf {Y}_i)= & {} \Delta ^{(j - 1)}\mathbf {r}(\mathbf {Y}_{i+1}) - \Delta ^{(j - 1)}\mathbf {r}(\mathbf {Y}_{i}),\nonumber \\ \Delta ^{(1)}\mathbf {r}(\mathbf {Y}_i)= & {} \mathbf {r}(\mathbf {Y}_{i+1}) - \mathbf {r}(\mathbf {Y}_i). \end{aligned}$$

(33)

The K-method uses a specific Krylov-subspace based approximation of the Jacobian ($\mathbf {J}_{n}$). An M-dimensional Krylov-subspace is built as,

$$\begin{aligned} \mathcal {K}_M = \text {span}\{\mathbf {f}_n, \mathbf {J}_{n}\mathbf {f}_n, \mathbf {J}_{n}^2\mathbf {f}_n, \ldots , \mathbf {J}_{n}^{M-1}\mathbf {f}_n\}, \end{aligned}$$

(34)

whose basis is the orthonormal matrix $\mathbf {V}$ and $\mathbf {H}$ is the upper-Hessenberg matrix obtained from Arnoldi iteration defined as

$$\begin{aligned} \mathbf {H}= \mathbf {V}^{T} \mathbf {J}_{n} \,\mathbf {V}. \end{aligned}$$

(35)

The corresponding Krylov-subspace based approximation of the Jacobian is built as

$$\begin{aligned} \mathbf {A}_{n} = \mathbf {V}\,\mathbf {H}\,\mathbf {V}^{T} = \mathbf {V}\mathbf {V}^{T} \mathbf {J}_{n} \,\mathbf {V}\mathbf {V}^{T}. \end{aligned}$$

(36)

The use of Krylov-subspace based approximation of the Jacobian reduces the $\varvec{\varphi }$ and $\varvec{\psi }$ function in accordance with Lemmas 2 and 3 to the following

$$\begin{aligned} \varvec{\varphi }_k(h\,\gamma \,\mathbf {A}_{n})= & {} \frac{1}{k!} \left( \mathbf {I} - \mathbf {V}\mathbf {V}^{T}\right) + \mathbf {V}\varvec{\varphi }_k(h\,\gamma \,\mathbf {H}) \mathbf {V}^{T}, \end{aligned}$$

(37)

$$\begin{aligned} \varvec{\psi }_{j}(h\gamma \mathbf {A}_{n})= & {} \widetilde{p}_j \left( \mathbf {I} - \mathbf {V}\mathbf {V}^{T}\right) + \mathbf {V}\,\varvec{\psi }_{j}(h\gamma \mathbf {H}) \mathbf {V}^{T}, \end{aligned}$$

(38)

where $\widetilde{p}_j$ is defined as

$$\begin{aligned} \widetilde{p}_j = \displaystyle \sum _{k=1}^{j}\frac{p_{j,k}}{k!}. \end{aligned}$$

(39)

In order to derive the reduced stage formulation of the epirkk method we need to resolve the vectors in the formulation into components in the Krylov-subspace and orthogonal to it. Repeating the splittings from the main text:

• Splitting the internal stage vectors noting that $\mathbf {Y}_0 \equiv \mathbf {y}_n$:

  $$\begin{aligned} \mathbf {Y}_i = \mathbf {V}\varvec{\lambda }_i + \mathbf {Y}_i^{\bot } \qquad \text {where} \quad \mathbf {V}^{T} \mathbf {Y}_i = \varvec{\lambda }_i, \quad \left( \mathbf {I} - \mathbf {V}\mathbf {V}^{T}\right) \,\mathbf {Y}_i = \mathbf {Y}_i^{\bot }. \end{aligned}$$

  (40)

• Splitting the right-hand side function evaluated at internal stage vectors while noting that $\mathbf {f}_0 \equiv \mathbf {f}(\mathbf {y}_n)$:

  $$\begin{aligned} \mathbf {f}_i := \mathbf {f}(\mathbf {Y}_i) = \mathbf {V}\varvec{\eta }_i + \mathbf {f}_i^{\bot } \qquad \text {where} \quad \mathbf {V}^{T} \mathbf {f}_i = \varvec{\eta }_i, \quad \left( \mathbf {I} - \mathbf {V}\mathbf {V}^{T}\right) \mathbf {f}_i = \mathbf {f}_i^{\bot }. \end{aligned}$$

  (41)

• Splitting the non-linear Taylor remainder of the right-hand side functions:

  $$\begin{aligned}&\mathbf {r}(\mathbf {Y}_i) = \mathbf {f}(\mathbf {Y}_i) - \mathbf {f}(\mathbf {y}_{n}) - \mathbf {A}_{n}\, (\mathbf {Y}_i - \mathbf {y}_{n}) = \mathbf {f}_i - \mathbf {f}_0 - \mathbf {V}\, \mathbf {H}\, \mathbf {V}^{T}\, (\mathbf {Y}_i - \mathbf {y}_{n}), \nonumber \\&\quad \text {where} \quad \mathbf {V}^{T}\, \mathbf {r}(\mathbf {Y}_i) = \varvec{\eta }_i - \varvec{\eta }_0 - \mathbf {H}\, (\varvec{\lambda }_i - \varvec{\lambda }_0), \nonumber \\&\quad \left( \mathbf {I} - \mathbf {V}\mathbf {V}^{T}\right) \, \mathbf {r}(\mathbf {Y}_i) = \mathbf {f}_i^{\bot } - \mathbf {f}_0^{\bot }. \end{aligned}$$

  (42)

• Splitting the forward differences of the non-linear remainder terms:

  $$\begin{aligned}&\mathbf {\widetilde{r}}_{(j-1)}:=\Delta ^{(j-1)}\mathbf {r}(\mathbf {y}_{n}) = \mathbf {V}\, \mathbf {d}_{(j-1)} + \mathbf {\widetilde{r}}_{(j-1)}^{\bot }, \nonumber \\&\quad \text {where} \quad \mathbf {V}^{T}\,\mathbf {\widetilde{r}}_{(j-1)} = \mathbf {d}_{(j-1)}, \nonumber \\&\quad \left( \mathbf {I} - \mathbf {V}\mathbf {V}^{T}\right) \, \mathbf {\widetilde{r}}_{(j-1)} = \mathbf {\widetilde{r}}_{(j-1)}^{\bot }. \end{aligned}$$

  (43)

Using these each internal stage of the epirkk method can be expressed as below:

$$\begin{aligned} \mathbf {V}\varvec{\lambda }_i + \mathbf {Y}_i^{\bot }= & {} \mathbf {y}_{n}\, + h\, a_{i,1} \bigg (\widetilde{p_1} \left( \mathbf {I} - \mathbf {V}\mathbf {V}^{T}\right) + \mathbf {V}\,\varvec{\psi }_{1}(h\,g_{i,1}\, \mathbf {H}) \mathbf {V}^{T}\bigg ) \bigg (\mathbf {V}\varvec{\eta }_0 + \mathbf {f}_0^{\bot }\bigg ) \nonumber \\&+\, \displaystyle \sum _{j = 2}^{i} h\, a_{i,j}\, \bigg (\widetilde{p}_j \left( \mathbf {I} - \mathbf {V}\mathbf {V}^{T}\right) + \mathbf {V}\,\varvec{\psi }_{j}(h\,g_{i,j}\, \mathbf {H}) \mathbf {V}^{T}\bigg ) \bigg (\mathbf {V}\mathbf {d}_{(j-1)} + \mathbf {\widetilde{r}}_{(j-1)}^{\bot }\bigg ) \nonumber \\= & {} \mathbf {y}_{n}\, + h\, a_{i,1} \bigg (\widetilde{p_1} \, \mathbf {f}_0^{\bot } + \mathbf {V}\,\varvec{\psi }_{1}(h\,g_{i,1}\, \mathbf {H}) \varvec{\eta }_0\bigg ) \nonumber \\&+\, \displaystyle \sum _{j = 2}^{i} h\, a_{i,j}\, \bigg (\widetilde{p}_j \, \mathbf {\widetilde{r}}_{(j-1)}^{\bot } + \mathbf {V}\,\varvec{\psi }_{j}(h\,g_{i,j}\, \mathbf {H}) \, \mathbf {d}_{(j-1)}\bigg ). \end{aligned}$$

(44)

The reduced stage formulation of the epirkk method is obtained by multiplying the above equation by $\mathbf {V}^{T}$ from the left

$$\begin{aligned} \varvec{\lambda }_i= & {} \mathbf {V}^{T} \mathbf {y}_{n}\, + h \, a_{i,1} \varvec{\psi }_{1}(h\,g_{i,1}\,\mathbf {H}) \varvec{\eta }_0 + \displaystyle \sum _{j = 2}^{i} h\, a_{i,j}\, \varvec{\psi }_{j}(h\,g_{i,j}\, \mathbf {H}) \, \mathbf {d}_{(j-1)} \nonumber \\= & {} \varvec{\lambda }_0 + h \, a_{i,1} \varvec{\psi }_{1}(h\,g_{i,1}\,\mathbf {H}) \varvec{\eta }_0 + \displaystyle \sum _{j = 2}^{i} h\, a_{i,j}\, \varvec{\psi }_{j}(h\,g_{i,j}\, \mathbf {H}) \, \mathbf {d}_{(j-1)}. \end{aligned}$$

(45)

And the full stage vector can be recovered by first computing the reduced stage vector and adding the orthogonal piece $\mathbf {Y}_i^{\bot }$ obtained when multiplying Eq. (44) by $\left( \mathbf {I} - \mathbf {V}\mathbf {V}^{T}\right)$

$$\begin{aligned} \mathbf {Y}_i^{\bot }= & {} \left( \mathbf {I} - \mathbf {V}\mathbf {V}^{T}\right) \mathbf {y}_{n}\, + h \, a_{i,1} \, \widetilde{p_1} \, \mathbf {f}_0^{\bot } + \displaystyle \sum _{j = 2}^{i} h\, a_{i,j}\, \widetilde{p}_j \, \mathbf {\widetilde{r}}_{(j-1)}^{\bot } \nonumber \\= & {} (\mathbf {y}_{n}\, - \mathbf {V}\varvec{\varvec{\lambda }}_0) + h \, a_{i,1} \, \widetilde{p_1} \, \mathbf {f}_0^{\bot } + \displaystyle \sum _{j = 2}^{i} h\, a_{i,j}\, \widetilde{p}_j \, \mathbf {\widetilde{r}}_{(j-1)}^{\bot }. \end{aligned}$$

(46)

The final stage can also be written in the above form with multipliers $b_i$ in place of $a_{ij}$. Notice that the expensive computations are performed in the reduced space, i.e. the $\varvec{\psi }$ function is computed in the reduced space instead of the full space offering potential computational savings. In the above equations, the quantities $\mathbf {d}_{(j-1)}$ and $\mathbf {\widetilde{r}}_{(j-1)}^{\bot }$ can be shown to be

$$\begin{aligned} \mathbf {d}_{(j-1)}= & {} \displaystyle \sum _{k=0}^{j-1} \bigg ((-1)^k {j-1 \atopwithdelims ()k} \varvec{\eta }_{j-1-k} - \mathbf {H}\bigg ((-1)^k {j-1 \atopwithdelims ()k} \varvec{\varvec{\lambda }}_{j-1 -k}\bigg )\bigg ), \end{aligned}$$

(47)

$$\begin{aligned} \mathbf {\widetilde{r}}_{(j-1)}^{\bot }= & {} \displaystyle \sum _{k=0}^{j-1} \bigg ((-1)^k {j-1 \atopwithdelims ()k} \mathbf {f}_{j-1-k}^{\bot }\bigg ), \end{aligned}$$

(48)

as is done in the following “Appendix”.

B Proofs

In order to prove Eqs. (23a) and (23b), we start with the definition of the remainder function and forward difference.

$$\begin{aligned} \mathbf {r}(\mathbf {y}) = \mathbf {f}(\mathbf {y}) - \mathbf {f}(\mathbf {y}_{n}) - \mathbf {A}_{n} (\mathbf {y} - \mathbf {y}_{n}), \end{aligned}$$

(49)

$$\begin{aligned} \Delta ^{(j)}\mathbf {r}(\mathbf {Y}_i)= & {} \Delta ^{(j - 1)}\mathbf {r}(\mathbf {Y}_{i+1}) - \Delta ^{(j - 1)}\mathbf {r}(\mathbf {Y}_{i}), \end{aligned}$$

(50a)

$$\begin{aligned} \Delta ^{(1)}\mathbf {r}(\mathbf {Y}_i)= & {} \mathbf {r}(\mathbf {Y}_{i+1}) - \mathbf {r}(\mathbf {Y}_i). \end{aligned}$$

(50b)

Lemma 4

$\Delta ^{(j)}\mathbf {r}(\mathbf {Y}_i) = \displaystyle \sum _{k=0}^{j} (-1)^k {j \atopwithdelims ()k} \mathbf {r}(\mathbf {Y}_{i+j-k})$.

Proof

In order to prove the lemma, we resort to mathematical induction. Base case $j = 1$,

$$\begin{aligned} \Delta ^{(1)}\mathbf {r}(\mathbf {Y}_i)= & {} \displaystyle \sum _{k=0}^{1} (-1)^k {1 \atopwithdelims ()k} \mathbf {r}(\mathbf {Y}_{i+1-k})\\= & {} \mathbf {r}(\mathbf {Y}_{i+1}) - \mathbf {r}(\mathbf {Y}_i). \end{aligned}$$

The base case is true by definition. We now assume that the proposition holds true for all j up to $k-1$. We have,

$$\begin{aligned} \Delta ^{(k-1)}\mathbf {r}(\mathbf {Y}_i)= & {} \displaystyle \sum _{l=0}^{k-1} (-1)^l {k-1 \atopwithdelims ()l} \mathbf {r}(\mathbf {Y}_{i+k-1-l}), \end{aligned}$$

(51)

$$\begin{aligned} \Delta ^{(k-1)}\mathbf {r}(\mathbf {Y}_{i+1})= & {} \displaystyle \sum _{l=0}^{k-1} (-1)^l {k-1 \atopwithdelims ()l} \mathbf {r}(\mathbf {Y}_{i+k-l}). \end{aligned}$$

(52)

Then for $j = k$,

$$\begin{aligned} \Delta ^{(k)}\mathbf {r}(\mathbf {Y}_i)= & {} \Delta ^{(k - 1)}\mathbf {r}(\mathbf {Y}_{i+1}) - \Delta ^{(k - 1)}\mathbf {r}(\mathbf {Y}_{i}) \\= & {} \displaystyle \sum _{l=0}^{k-1}(-1)^l {k-1 \atopwithdelims ()l} \mathbf {r}(\mathbf {Y}_{i+k-l}) - \displaystyle \sum _{l=0}^{k-1}(-1)^l {k-1 \atopwithdelims ()l} \mathbf {r}(\mathbf {Y}_{i+k -1 -l})\\= & {} \mathbf {r}(\mathbf {Y}_{i+k}) + \displaystyle \sum _{l=1}^{k-1}(-1)^l {k-1 \atopwithdelims ()l} \mathbf {r}(\mathbf {Y}_{i+k-l}) \\&-\, \displaystyle \sum _{l=0}^{k-2}(-1)^l {k-1 \atopwithdelims ()l} \mathbf {r}(\mathbf {Y}_{i+k -1 -l}) + (-1)^k \mathbf {r}(\mathbf {Y}_i). \end{aligned}$$

We perform a change of variable for the second summation, $m = l + 1 \; \implies l = (m - 1)$,

$$\begin{aligned} \Delta ^{(k)}\mathbf {r}(\mathbf {Y}_i)= & {} \mathbf {r}(\mathbf {Y}_{i+k}) + \displaystyle \sum _{l=1}^{k-1}(-1)^l {k-1 \atopwithdelims ()l} \mathbf {r}(\mathbf {Y}_{i+k-l}) \\&-\, \displaystyle \sum _{m=1}^{k-1}(-1)^{m-1} {k-1 \atopwithdelims ()m - 1} \mathbf {r}(\mathbf {Y}_{i+k -m}) + (-1)^k \mathbf {r}(\mathbf {Y}_i) \\= & {} \mathbf {r}(\mathbf {Y}_{i+k}) + \displaystyle \sum _{l=1}^{k-1}(-1)^l {k-1 \atopwithdelims ()l} \mathbf {r}(\mathbf {Y}_{i+k-l}) \\&+\, \displaystyle \sum _{m=1}^{k-1}(-1)^{m} {k-1 \atopwithdelims ()m - 1} \mathbf {r}(\mathbf {Y}_{i+k -m}) + (-1)^k \mathbf {r}(\mathbf {Y}_i). \end{aligned}$$

The summations run between the same start and end indices, and can be collapsed.

$$\begin{aligned} \Delta ^{(k)}\mathbf {r}(\mathbf {Y}_i)= & {} \mathbf {r}(\mathbf {Y}_{i+k}) + \displaystyle \sum _{l=1}^{k-1}(-1)^l \bigg ({k-1 \atopwithdelims ()l} + {k-1 \atopwithdelims ()l - 1}\bigg ) \mathbf {r}(\mathbf {Y}_{i+k-l}) + (-1)^k \mathbf {r}(\mathbf {Y}_i). \end{aligned}$$

We use the identity,

$$\begin{aligned} {k-1 \atopwithdelims ()l} + {k-1 \atopwithdelims ()l - 1} = {k \atopwithdelims ()l}, \nonumber \end{aligned}$$

and arrive at the desired result,

$$\begin{aligned} \Delta ^{(k)}\mathbf {r}(\mathbf {Y}_i)= & {} \mathbf {r}(\mathbf {Y}_{i+k}) + \displaystyle \sum _{l=1}^{k-1}(-1)^l {k \atopwithdelims ()l} \mathbf {r}(\mathbf {Y}_{i+k-l}) + (-1)^k \mathbf {r}(\mathbf {Y}_i) \nonumber \\= & {} (-1)^0 {k \atopwithdelims ()0} \mathbf {r}(\mathbf {Y}_{i+k}) + \displaystyle \sum _{l=1}^{k-1}(-1)^l {k \atopwithdelims ()l} \mathbf {r}(\mathbf {Y}_{i+k-l}) + (-1)^k {k \atopwithdelims ()k} \mathbf {r}(\mathbf {Y}_i) \nonumber \\= & {} \displaystyle \sum _{l=0}^{k} (-1)^l {k \atopwithdelims ()l} \mathbf {r}(\mathbf {Y}_{i+k-l}). \end{aligned}$$

(53)

$\square$

Lemma 5

Given

$$\begin{aligned} \Delta ^{(j-1)}\mathbf {r}(\mathbf {y}_{n}) = \mathbf {V}\mathbf {d}_{(j-1)} + \mathbf {\widetilde{r}}_{(j-1)}^{\bot }, \end{aligned}$$

(54)

we need to prove

$$\begin{aligned} \mathbf {d}_{(j-1)}= & {} \displaystyle \sum _{k=0}^{j-1} \bigg ((-1)^k {j-1 \atopwithdelims ()k} \varvec{\eta }_{j-1-k} - \mathbf {H}\bigg ((-1)^k {j-1 \atopwithdelims ()k} \varvec{\varvec{\lambda }}_{j-1 -k}\bigg )\bigg ),\end{aligned}$$

(55)

$$\begin{aligned} \mathbf {\widetilde{r}}_{(j-1)}^{\bot }= & {} \displaystyle \sum _{k=0}^{j-1} \bigg ((-1)^k {j-1 \atopwithdelims ()k} \mathbf {f}_{j-1-k}^{\bot }\bigg ). \end{aligned}$$

(56)

Proof

We start with Lemma 4 where we have proven that

$$\begin{aligned} \Delta ^{(j)}\mathbf {r}(\mathbf {Y}_i) = \displaystyle \sum _{k=0}^{j} (-1)^k {j \atopwithdelims ()k} \mathbf {r}(\mathbf {Y}_{i+j-k}). \end{aligned}$$

We plugin the value $i = 0$ which corresponds to $\Delta ^{(j)}\mathbf {r}(\mathbf {Y}_0) \equiv \Delta ^{(j)}\mathbf {r}(\mathbf {y}_n)$ and we get,

$$\begin{aligned} \Delta ^{(j)}\mathbf {r}(\mathbf {y}_n) = \displaystyle \sum _{k=0}^{j} (-1)^k {j \atopwithdelims ()k} \mathbf {r}(\mathbf {Y}_{j-k}). \end{aligned}$$

WLOG replacing j by $j-1$ yields

$$\begin{aligned} \Delta ^{(j-1)}\mathbf {r}(\mathbf {y}_n) = \displaystyle \sum _{k=0}^{j-1} (-1)^k {j-1 \atopwithdelims ()k} \mathbf {r}(\mathbf {Y}_{j-1-k}). \end{aligned}$$

(57)

Since the left-hand side of Eq. (57) can be written as

$$\begin{aligned} \Delta ^{(j-1)}\mathbf {r}(\mathbf {y}_{n}) = \mathbf {V}\mathbf {d}_{(j-1)} + \mathbf {\widetilde{r}}_{(j-1)}^{\bot }, \end{aligned}$$

we have the following result

$$\begin{aligned} \mathbf {V}\mathbf {d}_{(j-1)} + \mathbf {\widetilde{r}}_{(j-1)}^{\bot } = \displaystyle \sum _{k=0}^{j-1} (-1)^k {j-1 \atopwithdelims ()k} \mathbf {r}(\mathbf {Y}_{j-1-k}). \end{aligned}$$

(58)

The remainder function $\mathbf {r}(\mathbf {Y}_i)$ is defined as

$$\begin{aligned} \mathbf {r}(\mathbf {Y}_i) = \mathbf {f}(\mathbf {Y}_i) - \mathbf {f}(\mathbf {y}_n) - \mathbf {A}_{n} * (\mathbf {Y}_i - \mathbf {y}_n). \end{aligned}$$

Plugging in the definition of remainder function in (58) and observing that $\mathbf {r}(\mathbf {y}_n) = 0$, we get

$$\begin{aligned} \mathbf {V}\mathbf {d}_{(j-1)} + \mathbf {\widetilde{r}}_{(j-1)}^{\bot }= & {} \displaystyle \sum _{k=0}^{j-2} (-1)^k {j-1 \atopwithdelims ()k} \bigg (\mathbf {f}(\mathbf {Y}_{j-1-k}) - \mathbf {f}(\mathbf {y}_n) - \mathbf {A}_{n} * \bigg (\mathbf {Y}_{j-1-k} - \mathbf {y}_n\bigg )\bigg )\nonumber \\= & {} \displaystyle \sum _{k=0}^{j-2} (-1)^k {j-1 \atopwithdelims ()k} \bigg (\mathbf {f}(\mathbf {Y}_{j-1-k}) - \mathbf {A}_{n} * \mathbf {Y}_{j-1-k}\bigg ) \nonumber \\&\displaystyle -\,\sum _{k=0}^{j-2} (-1)^k {j-1 \atopwithdelims ()k} \bigg (\mathbf {f}(\mathbf {y}_{n}) - \mathbf {A}_{n} * \mathbf {y}_{n}\bigg ). \end{aligned}$$

(59)

Consider the identity involving alternating sum and difference of binomial coefficients,

$$\begin{aligned} \displaystyle \sum _{i = 0}^{k} (-1)^{i} {k \atopwithdelims ()i} = 0. \end{aligned}$$

(60)

Applying (60) to (59) we get,

$$\begin{aligned} \mathbf {V}\mathbf {d}_{(j-1)} + \mathbf {\widetilde{r}}_{(j-1)}^{\bot }= & {} \displaystyle \sum _{k=0}^{j-2} (-1)^k {j-1 \atopwithdelims ()k} \bigg (\mathbf {f}(\mathbf {Y}_{j-1-k}) - \mathbf {f}(\mathbf {y}_n) - \mathbf {A}_{n} * \bigg (\mathbf {Y}_{j-1-k} - \mathbf {y}_n\bigg )\bigg )\nonumber \\= & {} \displaystyle \sum _{k=0}^{j-2} (-1)^k {j-1 \atopwithdelims ()k} \bigg (\mathbf {f}(\mathbf {Y}_{j-1-k}) - \mathbf {A}_{n} * \mathbf {Y}_{j-1-k}\bigg )\nonumber \\&- \,(-(-1)^{(j-1)}) \bigg (\mathbf {f}(\mathbf {y}_{n}) - \mathbf {A}_{n} * \mathbf {y}_{n}\bigg ) \nonumber \\= & {} \displaystyle \sum _{k=0}^{j-2} (-1)^k {j-1 \atopwithdelims ()k} \bigg (\mathbf {f}(\mathbf {Y}_{j-1-k}) - \mathbf {A}_{n} * \mathbf {Y}_{j-1-k}\bigg )\nonumber \\&+\, (-1)^{(j-1)} {j-1 \atopwithdelims ()j-1} \bigg (\mathbf {f}(\mathbf {y}_{n}) - \mathbf {A}_{n} * \mathbf {y}_{n}\bigg ) \nonumber \\= & {} \displaystyle \sum _{k=0}^{j-1} (-1)^k {j-1 \atopwithdelims ()k} \bigg (\mathbf {f}(\mathbf {Y}_{j-1-k}) - \mathbf {A}_{n} * \mathbf {Y}_{j-1-k}\bigg ). \end{aligned}$$

(61)

Additionally, since we are replacing the Jacobian by the approximation in the Krylov-subspace, i.e. $\mathbf {A}_{n} = \mathbf {V}\,\mathbf {H}\,\mathbf {V}^T$ we have

$$\begin{aligned} \mathbf {V}\mathbf {d}_{(j-1)} + \mathbf {\widetilde{r}}_{(j-1)}^{\bot } = \displaystyle \sum _{k=0}^{j-1} (-1)^k {j-1 \atopwithdelims ()k} \bigg (\mathbf {f}(\mathbf {Y}_{j-1-k}) - \mathbf {V}\mathbf {H}\mathbf {V}^{T} * \mathbf {Y}_{j-1-k}\bigg ). \end{aligned}$$

(62)

We get the expression for $\mathbf {d}_{(j-1)}$ by multiplying Eq. (62) from the left by $V^{T}$ and that for $\mathbf {\widetilde{r}}_{(j-1)}^{\bot }$ by multiplying by $(\mathbf {I} - \mathbf {V}\mathbf {V}^{T})$. $\square$