Picard iteration-based variable-order integrator with dense output employing algorithmic differentiation

Abstract

Motivated by the high accuracy requirements and the huge ratio of the largest to smallest time scales of Coulomb collision simulations of a considerable number of charges, we developed a novel numerical integration scheme, which uses algorithmic differentiation to produce variable, high-order integrators with dense output. We show that Picard iterations are not only a nice theoretical tool, but can also be successfully implemented to develop competitive integrators, especially when accuracies close to machine precision are required. The numerical integrators’ performance and applications to the electrostatic n-body problem are illustrated.

Appendix A: Composition commutation with truncated Taylor series extraction

Our proof of Lemma 1 employs the Faà Di Bruno formula [12], which is a generalization of the chain rule.

Theorem 1

(Faà Di Bruno) If f and g are functions with a sufficient number of derivatives, then

$$\frac{\mathrm{d}^{k}}{\mathrm{d}t^{k}} f(g(t))=\sum \frac{k!}{{\prod}_{i = 1}^{k} b_{i}!} f^{(j)}(g(t)) \prod\limits_{i = 1}^{k}\left( \frac{g^{(i)}(t)}{i!}\right)^{b_{i}} $$

where the sum is over all different solutions in nonnegative integers b₁ , . . . , b_kof b₁ + 2b₂ + ⋯ + kb_k = k andb₁ + b₂ + … + b_k = j.

A few sets are helpful for our proof of Lemma 1. Let U_k,j denote the set of nonnegative integer solutions of

$$b_{1}+ 2b_{2}+\dots+k b_{k}=k, $$

and

$$b_{1} + b_{2} + {\dots} + b_{k}=j; $$

and let V_m,k,j denote the set of nonnegative integer solutions of

$$b_{1}+ 2b_{2}+\dots+m b_{m}=k, $$

and

$$b_{1} + b_{2} + {\dots} + b_{m}=j. $$

Proof Proof of Lemma 1

Direct evaluation of $\mathscr {T}^{m}_{t,a}\left [\mathscr {T}^{m}_{t,g(a)}[f] \circ \mathscr {T}^{m}_{t,a}[g]\right ]$shows (5) holds. To begin, we substitute $\mathscr {T}_{t,a}^{m}[g]$for t in $\mathscr {T}^{m}_{t,g(a)}[f]$yielding

$$\mathscr{T}^{m}_{t,g(a)}[f] \circ \mathscr{T}^{m}_{t,a}[g]=\sum\limits_{j = 0}^{m} f^{(j)}(g(a)) \frac{(\mathscr{T}_{t,a}^{m}[g]-g(a))^{j}}{j!}. $$

Rearranging and expanding the notation for $\mathscr {T}_{t,a}^{m}[g]$gives

$$\mathscr{T}^{m}_{t,g(a)}[f] \circ \mathscr{T}^{m}_{t,a}[g]=\sum\limits_{j = 0}^{m} \frac{f^{(j)}(g(a))}{j!} \left( \sum\limits_{i = 0}^{m} g^{(i)}(a) \frac{(t-a)^{i}}{i!}-g(a)\right)^{j}. $$

The 0th term in the innersum is g(a), which may becanceled:

$$\mathscr{T}^{m}_{t,g(a)}[f] \circ \mathscr{T}^{m}_{t,a}[g]=\sum\limits_{j = 0}^{m} \frac{f^{(j)}(g(a))}{j!} \left( \sum\limits_{i = 1}^{m} g^{(i)}(a) \frac{(t-a)^{i}}{i!}\right)^{j}. $$

Applying themultinomial theorem to $\left ({\sum }_{i = 1}^{m} g^{(i)}(a) \frac {(t-a)^{i}}{i!}\right )^{j}$yields

$$\mathscr{T}^{m}_{t,g(a)}[f] \circ \mathscr{T}^{m}_{t,a}[g]\,=\,\sum\limits_{j = 0}^{m} \frac{f^{(j)}(g(a))}{j!} \!\underset{k\geq 1 }{\sum\limits_{V_{m,k,j}}} { \binom{j}{b_{1}, b_{2}, \ldots, b_{m}}} \!\prod\limits_{i = 1}^{m} \!\left( \frac{g^{(i)}(a)}{i!}(t\,-\,a)^{i}\right)^{\!b_{i}}\!. $$

Replacing the multinomial coefficients, substitutingk = b₁ + 2b₂ + ⋯ + mb_mand distributing$\frac {f^{(j)}(g(a))}{j!}$over the inner summationsgives

$$\mathscr{T}^{m}_{t,g(a)}[f] \circ \mathscr{T}^{m}_{t,a}[g]=\sum\limits_{j = 0}^{m} \underset{k\geq 1 }{\sum\limits_{V_{m,k,j}}} \frac{f^{(j)}(g(a))}{j!} \frac{j!}{{\prod}_{i = 1}^{m} b_{i} !} (t-a)^{k} \prod\limits_{i = 1}^{m} \left( \frac{g^{(i)}(a)}{i!}\right)^{b_{i}}. $$

Separatingthe j = 0term, canceling the j factorials, and rearranging, wehave

$$\mathscr{T}^{m}_{t,g(a)}[f] \circ \mathscr{T}^{m}_{t,a}[g]\,=\,f(g(a))+\sum\limits_{j = 1}^{m} \underset{k\geq 1 }{\sum\limits_{V_{m,k,j}}} \!(t-a)^{k} \frac{1}{{\prod}_{i = 1}^{m} b_{i} !} f^{(j)}(g(a))\! \prod\limits_{i = 1}^{m} \!\left( \frac{g^{(i)}(a)}{i!}\right)^{\!b_{i}}\!. $$

Composing $\mathscr {T}_{t,a}^{m}$with $\mathscr {T}^{m}_{t,g(a)}[f] \circ \mathscr {T}^{m}_{t,a}[g]$ truncates the terms of degree higher than m of polynomial$\mathscr {T}^{m}_{t,g(a)}[f] \circ \mathscr {T}^{m}_{t,a}[g]$int − a. Also,since k = b₁ + 2b₂ + ⋯ + mb_min V_m,k,j, it mustbe that b_i = 0when i > k fori = 1, 2, ⋯ , m. In this case, summingover V_m,k,j is equivalentto summing over U_k,j,

$$\begin{array}{@{}rcl@{}} \mathscr{T}^{m}_{t,a}\left[\mathscr{T}^{m}_{t,g(a)}[f] \circ \mathscr{T}^{m}_{t,a}[g]\right]&=&f(g(a)) + \sum\limits_{j = 1}^{m} \underset{m\geq k\geq 1}{\sum\limits_{U_{k,j}}} (t-a)^{k} \frac{1}{{\prod}_{i = 1}^{m} b_{i} !} f^{(j)}(g(a))\\ && \times \prod\limits_{i = 1}^{k} \left( \frac{g^{(i)}(a)}{i!}\right)^{b_{i}}. \end{array} $$

We can introduce canceling k factorials as below:

$$\begin{array}{@{}rcl@{}} \mathscr{T}^{m}_{t,a}\left[\mathscr{T}^{m}_{t,g(a)}[f] \circ \mathscr{T}^{m}_{t,a}[g]\right] &=& f(g(a))+\sum\limits_{j = 1}^{m} \underset{m\geq k\geq 1}{\sum\limits_{U_{k,j}}} \frac{(t - a)^{k}}{k!} \frac{k!}{{\prod}_{i = 1}^{k} b_{i} !} f^{(j)}(g(a))\\ && \times \prod\limits_{i = 1}^{k} \left( \frac{g^{(i)}(a)}{i!}\right)^{b_{i}}\!. \end{array} $$

Changing the order of summation yields

$$\begin{array}{@{}rcl@{}} \mathscr{T}^{m}_{t,a}\left[\mathscr{T}^{m}_{t,g(a)}[f] \circ \mathscr{T}^{m}_{t,a}[g]\right] &=& f(g(a))+\sum\limits_{k = 1}^{m} \underset{m\geq j \geq 1}{\sum\limits_{U_{k,j}}} \frac{(t - a)^{k}}{k!} \frac{k!}{{\prod}_{i = 1}^{k} b_{i} !} f^{(j)}(g(a))\\ && \times \prod\limits_{i = 1}^{k} \left( \frac{g^{(i)}(a)}{i!}\right)^{b_{i}}\!. \end{array} $$

Factoring $\frac {(t-a)^{k}}{k!}$from the inner summations gives

$$\begin{array}{@{}rcl@{}} \mathscr{T}^{m}_{t,a}\left[\mathscr{T}^{m}_{t,g(a)}[f] \circ \mathscr{T}^{m}_{t,a}[g]\right]&=&f(g(a))+\sum\limits_{k = 1}^{m} \frac{(t-a)^{k}}{k!} \underset{m \geq j\geq 1}{\sum\limits_{U_{k,j}}} \frac{k!}{{\prod}_{i = 1}^{k} b_{i} !} f^{(j)}(g(a))\\ && \times \prod\limits_{i = 1}^{k} \left( \frac{g^{(i)}(a)}{i!}\right)^{b_{i}}\!. \end{array} $$

From FaàDi Bruno’sformula,

$$\mathscr{T}^{m}_{t,a}\left[\mathscr{T}^{m}_{t,g(a)}[f] \circ \mathscr{T}^{m}_{t,a}[g]\right]=f(g(a))+\sum\limits_{k = 1}^{m} \frac{(t-a)^{k}}{k!} \frac{\mathrm{d}^{k}}{\mathrm{d}t^{k}} f(g(t)). $$

The right-hand side of the above is the m th Taylor polynomial off(g(t))centered at a.Thus,

$$\mathscr{T}^{m}_{t,a}\left[\mathscr{T}^{m}_{t,g(a)}[f] \circ \mathscr{T}^{m}_{t,a}[g]\right]=\mathscr{T}^{m}_{t,a}[f \circ g]. $$

□

Proof Proof of Lemma 3

To show (6) holds, we consider the multivariable Taylor series forf centered atg(a),

$$\mathbf{f}(\mathbf{z})=\mathbf{f}(\mathbf{g}(a))+\underset{n_{1}n_{2}{\dots}n_{j}}{\sum\limits_{1\leq j<\infty}} \frac{1}{j!}\mathbf{f}_{z_{n_{1}}z_{n_{2}}{\cdots} z_{n_{j}}}(\mathbf{a}) \prod\limits_{k = 1}^{j} (z_{n_{k}}-g_{n_{k}}(a)). $$

Then_kth componentof $\mathbf {z}=\mathscr {T}_{t,a}^{m}[\mathbf {g}]$ is apolynomial in t − a with constant $g_{n_{k}}(a)$.Thus, after this substitution each factor of the product in the right-hand side is zero or a polynomial int − a of degree one ormore:

$$\mathbf{f}\left( \mathscr{T}_{t,a}^{m}[\mathbf{g}] \right) =\mathbf{f}(\mathbf{g}(a))+\underset{n_{1}n_{2}{\dots}n_{j}}{\sum\limits_{1\leq j<\infty}} \frac{1}{j!}\mathbf{f}_{z_{n_{1}}z_{n_{2}}{\cdots} z_{n_{j}}}(\mathbf{a}) \prod\limits_{k = 1}^{j} (\mathscr{T}_{t,a}^{m}[g_{n_{k}}]-g_{n_{k}}(a)). $$

Whenj > m, the productin the right-hand side of the above is degree m or less. Thus, truncating terms of degree higher than m in $\mathbf {f}(\mathscr {T}_{t,a}^{m-1}[\mathbf {g}])$ gives the sameresult as truncating terms of degree higher than m in the first m summands on the right-hand side. Thatis,

$$\mathscr{T}^{m}_{t,a} [ \mathbf{f}\left( \mathscr{T}_{t,a}^{m}[\mathbf{g}] \right) ] \!=\mathscr{\!T}_{t,a}^{m} \left[ \!\mathbf{f}(\mathbf{g}(a))\,+\,\!\underset{n_{1}n_{2}{\dots}n_{j}}{\sum\limits_{1\leq j \leq m}} \frac{1}{j!}\mathbf{f}_{z_{n_{1}}z_{n_{2}}{\cdots} z_{n_{j}}}(\mathbf{a})\! \prod\limits_{k = 1}^{j} (\mathscr{T}_{t,a}^{m}[g_{n_{k}}]\,-\,g_{n_{k}}(a))\! \right]\!\!. $$

Thebracketed expression on the right-hand side is the m th multivariable Taylor polynomial off evaluatedat $\mathscr {T}_{t,a}^{m}[\mathbf {g}]$withcenter g(a).Symbolically,

$$\mathscr{T}^{m}_{t,a} [ \mathbf{f}\left( \mathscr{T}_{t,a}^{m}[\mathbf{g}] \right) ] =\mathscr{T}_{t,a}^{m} \left[ \pmb{\mathscr{T}}_{\mathbf{z},\mathbf{g}(a)}[\mathbf{f}] \circ \mathscr{T}_{t,a}[g]\right]. $$

From Lemma 2, the left-hand side of the above is $\mathscr {T}_{t,a}^{m}[\mathbf {f} \circ \mathbf {g}],$and (6) holds. □