Theory of Harmonic Balance

Abstract

Harmonic Balance relies on the representation of time-periodic variables as Fourier series. Hence, we first compile useful definitions, notation, and theory of Fourier analysis. Next, we mathematically formulate the problem of finding periodic solutions of differential equation systems. We then describe the weighted residual approach, and identify Harmonic Balance and its relatives as special cases of this general approach. Finally, we distinguish important variants of Harmonic Balance and summarize the currently available convergence theorems.

Appendix: Fourier Transformers

The Fourier transformers allow compact and unified notation of Fourier analysis. In this appendix, we derive the Fourier transformers for the general case of vector-valued functions. To distinguish between complex-exponential and sine–cosine Fourier series representations, we use the indices $\left( {h}\right) _{\mathrm {ce}}$ and $\left( {h}\right) _{\mathrm {sc}}$, respectively. For instance, $\varvec{h}_{H}$ in the unified notation is expressed as $\varvec{h}_{\mathrm {ce},H}$ in the complex-exponential representation and as $\varvec{h}_{\mathrm {sc},H}$ in the sine–cosine representation.

Kronecker Product

To express the Fourier transformers for vector-valued functions in a compact way, the Kronecker product is particularly useful. Consider two matrices $\varvec{A},\varvec{B}$, where $\varvec{A}$ has dimension $m\times n$. The Kronecker product is then defined as the matrix

$$\begin{aligned} {\varvec{A}} \otimes {\varvec{B}} = \left[ \begin{array}{ccc} a_{11}\varvec{B} &{} \cdots &{} a_{1n}\varvec{B} \\ \vdots &{} \ddots &{} \vdots \\ a_{m1}\varvec{B} &{} \cdots &{} a_{mn}\varvec{B}\end{array}\right] \,,\end{aligned}$$

(2.55)

where $a_{ij}$ is the element of $\varvec{A}$ in the i-th row and j-th column.

The Kronecker product is bilinear and associative,

$$\begin{aligned} \varvec{A}\otimes \left( \varvec{B}+\varvec{C}\right)&= \varvec{A}\otimes \varvec{B} + \varvec{A}\otimes \varvec{C}\,,\end{aligned}$$

(2.56)

$$\begin{aligned} \left( \varvec{A} + \varvec{B}\right) \otimes \varvec{C}&= \varvec{A}\otimes \varvec{C} + \varvec{B}\otimes \varvec{C}\,,\end{aligned}$$

(2.57)

$$\begin{aligned} \left( \alpha \varvec{A}\right) \otimes \varvec{B}&= \varvec{A}\otimes \left( \alpha \varvec{B}\right) = \alpha \left( \varvec{A}\otimes \varvec{B}\right) \,,\end{aligned}$$

(2.58)

$$\begin{aligned} \left( \varvec{A}\otimes \varvec{B}\right) \otimes \varvec{C}&= \varvec{A}\otimes \left( \varvec{B} \otimes \varvec{C}\right) \,,\end{aligned}$$

(2.59)

but noncommutative, i.e., $\varvec{A}\otimes \varvec{B}\ne \varvec{B}\otimes \varvec{A}$, in general. We will also use the mixed-product property,

$$\begin{aligned} \left( \varvec{A} \otimes \varvec{B}\right) \left( \varvec{C} \otimes \varvec{D}\right) = \left( \varvec{A}\varvec{C}\right) \otimes \left( \varvec{B}\varvec{D}\right) \,.\end{aligned}$$

(2.60)

Herein, we presume that the inner dimensions of the matrices are such that the products $\varvec{A}\varvec{C}$ and $\varvec{B}\varvec{D}$ can be calculated.

We will often use Kronecker products where one of the factors is the identity matrix. Using the identity or, more generally, a diagonal matrix in a Kronecker product results in a sparse matrix (containing many zero elements). This should be taken into account when implementing linear algebra operations in a computationally efficient way.

Complex-Exponential Representation

In accordance with Eq. (2.25), we follow the truncated Fourier series $\varvec{f}_H(t)$ of a vector-valued function $\varvec{f}(t):\mathbb R\rightarrow \mathbb R^{n_f}$ from the definition in Eq. (2.1) as follows:

$$\begin{aligned} \varvec{f}_H(t)&= \sum \limits _{k=-H}^H \hat{\varvec{f}}(k){\mathrm {e}}^{{\mathrm {i}}k\varOmega t} \nonumber \\&= \left[ \begin{array}{ccccc} {\mathrm {e}}^{-{\mathrm {i}}H\varOmega t}\varvec{I}_{n_f}~&~\ldots ~&~\varvec{I}_{n_f}~&~\ldots ~&~ {\mathrm {e}}^{{\mathrm {i}}H\varOmega t}\varvec{I}_{n_f}\end{array}\right] \underbrace{\left[ \begin{array}{c} \hat{\varvec{f}}(-H) \\ \vdots \\ \hat{\varvec{f}}(0) \\ \vdots \\ \hat{\varvec{f}}(H)\end{array}\right] }_{\hat{\varvec{f}}_{\mathrm {ce},H}}\nonumber \\&= \underbrace{\overbrace{\left[ \begin{array}{ccccc} {\mathrm {e}}^{-{\mathrm {i}}H\varOmega t}&\ldots&1&\ldots&{\mathrm {e}}^{{\mathrm {i}}H\varOmega t}\end{array}\right] }^{\check{\varvec{h}}_{\mathrm {ce},H}(\varOmega t)} \otimes \varvec{I}_{n_f}}_{\varvec{h}_{\mathrm {ce},H}(\varOmega t)} \hat{\varvec{f}}_{\mathrm {ce},H}\,. \end{aligned}$$

(2.61)

As mentioned before, $\check{\varvec{h}}_H$ refers to the one-dimensional case, whereas $\varvec{h}_H$ refers to the ${n_f}$-dimensional case, and $\varvec{I}_{n_f}$ is the identity matrix of dimension ${n_f}$.

In accordance with Eq. (2.26), we follow the Fourier coefficients from Eq. (2.2) as

$$\begin{aligned} \hat{\varvec{f}}_{\mathrm {ce},H} = \left[ \begin{array}{c} \hat{\varvec{f}}(-H) \\ \vdots \\ \hat{\varvec{f}}(0) \\ \vdots \\ \hat{\varvec{f}}(H)\end{array}\right]&= \frac{1}{T}\int \limits _0^T \left[ \begin{array}{c} \varvec{f}(t) {\mathrm {e}}^{{\mathrm {i}}H\varOmega t} \\ \vdots \\ \varvec{f}(t) \\ \vdots \\ \varvec{f}(t) {\mathrm {e}}^{-{\mathrm {i}}H\varOmega t} \end{array}\right] {\mathrm {d}}t \nonumber \\&= \frac{1}{T}\int \limits _0^T \underbrace{\overbrace{\left[ \begin{array}{c} {\mathrm {e}}^{{\mathrm {i}}H\varOmega t} \\ \vdots \\ 1 \\ \vdots \\ {\mathrm {e}}^{-{\mathrm {i}}H\varOmega t}\end{array}\right] }^{\check{\varvec{h}}_{\mathrm {ce},H}^*(\varOmega t)} \otimes \varvec{I}_{n_f}}_{\varvec{h}_{\mathrm {ce},H}^*(\varOmega t)} \varvec{f}(t) {\mathrm {d}}t \end{aligned}$$

(2.62)

$$\begin{aligned}&= \langle \varvec{h}_{\mathrm {ce},H}^*, \varvec{f}\rangle \,. \end{aligned}$$

(2.63)

Note that $\check{\varvec{h}}_{\mathrm {ce},H}^*(\varOmega t) = \overline{\check{\varvec{h}}}_{\mathrm {ce},H}^{\mathrm T}(\varOmega t)$, and thus $\varvec{h}_{\mathrm {ce},H}^*(\varOmega t) = \overline{\varvec{h}}_{\mathrm {ce},H}^{\mathrm T}(\varOmega t)$.

From Eqs. (2.27), (2.1), and (2.19), we can follow

$$\begin{aligned} \tilde{\varvec{f}}_N = \left[ \begin{array}{c} \tilde{\varvec{f}}(0) \\ \vdots \\ \tilde{\varvec{f}}(N-1)\end{array}\right] \approx \underbrace{\overbrace{\left[ \begin{array}{ccc} {\mathrm {e}}^{{\mathrm {i}}(-H)\frac{2\pi }{N}(0)} &{} \cdots &{} {\mathrm {e}}^{{\mathrm {i}}(+H)\frac{2\pi }{N}(0)}\\ \vdots &{} &{} \vdots \\ {\mathrm {e}}^{{\mathrm {i}}(-H)\frac{2\pi }{N}(N-1)} &{} \cdots &{} {\mathrm {e}}^{{\mathrm {i}}(+H)\frac{2\pi }{N}(N-1)}\end{array}\right] }^{\check{\varvec{E}}_{\mathrm {ce},NH}} \otimes \varvec{I}_{n_f}}_{\varvec{E}_{\mathrm {ce},NH}} \left[ \begin{array}{c} \hat{\varvec{f}}(-H) \\ \vdots \\ \hat{\varvec{f}}(0) \\ \vdots \\ \hat{\varvec{f}}(H)\end{array}\right] \,.\end{aligned}$$

(2.64)

Equality, $\tilde{\varvec{f}}_N = \varvec{E}_{\mathrm {ce},NH}\hat{\varvec{f}}_{\mathrm {ce},H}$, holds if $\varvec{f}(t)=\varvec{f}_H(t)$. Note that the n-th row of $\check{\varvec{E}}_{\mathrm {ce},NH}$ is the vector $\check{\varvec{h}}_{\mathrm {ce},H}$ evaluated at $\varOmega t_n= 2\pi n/N$.

From Eqs. (2.28) and (2.20), we can follow analogously

$$\begin{aligned} \hat{\varvec{f}}_{\mathrm {ce},H} = \left[ \begin{array}{c} \hat{\varvec{f}}(-H) \\ \vdots \\ \hat{\varvec{f}}(0) \\ \vdots \\ \hat{\varvec{f}}(H)\end{array}\right] \approx \underbrace{\overbrace{\frac{1}{N} \left[ \begin{array}{ccc} {\mathrm {e}}^{-{\mathrm {i}}(-H)\frac{2\pi }{N}(0)} &{} \cdots &{} {\mathrm {e}}^{-{\mathrm {i}}(-H)\frac{2\pi }{N}(N-1)}\\ \vdots &{} &{} \vdots \\ {\mathrm {e}}^{-{\mathrm {i}}(+H)\frac{2\pi }{N}(0)} &{} \cdots &{} {\mathrm {e}}^{-{\mathrm {i}}(+H)\frac{2\pi }{N}(N-1)}\end{array}\right] }^{\check{\varvec{E}}_{\mathrm {ce},HN}^*} \otimes \varvec{I}_{n_f}}_{\varvec{E}_{\mathrm {ce},HN}^*} \left[ \begin{array}{c} \tilde{\varvec{f}}(0) \\ \vdots \\ \tilde{\varvec{f}}(N-1)\end{array}\right] \,.\end{aligned}$$

(2.65)

Equality, $\hat{\varvec{f}}_{\mathrm {ce},H} = \varvec{E}_{\mathrm {ce},HN}^*\tilde{\varvec{f}}_{N}$, holds if, in addition to $\varvec{f}(t)=\varvec{f}_H(t)$, $N\ge 2H+1$. Note that $\check{\varvec{E}}_{\mathrm {ce},NH}^* = \overline{\check{\varvec{E}}} ^{\mathrm T}_{\mathrm {ce},NH}/N$, and thus $\varvec{E}_{\mathrm {ce},NH}^* = \overline{\varvec{E}} ^{\mathrm T}_{\mathrm {ce},NH}/N$.

In accordance with Eq. (2.29), the first-order time derivative of a truncated Fourier series is expressed as

$$\begin{aligned} \dot{\varvec{f}}_H&= \frac{{\mathrm {d}}}{{\mathrm {d}}t} \sum \limits _{k=-H}^H \hat{\varvec{f}}(k){\mathrm {e}}^{{\mathrm {i}}k\varOmega t} \\&= \sum \limits _{k=-H}^H {\mathrm {i}}k\varOmega \hat{\varvec{f}}(k){\mathrm {e}}^{{\mathrm {i}}k\varOmega t} \\&= \varvec{h}_{\mathrm {ce},H}(\varOmega t) \varOmega \varvec{\nabla }_{\mathrm {ce}} \hat{\varvec{f}}_{\mathrm {ce},H}\,,\end{aligned}$$

with

$$\begin{aligned} \varvec{\nabla }_{\mathrm {ce}} = \overbrace{{\text {diag}}\left[ -{\mathrm {i}}H,\ldots ,{\mathrm {i}}H\right] }^{\check{\varvec{\nabla }}_{\mathrm {ce}}} \otimes \varvec{I}_{n_f}\,.\end{aligned}$$

(2.66)

Herein, ${\text {diag}}\varvec{c}$ is a diagonal matrix with the elements of list $\varvec{c}$ on the diagonal. Note that the Fourier transformers, ${\varvec{h}}(\tau )$, ${\varvec{E}}$, ${\varvec{\nabla }}$, ${\varvec{E}}^*$, ${\varvec{h}}^*(\tau )$ only depend on H, N, and ${n_f}$, but not on $\varOmega$ or any other parameter.

Sine–Cosine Representation

Analogous to the complex-exponential representation, we can write the following:

$$\begin{aligned} \varvec{f}_H(t) = \underbrace{\overbrace{\left[ \begin{array}{cccccc} 1&\cos \left( \varOmega t\right)&\sin \left( \varOmega t\right)&\ldots&\cos \left( H\varOmega t\right)&\sin \left( H\varOmega t\right) \end{array}\right] }^{\check{\varvec{h}}_{\mathrm {sc},H}(\varOmega t)} \otimes \varvec{I}_{n_f}}_{\varvec{h}_{\mathrm {sc},H}(\varOmega t)} \underbrace{\left[ \begin{array}{c} \hat{\varvec{f}}(0) \\ \hat{\varvec{f}}_{\mathrm c}(1) \\ \hat{\varvec{f}}_{\mathrm s}(1) \\ \vdots \\ \hat{\varvec{f}}_{\mathrm c}(H) \\ \hat{\varvec{f}}_{\mathrm s}(H) \end{array}\right] }_{\hat{\varvec{f}}_{\mathrm {sc},H}}\,,\end{aligned}$$

(2.67)

and

$$\begin{aligned} \hat{\varvec{f}}_{\mathrm {sc},H} = \frac{1}{T} \int \limits _0^T ~\underbrace{\overbrace{\left[ \begin{array}{c} 1 \\ 2\cos \left( \varOmega t\right) \\ 2\sin \left( \varOmega t\right) \\ \vdots \\ 2\cos \left( H\varOmega t\right) \\ 2\sin \left( H\varOmega t\right) \end{array}\right] }^{\check{\varvec{h}}_{\mathrm {sc},H}^*(\varOmega t)} \otimes \varvec{I}_{n_f}}_{\varvec{h}_{\mathrm {sc},H}^*(\varOmega t)} ~ \varvec{f}(t) {\mathrm {d}}t \,.\end{aligned}$$

(2.68)

Similarly, we obtain

$$\begin{aligned} \tilde{\varvec{f}}_N \approx \underbrace{\left[ \begin{array}{cccc} 1 &{} \cos \left( \frac{2\pi (0)(1)}{N}\right) &{} \cdots &{} \sin \left( \frac{2\pi (0)(H)}{N}\right) \\ \vdots &{} \vdots &{} &{} \vdots \\ 1 &{} \cos \left( \frac{2\pi (N-1)(1)}{N}\right) &{} \cdots &{} \sin \left( \frac{2\pi (N-1)(H)}{N}\right) \end{array}\right] \otimes \varvec{I}_{n_f}}_{\varvec{E}_{\mathrm {sc},NH}} \left[ \begin{array}{c} \hat{\varvec{f}}(0) \\ \hat{\varvec{f}}_{\mathrm c}(1) \\ \vdots \\ \hat{\varvec{f}}_{\mathrm s}(H) \end{array}\right] \,, \end{aligned}$$

(2.69)

and

$$\begin{aligned} \hat{\varvec{f}}_{\mathrm {sc},H} \approx \underbrace{\frac{2}{N}\left[ \begin{array}{ccc} \frac{1}{2} &{} \cdots &{} \frac{1}{2} \\ \cos \left( \frac{2\pi (1)(0)}{N}\right) &{} \cdots &{} \cos \left( \frac{2\pi (1)(N-1)}{N}\right) \\ \vdots &{} &{} \vdots \\ \sin \left( \frac{2\pi (H)(0)}{N}\right) &{} \cdots &{} \sin \left( \frac{2\pi (H)(N-1)}{N}\right) \end{array}\right] \otimes \varvec{I}_{n_f}}_{\varvec{E}_{\mathrm {sc},HN}^*} \left[ \begin{array}{c} \tilde{\varvec{f}}(0) \\ \vdots \\ \tilde{\varvec{f}}(N-1)\end{array}\right] \,, \end{aligned}$$

(2.70)

where equality holds as discussed for the complex-exponential representation.

Finally, the first-order time derivative of a truncated Fourier series in sine–cosine representation reads as follows:

$$\begin{aligned}\dot{\varvec{f}}_{\mathrm {sc},H}&= \frac{{\mathrm {d}}}{{\mathrm {d}}t} \left( ~ \hat{\varvec{f}}(0) + \sum \limits _{k=1}^H \hat{\varvec{f}}_{\mathrm c}(k) \cos \left( k\varOmega t\right) + \hat{\varvec{f}}_{\mathrm s}(k) \sin \left( k\varOmega t\right) ~\right) \\&= \sum \limits _{k=1}^H k\varOmega \hat{\varvec{f}}_{\mathrm s}(k) \cos \left( k\varOmega t\right) - k\varOmega \hat{\varvec{f}}_{\mathrm c}(k) \sin \left( k\varOmega t\right) \\&= \varvec{h}_{\mathrm {sc},H}(\varOmega t) \varOmega \varvec{\nabla }_{\mathrm {sc}} \hat{\varvec{f}}_{\mathrm {sc},H}\,,\end{aligned}$$

with

$$\begin{aligned} \varvec{\nabla }_{\mathrm {sc}} = \overbrace{{\text {diag}}\left[ 0, \varvec{\nabla }_1, \ldots , \varvec{\nabla }_H\right] }^{\check{\varvec{\nabla }}_{\mathrm {sc}}}\otimes \varvec{I}_{n_f}\,,\quad \varvec{\nabla }_k = \left[ \begin{array}{rr} 0 &{} k~ \\ -k &{} 0~\end{array}\right] \,. \end{aligned}$$

(2.71)

Other Representations

Besides the already introduced representations, closely related alternatives exist. All of these are fully equivalent. It is a matter of convention, which of them is used. The only strict requirement is that the representation is used consistently and not mixed with other ones, without appropriate conversion. In the following paragraphs, we mention a few frequently used variants.

We ordered the vector $\hat{\varvec{f}}_H$ such that Fourier coefficients to the same harmonic index k are together. Within a harmonic index, the coefficients are ordered just as the elements within the vector $\varvec{f}(t)$ are ordered. Hence, the first ${n_f}$ elements of $\hat{\varvec{f}}_H$ belong to the first harmonic index, and so on. An alternative is to collect all Fourier coefficients for the same element of $\varvec{f}(t)$, and then stack these in a column vector. Then, the first $2H+1$ elements of this new vector belong to the first element of $\varvec{f}(t)$ and so on. The difference only shows itself in the multi-dimensional case, i.e., when ${n_f}>1$.

Attention

A wrongly assumed sorting is certainly a common source of programming errors.

In the complex-exponential formulation of the Discrete Fourier Transform, it is possible to use only positive indices. This can be most conveniently illustrated for the special case $N=2H+1$. Note that

$$\begin{aligned} {\mathrm {e}}^{{\mathrm {i}}\frac{2\pi }{N}(-k)} = {\mathrm {e}}^{{\mathrm {i}}\frac{2\pi }{N}N}{\mathrm {e}}^{{\mathrm {i}}\frac{2\pi }{N}(-k)} = {\mathrm {e}}^{{\mathrm {i}}\frac{2\pi }{N}(N-k)} = {\mathrm {e}}^{{\mathrm {i}}\frac{2\pi }{N}(m)}\,.\end{aligned}$$

(2.72)

This defines a mapping from $-k=-1,\ldots ,-H$ to $m=N-1,\ldots ,N-H$. Here, we have $N=2H+1$, and thus $N-1=2H$ and $N-H=H+1$. Hence, one can change the summation limits in the Fourier series from $-H,\ldots ,H$ to $0,\ldots ,N-1=2H$. In this modified representation, the first half of the vector $\hat{\varvec{f}}_H$ (harmonic indices $k=-H,\ldots ,-1$) has to be flipped upside down and re-stacked to the end of the vector.

The factor 1 / N can be distributed arbitrarily between $\varvec{E}_{NH}$ and $\varvec{E}_{HN}^*$. The Fourier coefficients as well as the quantities $\check{\varvec{h}}_H$ and $\check{\varvec{h}}_H^*$ have to be scaled accordingly. A common choice is to use the factor $1/\sqrt{N}$ for both. Then, $\varvec{E}_{NH}$ and $\varvec{E}_{HN}^*$ are conjugate transposes of each other. For $N=2H+1$ they are also square. Due to the orthogonality property, the conjugate transpose of either matrix is then its inverse, the matrices are then called unitary. In this special case, and with the abovementioned change of the summation limits $\check{\varvec{E}}_{NH}$ is commonly referred to as Fourier matrix or discrete Fourier transform matrix.

The sign in the argument of the complex exponentials can be inverted, as long as this is done both in the definition of the Fourier series and its coefficients. Similarly, in the sine–cosine representation, the order of the sine and cosine Fourier coefficients can be exchanged.

Finally, a common notation of a Fourier series is

$$\begin{aligned} \varvec{f}_H(t) =\mathfrak {R}\left\{ {\sum \limits _{k=0}^{H} \hat{\varvec{f}}^\dagger (k) {\mathrm {e}}^{{\mathrm {i}}k\varOmega t}}\right\} . \end{aligned}$$

(2.73)

Recalling that $\hat{\varvec{f}}(-k)=\overline{\hat{\varvec{f}}(k)}$, we can easily identify

$$\begin{aligned} \hat{\varvec{f}}^\dagger (0)&= \hat{\varvec{f}}(0)\,,\end{aligned}$$

(2.74)

$$\begin{aligned} \hat{\varvec{f}}^\dagger (k)&= 2\hat{\varvec{f}}(k) \quad k=1,\ldots ,H\,. \end{aligned}$$

(2.75)

A particular advantage of the notation in Eq. (2.73) is that it avoids the redundancy among the Fourier coefficients $\hat{\varvec{f}}(-k)$ and $\hat{\varvec{f}}(k)$ in the complex-exponential representation. This representation is used in the computer tool NLvib, cf. Appendix C.