The danger of combining block red–black ordering with modified incomplete factorizations and its remedy by perturbation or relaxation

Abstract

Modified incomplete LU/Cholesky factorizations without fill-ins are popular preconditioners for Krylov subspace methods, because they require no extra memory and have more potential of accelerating the convergence than simple ILU/IC preconditioners. For parallelizing preconditioners, the block red–black ordering is attractive due to its highly parallel nature and small number of synchronization points. Hence, their combination seems to produce powerful and parallelizable preconditioners. In fact, however, this combination can cause breakdown of the factorization due to the occurrence of zero pivots. We analyze this phenomenon and give necessary and sufficient conditions of zero pivots in the case of a regular grid. We also show both theoretically and experimentally that adding perturbation to the diagonal elements or relaxing the compensation of dropped fill-ins is useful to alleviate the problem. Numerical tests show that the resulting preconditioners are highly effective and are applicable for up to $10^3$ level of parallelism.

Appendix: Proof of Theorem 4

Proof

As we showed in the proof of Theorem 3, the off-diagonal nonzero elements of A are unchanged throughout the factorization and remain negative. Note also that if $k\in \mathcal {I}(i)\cup \{i\}$, $\sum _{l\ne k}(-a_{kl})=a_{kk}=1$ from the zero row sum property. Hence, $0<\xi \le \eta \le 1$.

Before going into the proof, we introduce some notations. Let $\mathcal {L}(k)$ and $\mathcal {R}(k)$ be the set of column indices of nonzero elements to the left and right of the diagonal, respectively, in the k-th row. Then, $\mathcal {L}(k)$ is the set of indices of the rows that update the k-th row. Also, since A has a symmetric nonzero pattern, $\mathcal {R}(k)$ is the set of indices of the rows updated by the k-th row. We also denote by $r_k$ the row sum of the k-th row after elimination. Since there are no nonzero elements to the left of the diagonal after elimination, we have

$$\begin{aligned} r_k = u_{kk}+\sum _{l\in \mathcal {R}(k)} a_{kl}. \end{aligned}$$

(9)

If $k\in \mathcal {I}(i)\cup \{i\}$, we have

$$\begin{aligned} \sum _{l\in \mathcal {L}(k)} a_{kl} + a_{kk} + \sum _{l\in \mathcal {R}(k)} a_{kl} = \sum _{l\in \mathcal {L}(k)} a_{kl} + 1 + \sum _{l\in \mathcal {R}(k)} a_{kl} =0 \end{aligned}$$

(10)

from the zero row sum property. Combining (9) with (10) gives

$$\begin{aligned} u_{kk} = a_{kk} + \sum _{l\in \mathcal {L}(k)} a_{kl} + r_k. \end{aligned}$$

(11)

Now, suppose that $k\in \mathcal {I}(i)\cup \{i\}$ and $\mathcal {L}(k)\ne \emptyset$. We consider the change of the row sum of row k due to elimination by row $l\in \mathcal {L}(k)$. By this elimination, $a_{kl}$ is annihilated and other off-diagonal elements remain intact. Hence, the contribution to the row sum from the off-diagonal elements increases by $-a_{kl}$. The diagonal element $a_{kk}$ increases by $-a_{kl}a_{lk}/u_{ll}$ by the elimination. The sum of fill-ins to be added to the diagonal is $-\alpha (a_{kl}/u_{ll})\sum _{j\in \mathcal {R}(l)\backslash \{k\}}a_{lj}$. Combining all of these contributions, we can write the change in the row sum as

$$\begin{aligned}&-a_{kl}-\frac{a_{kl}a_{lk}}{u_{ll}}-\alpha \frac{a_{kl}}{u_{ll}}\sum _{j\in \mathcal {R}(l)\backslash \{k\}}a_{lj} \nonumber \\&\quad =-\frac{a_{kl}}{u_{ll}}\left\{ u_{ll}+a_{lk}+\sum _{j\in \mathcal {R}(l)\backslash \{k\}}a_{lj}+(1-\alpha )\sum _{j\in \mathcal {R}(l)\backslash \{k\}}(-a_{lj})\right\} \nonumber \\&\quad =-\frac{a_{kl}}{u_{ll}}\left\{ r_l+(1-\alpha )\sum _{j\in \mathcal {R}(l)\backslash \{k\}}(-a_{lj})\right\} , \end{aligned}$$

(12)

where we used (9) in the last equality. We define $s_{lk}$ by

$$\begin{aligned} s_{lk}=\frac{1}{u_{ll}} \left\{ r_l+(1-\alpha )\sum _{j\in \mathcal {R}(l)\backslash \{k\}}(-a_{lj})\right\} . \end{aligned}$$

(13)

Then, since the initial row sum of row k is zero, the final row sum that takes into account the contributions from all $l\in \mathcal {L}(k)$ can be written as

$$\begin{aligned} r_k=\sum _{l\in \mathcal {L}(k)}(-a_{kl})s_{lk}. \end{aligned}$$

(14)

Equations (11), (13) and (14) describe the propagation of the change in the row sums during the process of relaxed modified incomplete factorization.

To prove the theorem, we show that for any $k\in \mathcal {I}(i)$ and $m\in \mathcal {R}(k)$,

$$\begin{aligned} s_{km} \ge (1-\alpha )(p-1)\xi . \end{aligned}$$

(15)

We proceed by induction. First consider the case of $\mathcal {D}(k)=0$. In this case, row k is not altered by any row since $\mathcal {I}(k)=\emptyset$. Hence, $u_{kk}=a_{kk}=1$ and $r_k=0$ from the condition (ii) of Theorem 1. Also, since row k is not altered by any row, it has no nonzero elements to the left of the diagonal and all of its p off-diagonal nonzero elements lie to the right of the diagonal. Thus, from Eq. (13), $s_{km}$ can be bounded from below as

$$\begin{aligned} s_{km} =(1-\alpha ) \sum _{j\in \mathcal {R}(k)\backslash \{m\}}(-a_{kj}) \ge (1-\alpha )(p-1)\xi \end{aligned}$$

(16)

and the claim holds. Next, assume that Eq. (15) holds for all rows in $\mathcal {I}(i)$ with depth smaller than d, where d is some positive integer. Now, pick up a row $k\in \mathcal {I}(i)$ with $\mathcal {D}(k)=d$. Then, the row l used in the elimination of row k satisfies $\mathcal {D}(l)<d$ and hence $s_{lk} \ge (1-\alpha )(p-1)\xi$ from the hypothesis. Thus, the row sum of row k after elimination can be evaluated as

$$\begin{aligned} r_k = \sum _{l\in \mathcal {L}(k)}(-a_{kl})s_{lk} \ge (1-\alpha )(p-1)\xi \sum _{l\in \mathcal {L}(k)}(-a_{kl}). \end{aligned}$$

(17)

On the other hand, putting (11) into the definition of $s_{km}$, Eq. (13), yields

$$\begin{aligned} s_{km}=\frac{(1-\alpha )\sum _{j\in \mathcal {R}(k) \backslash \{m\}}(-a_{kj})+r_k}{a_{kk} + \sum _{l\in \mathcal {L}(k)} a_{kl} + r_k}. \end{aligned}$$

(18)

We bound the right-hand side from below. Let $\mathcal {L}(k)=q$ and $\mathcal {R} (k)=p-q$. Since row k is updated by at least one row, $q\ge 1$. Also, $q\le p-1$, since otherwise $\mathcal {R} (k)=\emptyset$, which contradicts the assumption that $k\in \mathcal {I}(i)$. We note that the first term in the numerator of (18) can be bounded by the sum of the first and the second terms in the denominator as

$$\begin{aligned} (1-\alpha )\sum _{j\in \mathcal {R}(k) \backslash \{m\}}(-a_{kj}) \le \sum _{j\in \mathcal {R}(k)}(-a_{kl}) = a_{kk}+\sum _{j\in \mathcal {L}(k)}a_{kl}, \end{aligned}$$

(19)

where we used the zero row sum property in the last equality. Then, by applying the general inequality $\frac{b+\epsilon }{a+\epsilon }\ge \frac{b+\delta }{a+\delta }$ that holds for four positive numbers $a, b, \epsilon$ and $\delta$ with $a\ge b$ and $\epsilon \ge \delta$ to Eq. (18) (with $\epsilon =r_k$ and $\delta$ being the right-hand side of (17)), we have

$$\begin{aligned} s_{km}\ge \frac{(1-\alpha )\sum _{j\in \mathcal {R}(k)\backslash \{m\}}(-a_{kj})+(1-\alpha )(p-1)\xi \sum _ {l\in \mathcal {L}(k)}(-a_{kl})}{a_{kk} + \sum _{l\in \mathcal {L}(k)} a_{kl} + (1-\alpha )(p-1)\xi \sum _{l\in \mathcal {L}(k)}(-a_{kl})}. \end{aligned}$$

(20)

The numerator can be rewritten as

$$\begin{aligned}&(1-\alpha )(p-1)\xi \Biggl [\sum _{j\in \mathcal {L}(k)}(-a_{kj})+\sum _{j\in \mathcal {R}(k)\backslash \{m\}}(-a_{kj}) \nonumber \\&\qquad +\,\left\{ \frac{1}{(p-1)\xi }-1\right\} \sum _{j\in \mathcal {R}(k)\backslash \{m\}}(-a_{kj})\Biggr ] \nonumber \\&\quad = (1-\alpha )(p-1)\xi \Biggl [1+a_{km}+\left\{ \frac{1}{(p-1)\xi }-1\right\} \sum _{j\in \mathcal {R}(k)\backslash \{m\}}(-a_{kj})\Biggr ] \nonumber \\&\quad \ge (1-\alpha )(p-1)\xi \left[ 1-\eta +\left\{ \frac{1}{(p-1)\xi }-1\right\} (p-q-1)\xi \right] , \end{aligned}$$

(21)

where we used the zero row sum property in the first equality. The denominator can be bounded as

$$\begin{aligned} a_{kk} + \sum _{l\in \mathcal {L}(k)} a_{kl} + (1-\alpha )(p-1)\xi \sum _{l\in \mathcal {L}(k)}(-a_{kl}) \le 1-q\xi +(1-\alpha ), \end{aligned}$$

(22)

where we used $(p-1)\xi \le 1$ and $\sum _{l\in \mathcal {L}(k)}(-a_{kl})\le 1$. Inserting (21) and (22) into (20) gives

$$\begin{aligned} s_{km}\ge (1-\alpha )(p-1)\xi \cdot \frac{1-\eta +\left\{ \frac{1}{(p-1)\xi }-1\right\} (p-q-1)\xi }{1-q\xi +(1-\alpha )}. \end{aligned}$$

(23)

Noting that the denominator of the right-hand side is positive, we know that the claim (15) holds if

$$\begin{aligned} -\eta +\left\{ \frac{1}{(p-1)\xi }-1\right\} (p-q-1)\xi \ge -q\xi +(1-\alpha ), \end{aligned}$$

(24)

or

$$\begin{aligned} \eta \le \left( 2\xi -\frac{1}{p-1}\right) q+1-(p-1)\xi -(1-\alpha ). \end{aligned}$$

(25)

The right-hand side is a linear function of q and and its minimum in the interval $1\le q\le p-1$ is

$$\begin{aligned} \left\{ \begin{array}{lll} (p-1)\xi -(1-\alpha ) &{} \mathrm{at} \quad q=p-1 &{} \mathrm{when} \quad \xi \le \frac{1}{2(p-1)}, \\ \frac{p-2}{p-1}-(p-3)\xi -(1-\alpha ) &{} \mathrm{at} \quad q=1 &{} \mathrm{when} \quad \xi >\frac{1}{2(p-1)}. \end{array}\right. \end{aligned}$$

(26)

In either case, the minimum is larger than or equal to $\eta$ from the condition (7) and thus (15) holds also for k. Hence, the induction is complete and (15) holds for any $k\in \mathcal {I}(i)$ and $m\in \mathcal {R}(k)$.

Now that (15) has been established, we bound the pivots $u_{kk}$ for $k\in \mathcal {I}(i)\cup \{i\}$ from below. If $\mathcal {D}(k)=0$, row k is not eliminated by any row, so, from the scaling assumption, $u_{kk}=1> (1-\alpha )(p-1)\xi ^2$. Next, assume that $k\in \mathcal {I}(i)$ and $\mathcal {D}(k)>0$. Then, there is at least one nonzero element to the left of the diagonal. Thus, from (14) and (15), it follows that

$$\begin{aligned} r_k= \sum _{l\in \mathcal {L}(k)}(-a_{kl})s_{lk}\ge \xi \cdot (1-\alpha )(p-1)\xi \ge (1-\alpha )(p-1)\xi ^2. \end{aligned}$$

(27)

Combining this with (9), we have

$$\begin{aligned} u_{kk}=r_k-\sum _{l\in \mathcal {R}(k)} a_{kl} \ge r_k \ge (1-\alpha )(p-1)\xi ^2. \end{aligned}$$

(28)

Finally, consider the case of $k=i$. In this case, there is no nonzero element to the right of the diagonal and therefore $\sum _{l\in \mathcal {L}(i)}(-a_{il})=a_{ii}=1$. Hence,

$$\begin{aligned} r_i=\sum _{l\in \mathcal {L}(i)}(-a_{il})s_{li} \ge \left\{ \sum _{l\in \mathcal {L}(i)}(-a_{il})\right\} \cdot (1-\alpha )(p-1)\xi = (1-\alpha )(p-1)\xi \end{aligned}$$

(29)

and we have from (9),

$$\begin{aligned} u_{ii}=r_i\ge (1-\alpha )(p-1)\xi \ge (1-\alpha )(p-1)\xi ^2. \end{aligned}$$

(30)

This completes the proof. $\square$