A First Look at Block Coders

Abstract

This chapter shows how the code and the coder can be defined by two matrices: the parity-check matrix and the generator matrix. It is explained how the correction/detection capability of the code depends on the linear independence of sets of columns of the parity-check matrix. The arithmetic is performed in finite sets, called Galois fields. Three introductory examples of Galois fields of two, three, and four elements are presented. The field with 11 elements is also easy to obtain, and it is used to decode erasures. Erasure decoding is then applied to the recovery of lost packets in data networks using Vandermonde and Cauchy matrices defined in the field with eight elements constructed in Appendix C. The chapter finishes deriving the Hamming bound, presenting the family of Hamming codes, and explaining how to obtain new codes from a given code. Appendix D is a review of the concept of rank of a matrix.

Appendices

Appendix C: Constructing the Field F₈

In this appendix, I construct the addition and multiplication tables for the eight elements in the set {0, 1, 2, 3, 4, 5, 6, 7}. As usual, 0 and 1 represent the additive and multiplicative identities, respectively.

As said in Section 2.6, the addition table is constructed writing the elements in binary form and adding them bitwise . For instance

$$ 3+5=\left(0\ 1\ 1\right)+\left(1\ 0\ 1\right)=\left(1\ 1\ 0\right)=6 $$

The other sums are computed likewise. The results appear in Table 2.1.

Table 2.1 Addition in F₈

The construction of a multiplication table is not simple. I begin by filling the 0 and 1 rows (and columns) in Table 2.2. To complete the table, I need to find the values of 21 ( = 6 + (36 − 6)/2) additional elements. For instance, what could the value of 2 ∗ 2 be? Clearly, since all the elements in any row (and column) must be different, 2 ∗ 2 ≠ 0 and 2 ∗ 2 ≠ 2. Moreover, because for fields with 2^m elements x + x = 0, the entries on the diagonal must also be different. Take two different elements, x and y. We have x + y ≠ 0, and therefore (x  +  y)² ≠ 0. But,

$$ {\left(x+y\right)}^2={x}^2+ xy+ yx+{y}^2={x}^2+{y}^2\ne 0 $$

Table 2.2 Some entries of the multiplication table

Thus, we can conclude that x² ≠ y², as claimed. This implies that 2 ∗ 2 ≠ 1.

Let’s call a the product 2 ∗ 2. Using the distributive law, we have

$$ {\displaystyle \begin{array}{l}2\ast 3=2\left(1+2\right)=2+a\\ {}3\ast 3=\left(1+2\right)\left(1+2\right)={1}^2+{2}^2=1+a\end{array}} $$

These entries are also displayed in Table 2.2.

To continue, call 2 ∗ 4 = b. Then,

$$ {\displaystyle \begin{array}{l}2\ast 5=2\left(1+4\right)=2+b\\ {}3\ast 4=\left(1+2\right)4=4+b\\ {}3\ast 5=3\left(1+4\right)=3+4+b=7+b\end{array}} $$

Also

$$ {\displaystyle \begin{array}{l}2\ast 6=2\left(2+4\right)=a+b\\ {}2\ast 7=2\left(1+6\right)=2+a+b\\ {}3\ast 6=\left(1+2\right)6=6+a+b\\ {}3\ast 7=3\left(1+6\right)=3+6+a+b=5+a+b\end{array}} $$

I have entered all these new elements in Table 2.3.

Table 2.3 More entries of the multiplication table

Finally, call 4  ∗  4 = c. Using again the distributive law, we have the multiplication table in Table 2.4. Since a, b, and c are not known, this table is only a “blueprint.”

Table 2.4 A “blueprint” of the multiplication table

To construct the table from the “blueprint,” the only remaining task is to choose the values for a,  b, and c. There are some conditions a, b, and c must satisfy:

• 1. The three numbers must be different (a and c are in the diagonal, a and b are in the same row, and b and c in the same column).

• 2. a ≠ 0, a ≠ 1, a ≠ 2 (as said before),

• 3. b ≠ 0, b ≠ 2, b ≠ 4, b ≠ 7,

• 4. c ≠ 0, c ≠ 1, c ≠ 4.

Hence, the only cases we need to explore are

$$ {\displaystyle \begin{array}{l}\ a\ \epsilon \kern0.5em \left\{3,4,5,6,7\right\}\kern1.1em b\ \epsilon\ \left\{1,3,5,6\right\}\kern0.84em c\ \epsilon\ \left\{2,3,5,6,7\right\}\\ {}\kern7.5em b\ne a,\kern4.599999em c\ne a,\kern0.6em c\ne b\end{array}} $$

Let’s try a = 3. See what values we can find for b.

We have 2 ∗ 3 = a + 2 = 3 + 2 = 1. Since b is in the same row as 2 ∗ 3 , b cannot be 1. Therefore, the only possible values for b are 5 and 6.

If we choose b = 5, then 2 ∗ 6 = a + b = 3 + 5 = 6, which is forbidden (6 would appear twice in column 6).

If we choose b = 6, then 2 ∗ 7 = 2  +  a  +  b  =  2  +  3  +  6  =  7 and 7 would appear twice in column 7.

Thus, a = 3 must be discarded.

Continue with a = 4. Proceeding by trial and error as before, two choices are now possible for b, namely: b = 3 and b = 5.

For a = 4,  b = 3, only c = 6 works. Analogously, for a = 4,  b = 5,  the only valid choice for c is 7.

Using these values, and the “blueprint” given in Table 2.4, we can construct two multiplication tables

• Table 2.5a (a = 4, b = 3, c = 6)

Table 2.5a Multiplication table when a = 4, b = 3, c = 6

and

• Table 2.5b (a = 4, b = 5, c = 7)

Table 2.5b Multiplication table when a = 4, b = 5, c = 7

In view of all the above, we have already obtained two different “versions” of the field F₈ (and more to come!). Call ${\boldsymbol{F}}_{\mathbf{8}}^{(1)}$ and ${\boldsymbol{F}}_{\mathbf{8}}^{(2)}$, respectively, the versions corresponding to Tables 2.5a and 2.5b. The coding and decoding methods presented in this book require only algebraic operations. Thus, if I find a bijective application between ${\boldsymbol{F}}_{\mathbf{8}}^{(1)}$ and ${\boldsymbol{F}}_{\mathbf{8}}^{(2)}$ that preserves additions and multiplications, all the computations in ${\boldsymbol{F}}_{\mathbf{8}}^{(1)}$ can be “mimicked” in ${\boldsymbol{F}}_{\mathbf{8}}^{(2)}$ and vice versa. We say that an application preserves additions and multiplications when, denoting by x, x^′ two elements in ${\boldsymbol{F}}_{\mathbf{8}}^{(1)}$ and y, y′ their respective images in ${\boldsymbol{F}}_{\mathbf{8}}^{(2)}$, we have

$$ x+{x}^{\prime}\to y+{y}^{\prime}\kern1em \mathrm{and}\kern1em {xx}^{\prime}\to {yy}^{\prime } $$

In this case, the choice of the version we utilize is immaterial unless the calculations performed with one of them are easier to implement. See later in this appendix.

Let’s now find an application that preserves the field structure.

Clearly the above conditions imply that 0 → 0 and 1 → 1. But what about the other elements? To answer that question, observe that any element in ${\boldsymbol{F}}_{\mathbf{8}}^{(1)}$ and ${\boldsymbol{F}}_{\mathbf{8}}^{(2)}$, different from 0 and 1, generates by successive powers all the others. This is due to the fact that the number of nonzero elements (7) is prime. In other words, all elements of the field, different from 0 and 1, are generators. This is not a general property of finite fields. However, any Galois field has generators.

Let’s check, for instance, that 2 is a generator in both ${\boldsymbol{F}}_{\mathbf{8}}^{(1)}$ and ${\boldsymbol{F}}_{\mathbf{8}}^{(2)}$.

Powers of 2 using Table 2.5a
21 = 2
22 = 4
23 = 3
24 = 6
25 = 7
26 = 5
27 = 1

Powers of 2 using Table 2.5b
21 = 2
22 = 4
23 = 5
24 = 7
25 = 3
26 = 6
27 = 1

Thus, the application is completely defined by giving the image of 2. In fact, 2 → y implies 2ⁱ → yⁱ (to preserve multiplication) Recall, though, that addition has also to be preserved. To begin with, we should have

$$ 1+{2}^i\to 1+{y}^i $$

For i = 7, the above is clearly true. Let’s see what happens for the other values of i. For i ≤ 6, 1 + 2ⁱ and 1 + yⁱ are nonzero elements of ${\boldsymbol{F}}_{\mathbf{8}}^{(1)}$ and ${\boldsymbol{F}}_{\mathbf{8}}^{(1)}$, respectively. Hence, we can write

$$ 1+{2}^i={2}^{p(i)}\kern0.36em \mathrm{and}\kern0.36em 1+{y}^i={y}^{q(i)} $$

The functions p(⋅) and q(⋅) are, in general, different. Suppose, though, that for some y we had p(⋅) = q(⋅). Then

$$ {\displaystyle \begin{array}{l}{2}^i+{2}^j={2}^i\left(1+{2}^{j-i}\right)={2}^{i+p\left(j-i\right)},j>i\\ {}{2}^{i+p\left(j-i\right)}\to {y}^{i+p\left(j-i\right)}\\ {}{y}^{i+p\left(j-i\right)}={y}^{i+q\left(j-i\right)}={y}^i\left(1+{y}^{j-i}\right)={y}^i+{y}^j\end{array}} $$

And the application would also preserve addition.

Operating in ${\boldsymbol{F}}_{\mathbf{8}}^{(1)}$, we have

$$ {\displaystyle \begin{array}{cc}1+2=3={2}^3& 1+{2}^2=1+4=5={2}^6\\ {}1+{2}^3=1+3=2& 1+{2}^4=1+6=7={2}^5\\ {}1+{2}^5=1+7=6={2}^4& 1+{2}^6=1+5=4={2}^2\end{array}} $$

Thus, the function p(⋅) is the one given in Table 2.6a

Table 2.6a The function p(⋅)

Denote by q_k(⋅) , with 2 ≤ k ≤ 7, the function q(⋅) using k as a generator in ${\boldsymbol{F}}_{\mathbf{8}}^{(1)}$. Table 2.6b shows the results of the computations.

Table 2.6b q(⋅), computed in ${\boldsymbol{F}}_8^{(2)}$ for all the generators

Therefore, the only values of y that make the application 2ⁱ → yⁱ addition preserving are 3, 5, and 6.

An application between two fields that preserves the field structure is called isomorphism and the two fields are said to be isomorphic . So, ${\boldsymbol{F}}_{\mathbf{8}}^{(1)}$ and ${\boldsymbol{F}}_{\mathbf{8}}^{(1)}$ are isomorphic.

More isomorphic fields can be obtained using other values for a, b, and c. Proceeding as before, the only legitimate multiplication tables correspond to the following choices for (a b c)

$$ \left(5\kern0.24em 1\kern0.24em 6\right)\kern1em \left(5\kern0.24em 6\kern0.24em 7\right)\kern2em \left(6\kern0.24em 5\kern0.24em 2\right)\kern2em \left(6\kern0.24em 1\kern0.24em 3\right)\kern2em \left(7\kern0.24em 6\kern0.24em 3\right)\kern1.24em \left(7\kern0.24em 3\kern0.24em 2\right) $$

The resulting multiplication tables are Tables 2.7a, 2.7b, 2.7c, 2.7d, 2.7e, and 2.7f.

Constructing multiplication tables for Galois fields using the method just exposed is a very laborious process that becomes even more cumbersome as the size of the field increases. Furthermore, since all the “versions” are isomorphic, and therefore equivalent from a mathematical point of view, which ones should we utilize when it comes to doing the calculations?

As a benchmark for comparison, I’ll use the circuits represented in Figs. 2.20 and 2.21 to compute the powers of an element. In Fig. 2.20, I have drawn four circuits to compute the powers of 2 and 3 using Tables 2.5a and 2.5b. On the other hand, the six circuits in Fig. 2.21 compute the powers of 2 using Tables 2.7a, 2.7b, 2.7c, 2.7d, 2.7e, and 2.7f.

Fig. 2.20

Circuits that compute the powers of 2 and 3 using Tables 2.5a and 2.5b

Fig. 2.21

Circuits to compute the powers of 2 using Tables 2.7a, 2.7b, 2.7c, 2.7d, 2.7e, and 2.7f

Table 2.7a Multiplication table when a = 5, b = 1, c = 6

Table 2.7b Multiplication table when a = 5, b = 6, c = 7

Table 2.7c Multiplication table when a = 6, b = 5, c = 2

Table 2.7d Multiplication table when a = 6, b = 1, c = 3

Table 2.7e Multiplication table when a = 7, b = 6, c = 3

Table 2.7f Multiplication table when a = 7, b = 3, c = 2

All these circuits are shift registers that change their state at each time tick. For instance, for the circuit that computes the powers of 2 using Table 2.5b, the state of the register changes from (c b a) to (a c a + b). The register is initialized with 1, stored in the register as (1 0 0) (see Fig. 2.22). Observe that, in contrast to what happens when we operate by hand, the leftmost bit is the least significant. The sequence of states is represented in Fig. 2.22. Converted to decimal, the sequence is 1, 2, 4, 5, 7, 3, 6 in agreement to what we obtained before.

Fig. 2.22

Sequence of states for the circuit on top

Remarks

• The complexity of the circuits that compute the powers of 2 using Tables 2.5a and 2.5b is the same.

• The circuits that compute the powers of 2 using Tables 2.5a and 2.5b are simpler than the circuits that compute the powers of 3: Multiplication by 2 is easier to implement.

• The circuits that multiply by 2 using Tables 2.7a, 2.7b, 2.7c, 2.7d, 2.7e, and 2.7f are more complex than the circuits that multiply by 2 using Tables 2.5a and 2.5b. Therefore, Tables 2.5a and 2.5b should be preferred over the others.

• Additionally, when 2^m − 1 is not a prime, 2 won’t be a generator for all tables, and those tables should also be disregarded.

In Chapter 3, I’ll address the systematic construction of multiplication tables for Galois fields of 2^m elements, a method that avoids the painstaking process used in this appendix. I will also indicate which tables we should select to ease the calculations.

Appendix D: The Rank of a Matrix

The row rank of a matrix is the maximum number of linearly independent (LI ) rows. Similarly, the column rank , is the maximum number of LI columns. An important, and perhaps unexpected, result is that the two ranks coincide. This number is called the matrix rank . The aim of this appendix is to illustrate such fact, since I used it repeatedly in this chapter.

Say that the row rank of the following 5 × 7 matrix is 3.

$$ \left(\begin{array}{ccccccc}{a}_{11}& {a}_{12}& {a}_{13}& {a}_{14}& {a}_{15}& {a}_{16}& {a}_{17}\\ {}{a}_{21}& {a}_{22}& {a}_{23}& {a}_{24}& {a}_{25}& {a}_{26}& {a}_{27}\\ {}{a}_{31}& {a}_{32}& {a}_{33}& {a}_{34}& {a}_{35}& {a}_{36}& {a}_{37}\\ {}{a}_{41}& {a}_{42}& {a}_{43}& {a}_{44}& {a}_{45}& {a}_{46}& {a}_{47}\\ {}{a}_{51}& {a}_{52}& {a}_{53}& {a}_{54}& {a}_{55}& {a}_{56}& {a}_{57}\end{array}\right) $$

that is: only three of the five rows are LI. Suppose, for instance, that rows 1, 2, and 4 are LI. Then, we have:

$$ {R}_3={aR}_1+{bR}_2+{cR}_4 $$

(2.2)

$$ {R}_5={dR}_1+{eR}_2+{fR}_4 $$

(2.3)

where R_i stands for row i. From (2.2), we obtain the following equalities

$$ {\displaystyle \begin{array}{l}{a}_{31}={aa}_{11}+{ba}_{21}+{ca}_{41}\\ {}{a}_{32}={aa}_{12}+{ba}_{22}+{ca}_{42}\\ {}\kern2.5em \dots \kern1.5em \dots \kern1.25em \dots .\\ {}{a}_{37}={aa}_{17}+{ba}_{27}+{ca}_{47}\end{array}} $$

Similarly, from (2.3) we can write

$$ {\displaystyle \begin{array}{l}{a}_{51}={da}_{11}+{ea}_{21}+{fa}_{41}\\ {}{a}_{52}={da}_{12}+{ea}_{22}+{fa}_{42}\\ {}\kern2.25em \dots \kern1.5em \dots \kern1.25em \dots .\\ {}{a}_{57}={da}_{17}+{ea}_{27}+{fa}_{47}\end{array}} $$

Then, we have

$$ {\displaystyle \begin{array}{l}\left({a}_{11}{a}_{21}{a}_{31}{a}_{41}{a}_{51}\right)={a}_{11}\left(1\ 0\ a\ 0\ d\right)+{a}_{21}\left(0\ 1\ b\ 0\ e\right)+{a}_{41}\left(0\ 0\ c\ 1\ f\right)\\ {}\left({a}_{12}{a}_{22}{a}_{32}{a}_{42}{a}_{52}\right)={a}_{12}\left(1\ 0\ a\ 0\ d\right)+{a}_{22}\left(0\ 1\ b\ 0\ e\right)+{a}_{42}\left(0\ 0\ c\ 1\ f\right)\\ {}\kern8.6em \dots \kern5em \dots \kern5em \dots \\ {}\left({a}_{17}{a}_{27}{a}_{37}{a}_{47}{a}_{57}\right)={a}_{17}\left(1\ 0\ a\ 0\ d\right)+{a}_{27}\left(0\ 1\ b\ 0\ e\right)+{a}_{47}\left(0\ 0\ c\ 1\ f\right)\end{array}} $$

The seven columns are a linear combination of the three LI columns, namely: (1 0 a 0 d), (0 1 b 0 e), and (0 0 c 1 f). Therefore, the column rank is also 3.

The following is the same proof written more compactly.

The linear dependency of rows R₃ and R₅ on R₁, R₂, and R₄ can be written as

$$ \left(\begin{array}{ccccccc}{a}_{11}& {a}_{12}& {a}_{13}& {a}_{14}& {a}_{15}& {a}_{16}& {a}_{17}\\ {}{a}_{21}& {a}_{22}& {a}_{23}& {a}_{24}& {a}_{25}& {a}_{26}& {a}_{27}\\ {}{a}_{31}& {a}_{32}& {a}_{33}& {a}_{34}& {a}_{35}& {a}_{36}& {a}_{37}\\ {}{a}_{41}& {a}_{42}& {a}_{43}& {a}_{44}& {a}_{45}& {a}_{46}& {a}_{47}\\ {}{a}_{51}& {a}_{52}& {a}_{53}& {a}_{54}& {a}_{55}& {a}_{56}& {a}_{57}\end{array}\right)=\left(\begin{array}{ccc}1& 0& 0\\ {}0& 1& 0\\ {}a& b& c\\ {}0& 0& 1\\ {}d& e& f\end{array}\right)\left(\begin{array}{ccccccc}{a}_{11}& {a}_{12}& {a}_{13}& {a}_{14}& {a}_{15}& {a}_{16}& {a}_{17}\\ {}{a}_{21}& {a}_{22}& {a}_{23}& {a}_{24}& {a}_{25}& {a}_{26}& {a}_{27}\\ {}{a}_{41}& {a}_{42}& {a}_{43}& {a}_{44}& {a}_{45}& {a}_{46}& {a}_{47}\end{array}\right) $$

Taking transpose on both sides, we obtain:

$$ \left(\begin{array}{ccccc}{a}_{11}& {a}_{21}& {a}_{31}& {a}_{41}& {a}_{51}\\ {}{a}_{12}& {a}_{22}& {a}_{32}& {a}_{42}& {a}_{52}\\ {}{a}_{13}& {a}_{23}& {a}_{33}& {a}_{43}& {a}_{53}\\ {}{a}_{14}& {a}_{24}& {a}_{34}& {a}_{44}& {a}_{54}\\ {}{a}_{15}& {a}_{25}& {a}_{35}& {a}_{45}& {a}_{55}\\ {}{a}_{16}& {a}_{26}& {a}_{36}& {a}_{46}& {a}_{56}\\ {}{a}_{17}& {a}_{27}& {a}_{37}& {a}_{47}& {a}_{57}\end{array}\right)=\left(\begin{array}{ccc}{a}_{11}& {a}_{21}& {a}_{41}\\ {}{a}_{12}& {a}_{22}& {a}_{42}\\ {}{a}_{13}& {a}_{23}& {a}_{43}\\ {}{a}_{14}& {a}_{24}& {a}_{44}\\ {}{a}_{15}& {a}_{25}& {a}_{45}\\ {}{a}_{16}& {a}_{26}& {a}_{46}\\ {}{a}_{17}& {a}_{27}& {a}_{47}\end{array}\right)\left(\begin{array}{ccccc}1& 0& a& 0& d\\ {}0& 1& b& 0& e\\ {}0& 0& c& 1& f\end{array}\right) $$

which expresses that the column space has dimension 3, the same as the row space.