Compressive Sensing and Algebraic Coding: Connections and Challenges

Abstract

Compressive sensing refers to the reconstruction of high dimensional but low-complexity objects from relatively few measurements. Examples of such objects include: high dimensional but sparse vectors, large images with very few sharp edges, and high-dimensional matrices of low rank. One of the most popular methods for reconstruction is to solve a suitably constrained $\ell _1$-norm minimization problem, otherwise known as basis pursuit (BP). In this approach, a key role is played by the measurement matrix, which converts the high dimensional but sparse vector (for example) into a low-dimensional real-valued measurement vector. The widely used sufficient conditions for guaranteeing that BP recovers the unknown vector are the restricted isometry property (RIP), and the robust null space property (RNSP). It has recently been shown that the RIP implies the RNSP. There are two approaches for generating matrices that satisfy the RIP, namely, probabilistic and deterministic. Probabilistic methods are older. In this approach, the measurement matrix consists of samples of a Gaussian or sub-Gaussian random variable. This approach leads to measurement matrices that are “order optimal,” in that the number of measurements required is within a constant factor of the optimum achievable. However, in practice, such matrices have no structure, which leads to enormous storage requirements and CPU time. Recently, the emphasis has shifted to the use of sparse binary matrices, which require less storage and are much faster than randomly generated matrices. A recent trend has been the use of methods from algebraic coding theory, in particular, expander graphs and low-density parity-check (LDPC) codes, to construct sparse binary measurement matrices. In this chapter, we will first briefly summarize the known results on compressed sensing using both probabilistic and deterministic approaches. In the first part of the chapter, we introduce some new constructions of sparse binary measurement matrices based on low-density parity-check (LDPC) codes. Then, we describe some of our recent results that lead to the fastest available algorithms for compressive sensing in specific situations. We suggest some interesting directions for future research.

This research was supported by the National Science Foundation, USA under Award #ECCS-1306630, and by the Department of Science and Technology, Government of India.

Appendix

In this appendix, we compare the number of measurements used by probabilistic as well as deterministic methods to guarantee that the corresponding measurement matrix A satisfies the restricted isometry property (RIP), as stated in Theorem 1. Note that the number of measurements is computed from the best available sufficient condition. In principle, it is possible that matrices with fewer rows might also satisfy the RIP. But there would not be any theoretical justification for using such matrices.

In probabilistic methods, the number of measurements m is $O(k \log (n/k))$. However, in reality, the O symbol hides a huge constant. It is possible to replace the O symbol by carefully collating the relevant theorems in [18]. This leads to the following explicit bounds.

Table 8 Best available bounds for the number of measurements for various choices of n and k using both probabilistic and deterministic constructions. For probabilistic constructions, the failure probability is $\xi = 10^{-9}$. $m_G, m_{SG}, m_A$ denote, respectively, the bounds on the number of measurements using a normal Gaussian, a sub-Gaussian with $c = 1/2$, and a bipolar random variable and the bound of Achlioptas. For deterministic methods, $m_D$ denotes the number of measurements using DeVore’s construction, while $m_C$ denotes the number of measurements using chirp matrices

Theorem 24

Suppose X is a random variable with zero mean, unit variance, and suppose in addition that there exists a constant c such that⁷

$$\begin{aligned} E[ \exp (\theta X)] \le \exp (c \theta ^2) , \; \forall \theta \in {\mathbb R}. \end{aligned}$$

(61)

Define

$$\begin{aligned} \gamma = 2 , \zeta = 1/(4c) , \alpha = \gamma e^{- \zeta } + e^\zeta , \beta = \zeta , \end{aligned}$$

(62)

$$\begin{aligned} \tilde{c}:= \frac{ \beta ^2 }{ 2 ( 2 \alpha + \beta ) } . \end{aligned}$$

(63)

Suppose an integer k and real numbers $\delta , \xi \in (0,1)$ are specified, and that $A = (1/\sqrt{m}) \varPhi$, where $\varPhi \in {\mathbb R}^{m \times n}$ consists of independent samples of X. Then, A satisfies the RIP of order k with constant $\delta$ with probability $\ge 1 - \xi$ provided

$$\begin{aligned} m \ge \frac{1}{ \tilde{c}\delta ^2 } \left( \frac{4}{3} k \ln \frac{en}{k} + \frac{14k}{3} + \frac{4}{3} \ln \frac{2}{\xi } \right) . \end{aligned}$$

(64)

In (64), the number of measurements m is indeed $O(k \log (n/k))$. However, for realistic values of n and k, the number of measurements n would be comparable to, or even to exceed, n, which would render “compressed” sensing meaningless.⁸ For “pure” Gaussian variables, it is possible to find improved bounds for m (see Theorem 2 which is based on [18, Theorem 9.27]. Also, for binary random variables where X equals $\pm 1$ with equal probability, another set of bounds is available [52]. While all of these bounds are $O(k \log (n/k))$, in practical situations the bounds are not useful.

This suggests that it is worthwhile to study deterministic methods for generating measurement matrices that satisfy the RIP. There are very few such methods. Indeed, the authors are aware of only three methods. The paper [15] uses a finite field method to construct a binary matrix, and this method is used in the present chapter. The paper [53] gives a procedure for choosing rows from a unitary Fourier matrix such that the resulting matrix satisfies the RIP. This method leads to the same values for the number of measurements m as that in [15]. Constructing partial Fourier matrices is an important part of reconstructing time-domain sparse signals from a limited number of frequency measurements (or vice versa). Therefore, the results of [53] can be used in this situation. In both of these methods, m equals $q^2$ where q is appropriately chosen prime number. Finally, in [54] a method is given based on chirp matrices. In this case, m equals a prime number q. Note that the partial Fourier matrix and the chirp matrix are complex, whereas the method in [15] leads to a binary matrix. In all three methods, $m = O(n^{1/2})$, which grows faster than $O(k \log (n/k))$. However, the constant under this O symbol is quite small. Therefore, for realistic values of k and n, the bounds for m from these methods are much smaller than those derived using probabilistic methods.

Table 8 gives the values of m for various values of n and k. Also, while the chirp matrix has fewer measurements than the binary matrix, $\ell _1$-norm minimization with the binary matrix runs much faster than with the chirp matrix, due to the sparsity of the binary matrix. In view of these numbers, in the present chapter, we used DeVore’s construction as the benchmark for the recovery of sparse vectors.