Bayesian compressive sensing (BCS) helps address ill-posed signal recovery problems using the Bayesian estimation framework. Gibbs sampling is a technique used in Bayesian estimation that iteratively draws samples from conditional posterior distributions, which is inherently sequential. In this work, we propose a two-stage parallel coefficient update scheme for wavelet-based BCS, where the first stage approximates the real distributions of the wavelet coefficients and the second stage computes the final estimate of the coefficients. While in the first stage, the parallel computing units share information with each other, in the second stage, the parallel units work independently. Even when the computing units share information, when the number of computing units is large, the process deviates from the sequential Gibbs sampler resulting in large reconstruction error. We propose two new coefficient re-computation schemes to reduce the reconstruction error at the cost of longer computation time. We also propose a new coefficient update scheme that updates coefficients in both stages based on data generated a few rounds ago. Such a scheme helps in relaxing the timing constraints for communication in the first stage and computations in the second stage. We design the corresponding parallel architecture and synthesize it in 7 nm technology node. For the system with 8 computing units, the proposed algorithm reduces the execution time up to 6.8× at maximum compared to the sequential implementation.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Antonini, M., Barlaud, M., Mathieu, P., & Daubechies, I. (1992). Image coding using wavelet transform. IEEE Transactions on Image Processing, 1, 205–220.
Usevitch, B.E. (2001). Tutorial on modern lossy wavelet image compression: foundations of JPEG 2000. IEEE Signal Processing Magazine, 18, 22–35.
Candès, E.J., & Wakin, M.B. (2008). An introduction to compressive sampling. IEEE Signal Processing Magazine, 25(2), 21–30.
Duarte, M.F., Davenport, M.A., Takhar, D., Laska, J.N., Sun, T., Kelly, K.F., & Baraniuk, R.G. (2008). Single-pixel imaging via compressive sampling. IEEE Signal Processing Magazine, 25(2), 83–91.
Donoho, D.L. (2006). Compressed sensing. IEEE Transactions on Information Theory, 52(4), 1289–1306.
Tropp, J.A., & Gilbert, A.C. (2007). Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory, 53(12), 4655–4666.
Baraniuk, R.G., Cevher, V., Duarte, M.F., & Hegde, C. (2010). Model-based compressive sensing. IEEE Transactions on Information Theory, 56, 1982–2001.
Ji, S., Xue, Y., & Carin, L. (2008). Bayesian compressive sensing. IEEE Transactions on Signal Processing, 56(6), 2346–2356.
He, L., & Carin, L. (2009). Exploiting structure in wavelet-based Bayesian compressive sensing. IEEE Transactions on Signal Processing, 57(9), 3488–3497.
Robert, C.P., & Casella, G. (2004). Monte Carlo statistical methods, 2nd edn. New York: Springer.
Johnson, M., Saunderson, J., & Willsky, A. (2013). Analyzing Hogwild parallel Gaussian Gibbs sampling. In Advances in neural information processing systems (pp. 2715–2723).
Terenin, A., Simpson, D., & Draper, D. (2018). Asynchronous Gibbs sampling. arXiv:https://arxiv.org/abs/1509.08999v5.
Zhou, J., & Chakrabarti, C. (2018). Parallel wavelet-based Bayesian compressive sensing based on Gibbs sampling. In IEEE International workshop on signal processing systems (SiPS) (pp. 140–145).
Zhang, S., Choromanska, A.E., & LeCun, Y. (2015). Deep learning with elastic averaging SGD. In Advances in neural information processing systems (pp. 685–693).
Crouse, M.S., Nowak, R.D., & Baraniuk, R.G. (1998). Wavelet-based statistical signal processing using hidden Markov model. IEEE Transactions on Signal Processing, 46, 886–902.
Mallat, S. (2009). A wavelet tour of signal processing, 3rd edn. New York: Academic Press.
Sahlin, K. (2011). Estimating convergence of Markov chain Monte Carlo simulations, Stockholm University Master Thesis.
Chib, S., & Greenberg, E. (1995). Understanding the metropolis-hastings algorithm. The American Statistician, 49(4), 327–335.
Daubechies, I. (1992). Ten lectures on wavelets. SIAM.
Clark, L.T., Vashishtha, V., Shifren, L., Gujja, A., Sinha, S., Cline, B., Ramamurthy, C., & Yeric, G. (2016). ASAP7: a 7-nm FinFET predictive process design kit. Microelectronics Journal, 53, 105–115.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Zhou, J., Papandreou-Suppappola, A. & Chakrabarti, C. Parallel Gibbs Sampler for Wavelet-Based Bayesian Compressive Sensing with High Reconstruction Accuracy. J Sign Process Syst (2020). https://doi.org/10.1007/s11265-020-01541-2
- Gibbs sampling
- Parallel implementation
- Relaxed computing
- Bayesian compressive sensing