Abstract
In this chapter, correlated and distributed sources without cooperation at the encoder are considered. For these sources, the best achievable performance in the rate-distortion sense of any distributed compressed sensing scheme is derived, under the constraint of high-rate quantization. Moreover, under this model we derive a closed-form expression of the rate gain achieved by taking into account the correlation of the sources at the receiver and a closed-form expression of the average performance of the oracle receiver for independent and joint reconstruction. Finally, we show experimentally that the exploitation of the correlation between the sources performs close to optimal and that the only penalty is due to the missing knowledge of the sparsity support as in (non-distributed) compressed sensing. Even if the derivation is performed in the large system regime, where signal and system parameters tend to infinity, numerical results show that the equations match simulations for parameter values of practical interest.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
This definition complies with the usual form \(y = {\varPhi }x\) where the variance \({\sigma }^{2}_{\varPhi }\) of the elements of \({\varPhi }\) depends on m. Here, we wanted to keep \({\sigma }^{2}_{\varPhi }\) independent of system parameters.
- 2.
The average performance is obtained averaging over all random variables i.e., the measurement matrix, the nonzero components \({\theta }\) and noise, as for example in [10].
References
Dai, W., Milenkovic, O.: Information theoretical and algorithmic approaches to quantized compressive sensing. IEEE Trans. Commun. 59, 1857–1866 (2011)
Weidmann, C., Vetterli, M.: Rate distortion behavior of sparse sources. IEEE Trans. Inf. Theory 58(8), 4969–4992 (2012)
Fletcher, A., Rangan, S., Goyal, V.: On the rate-distortion performance of compressed sensing. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), vol. 3. IEEE (2007)
Goyal, V., Fletcher, A., Rangan, S.: Compressive sampling and lossy compression. IEEE Sig. Process. Mag. 25(2), 48–56 (2008)
Baron, D., Duarte, M.F., Wakin, M.B., Sarvotham, S., Baraniuk, R.G.: Distributed compressive sensing. arXiv preprint arXiv:0901.3403 (2009)
Duarte, M.F., Wakin, M.B., Baron, D., Sarvotham, S., Baraniuk, R.G.: Measurement bounds for sparse signal ensembles via graphical models. IEEE Trans. Inf. Theory 59(7), 4280–4289 (2013)
Coluccia, G., Magli, E., Roumy, A., Toto-Zarasoa, V., et al.: Lossy compression of distributed sparse sources: a practical scheme. In: 2011 European Signal Processing Conference (EUSIPCO) (2011)
Liu, Z., Cheng, S., Liveris, A.D., Xiong, Z.: Slepian-Wolf coded nested lattice quantization for Wyner-Ziv coding: high-rate performance analysis and code design. IEEE Trans. Inf. Theory 52(10), 4358–4379 (2006)
Davenport, M.A., Laska, J.N., Treichler, J.R., Baraniuk, R.G.: The pros and cons of compressive sensing for wideband signal acquisition: noise folding versus dynamic range. CoRR abs/1104.4842 (2011)
Laska, J.N., Baraniuk, R.G.: Regime change: bit-depth versus measurement-rate in compressive sensing. IEEE Trans. Sig. Process. 60(7), 3496–3505 (2012)
Cook, R.D., Forzani, L.: On the mean and variance of the generalized inverse of a singular wishart matrix. Electron. J. Stat. 5, 146–158 (2011)
Slepian, D., Wolf, J.: Noiseless coding of correlated information sources. IEEE Trans. Inf. Theory 19(4), 471–480 (1973)
Cover, T.: A proof of the data compression theorem of Slepian and Wolf for ergodic sources (Corresp.). IEEE Trans. Inf. Theory 21(2), 226–228 (1975)
Rebollo-Monedero, D., Rane, S., Aaron, A., Girod, B.: High-rate quantization and transform coding with side information at the decoder. Sig. Process. 86(11), 3160–3179 (2006). November
Wyner, A.: The rate-distortion function for source coding with side information at the decoder-ii: general sources. Inf. Control 38(1), 60–80 (1978)
Bassi, F., Kieffer, M., Weidmann, C.: Wyner-Ziv coding with uncertain side information quality. In: European Signal Processing Conference (EUSIPCO) (2010)
Khoshnevisan, D.: Probability. Graduate Studies in Mathematics, AMS, New York (2007)
Coluccia, G., Roumy, A., Magli, E.: Operational rate-distortion performance of single-source and distributed compressed sensing. IEEE Trans. Commun. 62(6), 2022–2033 (2014). June
Coluccia, G., Roumy, A., Magli, E.: Exact performance analysis of the oracle receiver for compressed sensing reconstruction. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE (2014), pp. 1005–1009
Candès, E.J., Romberg, J.K., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)
Candès, E.J.: The restricted isometry property and its implications for compressed sensing. Compte Rendus de l’Academie des Sciences. Paris, France, ser. I 346, 589–592 (2008)
Eldar, Y.C., Kutyniok, G., (eds.): Compressed Sensing: Theory and Applications. Cambridge University Press, Cambridge (2012)
Díaz-García, J.A., Gutiérrez-Jáimez, R.: Distribution of the generalised inverse of a random matrix and its applications. J. Stat. Plan. Inf. 136(1), 183–192 (2006)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2015 The Author(s)
About this chapter
Cite this chapter
Coluccia, G., Ravazzi, C., Magli, E. (2015). Rate-Distortion Theory of Distributed Compressed Sensing. In: Compressed Sensing for Distributed Systems. SpringerBriefs in Electrical and Computer Engineering(). Springer, Singapore. https://doi.org/10.1007/978-981-287-390-3_3
Download citation
DOI: https://doi.org/10.1007/978-981-287-390-3_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-287-389-7
Online ISBN: 978-981-287-390-3
eBook Packages: EngineeringEngineering (R0)