Decreasing Cramer–Rao lower bound by preprocessing steps


In this paper, having reviewed necessary preliminaries, including sparsity, Tsallis entropy, diversity, preprocessing, fisher information, and Cramer–Rao bound, we analyze the impact of preprocessing a signal on the signal sparsity related to Cramer–Rao lower bound and its main feature, for example, its reconstruction error. The main idea of this paper is to increase the sparsity of a vector, or to decrease its nonzero elements, then to compute the estimation error bound before and after sparsifying the signal. Finally, the claims are validated numerically. We implement Savitzky–Golay filtering on some ECG signals (applying MIT-BIH database of cardiac signals) and then compress them, to illustrate that the sparsity (the reconstruction error) of non-filtered signal was less (more) than that of filtered one. The results can be useful in signal compression and transmission procedures to have fewer recovery errors.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3


  1. 1.

    Motghare, N., Mewada, A.: Adapting ECG data stream mining for health care application. Int. J. Sci. Eng. Res. 5(9), 2122–2132 (2014)

    Google Scholar 

  2. 2.

    Zonoobi, D., Kassim, A.A., Venkatesh, Y.V.: Gini index as sparsity measure for signal reconstruction from compressive samples. IEEE J. Sel. Top. Signal Process. 5(5), 927–932 (2011)

    Article  Google Scholar 

  3. 3.

    Reeves, G., Gastpar, M.: The sampling rate-distortion tradeoff for sparsity pattern recovery in compressed sensing. IEEE Trans. Inf. Theory 58(5), 3065–3092 (2012)

    MathSciNet  Article  Google Scholar 

  4. 4.

    Bi, X., Chen, X., Li, X., Leng, L.: Energy-based adaptive matching pursuit algorithm for binary sparse signal reconstruction in compressed sensing. Signal Image Video Process. 8(6), 1039–1048 (2014)

    Article  Google Scholar 

  5. 5.

    Protter, M., Elad, M.: Image sequence denoising via sparse and redundant representations. IEEE Trans. Image Process. 18(1), 27–35 (2009)

    MathSciNet  Article  Google Scholar 

  6. 6.

    Yang, Z., Zhang, C., Xie, L.: Robustly stable signal recovery in compressed sensing with structured matrix perturbation. IEEE Trans. Signal Process. 60(9), 4658–4671 (2012)

    MathSciNet  Article  Google Scholar 

  7. 7.

    Li, Y., Huffel, S.V.: Multi-structural signal recovery for biomedical compressive sensing. IEEE Trans. Biomed. Eng. 60(10), 2794–2805 (2013)

    Article  Google Scholar 

  8. 8.

    Ben-Haim, Z., Eldar, Y.C.: The Cramér-Rao bound for estimating a sparse parameter vector. IEEE Trans. Signal Process. 58(6), 2671–2682 (2010)

    Article  Google Scholar 

  9. 9.

    Tune, P.: Computing constrained Cramér-Rao bounds. IEEE Trans. Signal Process. 60(10), 305–449 (2012)

    MathSciNet  Article  Google Scholar 

  10. 10.

    Stoica, P., Li, J.: Study of the Cramer–Rao bound as the numbers of observations and unknown parameters increase. IEEE Signal Process. Lett. 3(2), 299–300 (1996)

    Article  Google Scholar 

  11. 11.

    Babadi, B., Kalouptsidis, N., Tarokh, V.: Asymptotic achievability of the Cramér-Rao bound for noisy compressive sampling. IEEE Trans. Signal Process. 57(3), 1233–1236 (2009)

    MathSciNet  Article  Google Scholar 

  12. 12.

    Pakrooh, P., Pezeshki, A., Scharf, L.L., Cochran, D., Howard, S.D.: Analysis of Fisher information and the Cramér-Rao bound for nonlinear parameter estimation after compressed sensing. In: IEEE International Conference on Acoustic, Speech and Signal Processing, ICASSP (2013)

  13. 13.

    Niazadeh, R., Babaie-Zadeh, M., Jutten, Ch.: On the achievability of Cramér-Rao bound in noisy compressed sensing. IEEE Trans. Signal Process. 60(1), 518–526 (2012)

    MathSciNet  Article  Google Scholar 

  14. 14.

    Stein, M., Mezghani, A., Nossek, J.A.: A lower bound for the fisher information measure. IEEE Signal Process. Lett. 21(7), 796–799 (2014)

    Article  Google Scholar 

  15. 15.

    Sellone, F., Falleti, E.: Cramer–Rao bound of spatio-temporal linear pre-processing in parameter estimation from sensor array. Signal Process. J. 84(2), 387–405 (2004)

    Article  Google Scholar 

  16. 16.

    Shaghaghi, M., Vorobyov, S.A.: Cramér-Rao bound for sparse signals fitting the low-rank model with small number of parameters. IEEE Signal Process. Lett. 22(9), 1497–1501 (2015)

    Article  Google Scholar 

  17. 17.

    Khomchuk, P., Blum, R.S., Bilik, I.: Performance analysis of target parameters estimation using multiple widely separated antenna arrays. IEEE Trans. Aerosp. Electron. Syst. 52(5), 2413–2435 (2016)

    Article  Google Scholar 

  18. 18.

    Hasegawa, H.: Effects of correlated variability on information entropies in none extensive systems. Phys. Rev. E 78, 021102 (2008)

    MathSciNet  Article  Google Scholar 

  19. 19.

    Zachariah, D., Stoica, P.: Cramer–Rao bound analog of Bayes’ rule. IEEE Signal Process. Magazine 32(2), 164–168 (2015)

    Article  Google Scholar 

  20. 20.

    Pastor, G., Mora-Jimenez, I., Antti Abd, R.J., Caamano, A.J.: Mathematics of sparsity and entropy: axioms, core functions and sparse recovery. IEEE Trans. Inf. Theory (2015)

  21. 21.

    Tadejko, P., Rakowski, W.: Mathematical morphology based ECG feature extraction for the purpose of heartbeat classification. In: 6th International Conference on Computer Information Systems and Industrial Management Applications (CISIM) (2007)

  22. 22.

    Ravishankar, S., Bresler, Y.: Learning sparsifying transforms. IEEE Trans. On Signal Processing 61(5), 1072–1086 (2013)

    MathSciNet  Article  Google Scholar 

  23. 23.

    Hurley, N., Rickard, S.: Comparing measures of sparsity. IEEE Trans. Inf. Theory 55(10), 3601–3608 (2009)

    MathSciNet  Article  Google Scholar 

  24. 24.

    Furuichi, S.: On uniqueness theorems for Tsallis entropy and Tsallis relative entropy. IEEE Trans. Inf. Theory 51(10), 3638–3645 (2005)

    MathSciNet  Article  Google Scholar 

  25. 25.

    Schafer, R.W.: What is a Savitzky-Golay filter? IEEE Signal Process. Mag. 28, 111–117 (2011)

    Article  Google Scholar 

  26. 26.

    Paninski, L.: Master’s advanced undergraduate level course in mathematical statistics, part 3, estimation theory (2005)

  27. 27.

    Liu, A., Lau, V., Kong, X.: A sparse MLE approach for joint interference mitigation and data recovery. In: IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP) (2014)

  28. 28.

    Zhang, J., Kennedy, R.A., Abhayapala, T.D.: Cramér-Rao lower bounds for the time delay estimation of UWB signals. In: IEEE International Conference on Communications (2004)

  29. 29.

    Ramasamy, D., Venkateswaran, S., Madhow, U.: Compressive estimation in AWGN: general observations and a case study. In: Conference Record of the Forty Sixth Asilomar Conference on Signals, Systems and Computers (2012)

  30. 30.

    He, Q., Hu, J., Blum, R.S., Wu, Y.: Generalized Cramer–Rao bound for joint estimation of target position and velocity for active and passive radar networks. IEEE Trans. Signal Process. 64(8), 2078–2089 (2016)

    MathSciNet  Article  Google Scholar 

  31. 31.

    Steinwandt, J., Roemer, F., Haardt, M., Del Galdo, G.: Deterministic Cramér-Rao bound for strictly non-circular sources and analytical analysis of the achievable gains. IEEE Trans. Signal Process. 64(17), 4417–4431 (2016)

    MathSciNet  Article  Google Scholar 

  32. 32.

    Begriche, Y., Thameri, M., Abed-Meraim, K.: Exact Cramér-Rao bound for near field localization, exact Cramer Rao bound for near field source localization. In: 11th International Conference on Information Science, Signal Processing and their Applications (ISSPA) (2012)

  33. 33.

    Zayyani, H., Babaie-zadeh, M., Haddadi, F., Jutten, Ch.: On the Cramér-Rao bound for estimating the mixing matrix in noisy sparse component analysis. IEEE Signal Process. Lett. 15, 609–612 (2008)

    Article  Google Scholar 

  34. 34.

    Routtenberg, T., Tong, L.: Estimation after parameter selection: performance analysis and estimation methods. IEEE Trans. Signal Process. 64(20), 5268–5281 (2016)

    MathSciNet  Article  Google Scholar 

  35. 35.

    Park, S., Serpedin, E., Qaraqe, Kh: Gaussian assumption: the least favorable but the most useful. IEEE Signal Process. Mag. 30(3), 183–186 (2013)

    Article  Google Scholar 

  36. 36.

    MIT-BIH Arrhythmia Database. (2005)

Download references

Author information



Corresponding author

Correspondence to Ghosheh Abed Hodtani.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Khorasani, S.M., Hodtani, G.A. & Kakhki, M.M. Decreasing Cramer–Rao lower bound by preprocessing steps. SIViP 14, 781–789 (2020).

Download citation


  • Sparsity
  • Gini index
  • Estimator variance
  • Cramer–Rao bound (CRB)
  • Fisher information matrix (FIM)
  • Preprocessing