Decreasing Cramer–Rao lower bound by preprocessing steps

  • Sara Monem Khorasani
  • Ghosheh Abed HodtaniEmail author
  • Mohammad Molavi Kakhki
Original Paper


In this paper, having reviewed necessary preliminaries, including sparsity, Tsallis entropy, diversity, preprocessing, fisher information, and Cramer–Rao bound, we analyze the impact of preprocessing a signal on the signal sparsity related to Cramer–Rao lower bound and its main feature, for example, its reconstruction error. The main idea of this paper is to increase the sparsity of a vector, or to decrease its nonzero elements, then to compute the estimation error bound before and after sparsifying the signal. Finally, the claims are validated numerically. We implement Savitzky–Golay filtering on some ECG signals (applying MIT-BIH database of cardiac signals) and then compress them, to illustrate that the sparsity (the reconstruction error) of non-filtered signal was less (more) than that of filtered one. The results can be useful in signal compression and transmission procedures to have fewer recovery errors.


Sparsity Gini index Estimator variance Cramer–Rao bound (CRB) Fisher information matrix (FIM) Preprocessing 



  1. 1.
    Motghare, N., Mewada, A.: Adapting ECG data stream mining for health care application. Int. J. Sci. Eng. Res. 5(9), 2122–2132 (2014)Google Scholar
  2. 2.
    Zonoobi, D., Kassim, A.A., Venkatesh, Y.V.: Gini index as sparsity measure for signal reconstruction from compressive samples. IEEE J. Sel. Top. Signal Process. 5(5), 927–932 (2011)CrossRefGoogle Scholar
  3. 3.
    Reeves, G., Gastpar, M.: The sampling rate-distortion tradeoff for sparsity pattern recovery in compressed sensing. IEEE Trans. Inf. Theory 58(5), 3065–3092 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Bi, X., Chen, X., Li, X., Leng, L.: Energy-based adaptive matching pursuit algorithm for binary sparse signal reconstruction in compressed sensing. Signal Image Video Process. 8(6), 1039–1048 (2014)CrossRefGoogle Scholar
  5. 5.
    Protter, M., Elad, M.: Image sequence denoising via sparse and redundant representations. IEEE Trans. Image Process. 18(1), 27–35 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  6. 6.
    Yang, Z., Zhang, C., Xie, L.: Robustly stable signal recovery in compressed sensing with structured matrix perturbation. IEEE Trans. Signal Process. 60(9), 4658–4671 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Li, Y., Huffel, S.V.: Multi-structural signal recovery for biomedical compressive sensing. IEEE Trans. Biomed. Eng. 60(10), 2794–2805 (2013)CrossRefGoogle Scholar
  8. 8.
    Ben-Haim, Z., Eldar, Y.C.: The Cramér-Rao bound for estimating a sparse parameter vector. IEEE Trans. Signal Process. 58(6), 2671–2682 (2010)zbMATHCrossRefGoogle Scholar
  9. 9.
    Tune, P.: Computing constrained Cramér-Rao bounds. IEEE Trans. Signal Process. 60(10), 305–449 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Stoica, P., Li, J.: Study of the Cramer–Rao bound as the numbers of observations and unknown parameters increase. IEEE Signal Process. Lett. 3(2), 299–300 (1996)CrossRefGoogle Scholar
  11. 11.
    Babadi, B., Kalouptsidis, N., Tarokh, V.: Asymptotic achievability of the Cramér-Rao bound for noisy compressive sampling. IEEE Trans. Signal Process. 57(3), 1233–1236 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  12. 12.
    Pakrooh, P., Pezeshki, A., Scharf, L.L., Cochran, D., Howard, S.D.: Analysis of Fisher information and the Cramér-Rao bound for nonlinear parameter estimation after compressed sensing. In: IEEE International Conference on Acoustic, Speech and Signal Processing, ICASSP (2013)Google Scholar
  13. 13.
    Niazadeh, R., Babaie-Zadeh, M., Jutten, Ch.: On the achievability of Cramér-Rao bound in noisy compressed sensing. IEEE Trans. Signal Process. 60(1), 518–526 (2012)MathSciNetzbMATHCrossRefGoogle Scholar
  14. 14.
    Stein, M., Mezghani, A., Nossek, J.A.: A lower bound for the fisher information measure. IEEE Signal Process. Lett. 21(7), 796–799 (2014)CrossRefGoogle Scholar
  15. 15.
    Sellone, F., Falleti, E.: Cramer–Rao bound of spatio-temporal linear pre-processing in parameter estimation from sensor array. Signal Process. J. 84(2), 387–405 (2004)zbMATHCrossRefGoogle Scholar
  16. 16.
    Shaghaghi, M., Vorobyov, S.A.: Cramér-Rao bound for sparse signals fitting the low-rank model with small number of parameters. IEEE Signal Process. Lett. 22(9), 1497–1501 (2015)CrossRefGoogle Scholar
  17. 17.
    Khomchuk, P., Blum, R.S., Bilik, I.: Performance analysis of target parameters estimation using multiple widely separated antenna arrays. IEEE Trans. Aerosp. Electron. Syst. 52(5), 2413–2435 (2016)CrossRefGoogle Scholar
  18. 18.
    Hasegawa, H.: Effects of correlated variability on information entropies in none extensive systems. Phys. Rev. E 78, 021102 (2008)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Zachariah, D., Stoica, P.: Cramer–Rao bound analog of Bayes’ rule. IEEE Signal Process. Magazine 32(2), 164–168 (2015)CrossRefGoogle Scholar
  20. 20.
    Pastor, G., Mora-Jimenez, I., Antti Abd, R.J., Caamano, A.J.: Mathematics of sparsity and entropy: axioms, core functions and sparse recovery. IEEE Trans. Inf. Theory (2015)Google Scholar
  21. 21.
    Tadejko, P., Rakowski, W.: Mathematical morphology based ECG feature extraction for the purpose of heartbeat classification. In: 6th International Conference on Computer Information Systems and Industrial Management Applications (CISIM) (2007)Google Scholar
  22. 22.
    Ravishankar, S., Bresler, Y.: Learning sparsifying transforms. IEEE Trans. On Signal Processing 61(5), 1072–1086 (2013)MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Hurley, N., Rickard, S.: Comparing measures of sparsity. IEEE Trans. Inf. Theory 55(10), 3601–3608 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  24. 24.
    Furuichi, S.: On uniqueness theorems for Tsallis entropy and Tsallis relative entropy. IEEE Trans. Inf. Theory 51(10), 3638–3645 (2005)MathSciNetzbMATHCrossRefGoogle Scholar
  25. 25.
    Schafer, R.W.: What is a Savitzky-Golay filter? IEEE Signal Process. Mag. 28, 111–117 (2011)CrossRefGoogle Scholar
  26. 26.
    Paninski, L.: Master’s advanced undergraduate level course in mathematical statistics, part 3, estimation theory (2005)Google Scholar
  27. 27.
    Liu, A., Lau, V., Kong, X.: A sparse MLE approach for joint interference mitigation and data recovery. In: IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP) (2014)Google Scholar
  28. 28.
    Zhang, J., Kennedy, R.A., Abhayapala, T.D.: Cramér-Rao lower bounds for the time delay estimation of UWB signals. In: IEEE International Conference on Communications (2004)Google Scholar
  29. 29.
    Ramasamy, D., Venkateswaran, S., Madhow, U.: Compressive estimation in AWGN: general observations and a case study. In: Conference Record of the Forty Sixth Asilomar Conference on Signals, Systems and Computers (2012)Google Scholar
  30. 30.
    He, Q., Hu, J., Blum, R.S., Wu, Y.: Generalized Cramer–Rao bound for joint estimation of target position and velocity for active and passive radar networks. IEEE Trans. Signal Process. 64(8), 2078–2089 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  31. 31.
    Steinwandt, J., Roemer, F., Haardt, M., Del Galdo, G.: Deterministic Cramér-Rao bound for strictly non-circular sources and analytical analysis of the achievable gains. IEEE Trans. Signal Process. 64(17), 4417–4431 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  32. 32.
    Begriche, Y., Thameri, M., Abed-Meraim, K.: Exact Cramér-Rao bound for near field localization, exact Cramer Rao bound for near field source localization. In: 11th International Conference on Information Science, Signal Processing and their Applications (ISSPA) (2012)Google Scholar
  33. 33.
    Zayyani, H., Babaie-zadeh, M., Haddadi, F., Jutten, Ch.: On the Cramér-Rao bound for estimating the mixing matrix in noisy sparse component analysis. IEEE Signal Process. Lett. 15, 609–612 (2008)CrossRefGoogle Scholar
  34. 34.
    Routtenberg, T., Tong, L.: Estimation after parameter selection: performance analysis and estimation methods. IEEE Trans. Signal Process. 64(20), 5268–5281 (2016)MathSciNetzbMATHCrossRefGoogle Scholar
  35. 35.
    Park, S., Serpedin, E., Qaraqe, Kh: Gaussian assumption: the least favorable but the most useful. IEEE Signal Process. Mag. 30(3), 183–186 (2013)CrossRefGoogle Scholar
  36. 36.
    MIT-BIH Arrhythmia Database. (2005)

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  • Sara Monem Khorasani
    • 1
  • Ghosheh Abed Hodtani
    • 1
    Email author
  • Mohammad Molavi Kakhki
    • 1
  1. 1.Ferdowsi University of MashhadMashhadIran

Personalised recommendations