Performance Evaluation of Gibbs Sampling for Bayesian Extracting Sinusoids

  • M. CevriEmail author
  • D. Üstündag
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 307)


This chapter involves problems of estimating parameters of sinusoids from white noisy data by using Gibbs sampling (GS) in a Bayesian inferential framework which allows us to incorporate prior knowledge about the nature of sinusoidal data into the model. Modifications of its algorithm is tested on data generated from synthetic signals and its performance is compared with conventional estimators such as Maximum Likelihood (ML) and Discrete Fourier Transform (DFT) under a variety of signal to noise ratio (SNR) conditions and different lengths of data sampling (N), regarding to Cramér–Rao lower bound (CRLB) that is a limit on the best possible performance achievable by an unbiased estimator given a dataset. All simulation results show its effectiveness in frequency and amplitude estimation of noisy sinusoids.


Bayesian inference Parameter estimation Gibbs sampling Cramér–Rao lower bound and Power spectral density 



This work has been supported by the Research Fund of Istanbul University with project numbers are UDP-33672 and YADOP-19681.


  1. 1.
    Andreiu C, Doucet A (1999) Joint Bayesian model selection and estimation of noisy sinusoids via reversible jump MCMC, IEEE Transactions on Signal Processing, 47: 2667–2676CrossRefGoogle Scholar
  2. 2.
    Bernardo JM, Smith AFM (2000) Bayesian theory, Willey Series in Probability and Statistics New YorkGoogle Scholar
  3. 3.
    Box GEP, Tiao C (1992) Bayesian inference in statistical analysis, New YorkGoogle Scholar
  4. 4.
    Bretthorst GL (1997) Bayesian spectrum analysis and parameter estimation, Lecture Notes in Statistics, Springer-Verlag Berlin Heidelberg New YorkGoogle Scholar
  5. 5.
    Brooks SP, Gelman A (1997) General methods for monitoring convergence of iterative simulations, Journal of Computational and Graphical Statistics, 7: 434–455MathSciNetGoogle Scholar
  6. 6.
    Cevri M, Ustundag D (2012) Bayesian recovery of sinusoids from noisy data with parallel tempering, IET Signal Process., 6 (7): 673–683Google Scholar
  7. 7.
    Cevri M, Ustundag D (2013) Performance analysis of Gibbs sampling for Bayesian extracting sinusoids, Proceedings of the 2013 International Conference on Systems, Control, Signal Processing and Informatics, Rhodes Island, Greece, 128–134Google Scholar
  8. 8.
    Cooley JW, Tukey, JW (1965) An algorithm for the machine calculation of complex Fourier series, Mathematics of Computation, 19: 297–301CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Cox RT (1946) Probability, frequency, and reasonable expectation, American Journal of Physics, 14: 1–13CrossRefzbMATHMathSciNetGoogle Scholar
  10. 10.
    Diaconis P, Khare K, Coste LS (2008) Gibbs sampling, exponential families and orthogonal polynomials, Statistical Science, 23(2): 151–178CrossRefMathSciNetGoogle Scholar
  11. 11.
    Dou L, Hodgson RJW (1995a) Bayesian inference and Gibbs sampling in spectral analysis and parameter estimation I, Inverse Problem, 11:1069–1085CrossRefzbMATHMathSciNetGoogle Scholar
  12. 12.
    Dou L, Hodgson RJW (1995b) Bayesian inference and Gibbs sampling in spectral analysis and parameter estimation II, Inverse Problem, 11:121–137CrossRefGoogle Scholar
  13. 13.
    Gelfand AE, Smith AFM (1990) Sampling based approaches to calculating marginal densities. J. Amer. Statist. Assoc., 85:398–409CrossRefzbMATHMathSciNetGoogle Scholar
  14. 14.
    Gelman AB, Stern HS, Rubin DB (1995) Bayesian data analysis, Chapman & Hall/CRCGoogle Scholar
  15. 15.
    Geman S, Geman D (1984) Stochastic relaxation, Gibbs distribution and Bayesian restoration of images. IEE Transactions on Pattern Analysis and machine Intelligence, 6:721–741zbMATHGoogle Scholar
  16. 16.
    Gregory P (2005) Bayesian logical data analysis for the physical science, Cambridge University Press, United KingdomGoogle Scholar
  17. 17.
    Händel P (2008) Parameter estimation employing a dual-channel sine-wave model under a Gaussian assumption, IEEE Transactions on Instrumentation and Measurement, 57(8): 1661–1669Google Scholar
  18. 18.
    Harney HL (2003) Bayesian inference: Parameter estimation and decisions, Springer-Verlag, Berlin HeidelbergGoogle Scholar
  19. 19.
    Hastings, W K (1970) Monte carlo sampling methods using Markov chains, and their applications, Biometrika, 57:97–109CrossRefzbMATHGoogle Scholar
  20. 20.
    Jackman S (2000) American journal of political science, 44(2):375–404Google Scholar
  21. 21.
    Jaynes ET (1987) Bayesian Spectrum and Chirp Analysis, In Proceedings of the Third Workshop on Maximum Entropy and Bayesian Methods, Ed. C. Ray Smith and D. Reidel, Boston, pp. 1–37Google Scholar
  22. 22.
    Jaynes ET (2003) Probability theory: The logic of science, Cambridge University Press, United KingdomCrossRefGoogle Scholar
  23. 23.
    Kay SM (1984) Accurate frequency estimation at low signal-to-noise ratio, IEEE Trans. Acoust., Speech, Signal Processing, ASSP-32, pp. 540–547Google Scholar
  24. 24.
    Kay SM (1993) Fundamentals of statistical signal processing: Estimation theory, Prentice-Hall, Englewood Cliffs, NJ, pp. 56–57Google Scholar
  25. 25.
    Kenefic RJ, Nuttall AH (1987) Maximum likelihood estimation of the parameters of tone using real discrete data, IEEE J. Oceanic Eng., 12 (1): 279–280Google Scholar
  26. 26.
    MacKay D (2003) Information theory, inference and learning algorithms, Cambridge University PressGoogle Scholar
  27. 27.
    Metropolis N, Rosenbluth A, Rosenbluth, M, Teller A, Teller E (1953) Equation of states calculations by fast computing machines, Journal of chemical physics, 21: 1087–1092Google Scholar
  28. 28.
    Michalopouloua ZH, Picarelli M (2005) Gibbs sampling for time-delay-and amplitude estimation in underwater acoustics, J. Acoust. Soc. Am., 117:799–808CrossRefGoogle Scholar
  29. 29.
    Quinn BG (1994) Estimating frequency by interpolation using Fourier coefficients, IEEE Trans. Signal Process., 42 (5): 1264–1268Google Scholar
  30. 30.
    Rife DC, Boorstyn RR (1974) Single-tone parameter estimation from discrete-time observations, IEEE Transactions on Information Theory, 20:591–598CrossRefzbMATHGoogle Scholar
  31. 31.
    Ristic B, Arulampalam S, Gordon N (2004) Beyond the Kalman filter particle filters for tracking applications, Artech House, LondonGoogle Scholar
  32. 32.
    Stoica P, Moses RL (2005) Spectral analysis of signals, Prentice HallGoogle Scholar
  33. 33.
    Swendsen RH, Wang JS (1986) Physical review of letters, 57:2607–2609Google Scholar
  34. 34.
    Tanner, MA (1996) Tools for Statistical Inference, 3rd ed. Springer-Verlag, New YorkCrossRefzbMATHGoogle Scholar
  35. 35.
    Tanner M, Wong W (1987) The calculation of posterior distributions by data augmentation (with discussion), J. Amer. Statist. Assoc., 82:528–550CrossRefzbMATHMathSciNetGoogle Scholar
  36. 36.
    Üstündağ D, Cevri M (2008) Estimating parameters of sinusoids from noisy data using Bayesian inference with simulated annealing, Wseas Transactions On Signal Processing, 7: 432–441Google Scholar
  37. 37.
    Üstündağ D, Cevri M (2011) Recovering sinusoids from noisy data using Bayesian inference with simulated annealing, Mathematical & Computational Applications, 16(2): 382–391Google Scholar
  38. 38.
    Ustundag D, Cevri M (2012) Simulated annealing—advances, applications and hybridizations, In Tech, Croatia, ISBN: 978-953-51-0710-1. pp. 67–90Google Scholar
  39. 39.
    Ustundag D, Cevri M (2013) Comparison of Bayesian methods for recovering sinusoids, Proceedings of the 2013 International Conference on Systems, Control, Signal Processing and Informatics, Rhodes Island, Greece, 120–127Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Faculty of Science, Department of MathematicsIstanbul UniversityIstanbulTurkey
  2. 2.Faculty of Science and Letters, Department of MathematicsMarmara UniversityIstanbulTurkey

Personalised recommendations