Monte Carlo methods

Original Paper

Abstract

Monte Carlo methods deal with generating random variates from probability density functions in order to estimate unknown parameters or general functions of unknown parameters and to compute their expected values, variances and covariances. One generally works with the multivariate normal distribution due to the central limit theorem. However, if random variables with the normal distribution and random variables with a different distribution are combined, the normal distribution is not valid anymore. The Monte Carlo method is then needed to get the expected values, variances and covariances for the random variables with distributions different from the normal distribution. The error propagation by Monte Carlo methods is discussed and methods for generating random variates from the multivariate normal distribution and from the multivariate uniform distribution. The Monte Carlo integration is presented leading to the sampling–importance-resampling algorithm. Markov chain Monte Carlo methods provide by the Metropolis algorithm and the Gibbs sampler additional ways of generating random variates. A special topic is the Gibbs sampler for computing and propagating large covariance matrices. This task arises, for instance, when the geopotential is determined from satellite observations. The example of the minimal detectable outlier shows, how the Monte Carlo method is used to determine the power of a hypothesis test.

Keywords

Bayesian statistics SIR algorithm Metropolis algorithm Gibbs sampler Markov chain Monte Carlo methods 

Mathematics Subject Classification

62 Statistics 

Notes

Acknowledgements

The author is indebted to Willi Freeden for his invitation of this paper for GEM and to Jan Martin Brockmann for his valuable comments.

References

  1. Acko, B., Godina, A.: Verification of the conventional measuring uncertainty evaluation model with Monte Carlo simulation. Int. J. Simul. Model. 4, 76–84 (2005)CrossRefGoogle Scholar
  2. Alkhatib, H., Kutterer, H.: Estimation of measurement uncertainty of kinematic TLS observation process by means of Monte-Carlo methods. J. Appl. Geod. 7, 125–133 (2013)Google Scholar
  3. Alkhatib, H., Schuh, W.D.: Integration of the Monte Carlo covariance estimation strategy into tailored solution procedures for large-scale least squares problems. J. Geod. 81, 53–66 (2007)CrossRefMATHGoogle Scholar
  4. Alkhatib, H., Neumann, I., Kutterer, H.: Uncertainty modeling of random and systematic errors by means of Monte Carlo and fuzzy techniques. J. Appl. Geod. 3, 67–79 (2009)Google Scholar
  5. Arnold, S.: The Theory of Linear Models and Multivariate Analysis. Wiley, New York (1981)MATHGoogle Scholar
  6. Baarda, W.: Statistical Concepts in Geodesy. Publications on Geodesy, Vol. 2, Nr. 4. Netherlands Geodetic Commission, Delft (1967)Google Scholar
  7. Baarda, W.: A Testing Procedure for Use in Geodetic Networks. Publications on Geodesy, Vol. 2, Nr. 5. Netherlands Geodetic Commission, Delft (1968)Google Scholar
  8. Baselga, S.: Nonexistence of rigorous tests for multiple outlier detection in least-squares adjustment. J. Surv. Eng. 137, 109–112 (2011)CrossRefGoogle Scholar
  9. Beckman, R., Cook, R.: Outlier....s. Technometrics 25, 119–149 (1983)MathSciNetMATHGoogle Scholar
  10. Besag, J.: Spatial interaction and the statistical analysis of lattice systems. J. R. Stat. Soc. B 36, 192–236 (1974)MathSciNetMATHGoogle Scholar
  11. Box, G., Muller, M.: A note on the generation of random normal deviates. Ann. Math. Stat. 29, 610–611 (1958)CrossRefMATHGoogle Scholar
  12. Cramér, H.: Mathematical Methods of Statistics. Princeton University Press, Princeton (1946)MATHGoogle Scholar
  13. Dagpunar, J.: Principles of Random Variate Generation. Clarendon Press, Oxford (1988)MATHGoogle Scholar
  14. Devroye, L.: Non-uniform Random Variate Generation. Springer, Berlin (1986)CrossRefMATHGoogle Scholar
  15. Dietrich, C.: Uncertainty, Calibration and Probability, 2nd edn. Taylor & Francis, Boca Raton (1991)Google Scholar
  16. Doucet, A., Godsill, S., Andrieu, C.: On sequential Monte Carlo sampling methods for Bayesian filtering. Stat. Comput. 10, 197–208 (2000)CrossRefGoogle Scholar
  17. Falk, M.: A simple approach to the generation of uniformly distributed random variables with prescribed correlations. Commun. Stat. Simul. 28, 785–791 (1999)MathSciNetCrossRefMATHGoogle Scholar
  18. Gaida, W., Koch, K.R.: Solving the cumulative distribution function of the noncentral \(F\)-distribution for the noncentrality parameter. Sci. Bull. Stanisl. Staszic Univ. Min. Metall. Geod. B 90(1024), 35–44 (1985)Google Scholar
  19. Gelfand, A., Smith, A.: Sampling-based approaches to calculating marginal densities. J. Am. Stat. Assoc. 85, 398–409 (1990)MathSciNetCrossRefMATHGoogle Scholar
  20. Gelman, A., Carlin, J., Stern, H., Rubin, D.: Bayesian Data Analysis, 2nd edn. Chapman and Hall, Boca Raton (2004)MATHGoogle Scholar
  21. Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. PAMI–6, 721–741 (1984)CrossRefMATHGoogle Scholar
  22. Geman, S., McClure, D.: Statistical methods for tomographic image reconstruction. Bull. Int. Stat. Inst. 52–21(1), 5–21 (1987)MathSciNetGoogle Scholar
  23. Geman, D., Geman, S., Graffigne, C.: Locating texture and object boundaries. In: Devijver, P., Kittler, J. (eds.) Pattern Recognition Theory and Applications, pp. 165–177. Springer, Berlin (1987)CrossRefGoogle Scholar
  24. Gentle, J.: Random Number Generation and Monte Carlo Methods, 2nd edn. Springer, Berlin (2003)MATHGoogle Scholar
  25. Gilks, W.: Full conditional distributions. In: Gilks, W., Richardson, S., Spiegelhalter, D. (eds.) Markov Chain Monte Carlo in Practice, pp. 75–88. Chapman and Hall, London (1996)Google Scholar
  26. Golub, G., van Loan, C.: Matrix Computations. The Johns Hopkins University Press, Baltimore (1984)MATHGoogle Scholar
  27. Gordon, N., Salmond, D.: Bayesian state estimation for tracking and guidance using the bootstrap filter. J. Guidance Control Dyn. 18, 1434–1443 (1995)CrossRefGoogle Scholar
  28. Gundlich, B., Kusche, J.: Monte Carlo integration for quasi-linear models. In: Xu, P., Liu, J., Dermanis, A. (eds.) VI Hotine-Marussi Symposium on Theoretical and Computational Geodesy, pp. 337–344. Springer, Berlin (2008)CrossRefGoogle Scholar
  29. Gundlich, B., Koch, K.R., Kusche, J.: Gibbs sampler for computing and propagating large covariance matrices. J. Geod. 77, 514–528 (2003)CrossRefMATHGoogle Scholar
  30. Guo, J.F., Ou, J.K., Yuan, Y.B.: Reliability analysis for a robust M-estimator. J. Surv. Eng. 137, 9–13 (2011)CrossRefGoogle Scholar
  31. Hennes, M.: Konkurrierende Genauigkeitsmaße—Potential und Schwächen aus der Sicht des Anwenders. Allg. Vermess. Nachr. 114, 136–146 (2007)Google Scholar
  32. Huber, P.: Robust estimation of a location parameter. Ann. Math. Stat. 35, 73–101 (1964)MathSciNetCrossRefMATHGoogle Scholar
  33. ISO: Guide to the Expression of Uncertainty in Measurement. International Organization for Standardization, Geneve (1995)Google Scholar
  34. JCGM: Evaluation of measurement data—supplement 2 to the “Guide to the expression of uncertainty in measurement”—extension to any number of output quantities. JCGM 102:2011. Joint Committee for Guides in Metrology (2011). www.bipm.org/en/publications/guides/
  35. Kacker, R., Jones, A.: On use of Bayesian statistics to make the guide to the expression of uncertainty in measurement consistent. Metrologia 40, 235–248 (2003)CrossRefGoogle Scholar
  36. Kargoll, B.: On the theory and application of model misspecification tests in geodesy. Universität Bonn, Institut für Geodäsie und Geoinformation, Schriftenreihe 8, Bonn (2008)Google Scholar
  37. Knight, N., Wang, J., Rizos, C.: Generalised measures of reliability for multiple outliers. J. Geod. 84, 625–635 (2010)CrossRefGoogle Scholar
  38. Koch, K.R.: Ausreißertests und Zuverlässigkeitsmaße. Vermess. Raumordn. 45, 400–411 (1983)Google Scholar
  39. Koch, K.R.: Parameter Estimation and Hypothesis Testing in Linear Models, 2nd edn. Springer, Berlin (1999)CrossRefMATHGoogle Scholar
  40. Koch, K.R.: Monte-Carlo-Simulation für Regularisierungsparameter. ZfV-Z Geod. Geoinf. Landmanag. 127, 305–309 (2002)Google Scholar
  41. Koch, K.R.: Determining the maximum degree of harmonic coefficients in geopotential models by Monte Carlo methods. Stud. Geophys. Geod. 49, 259–275 (2005)CrossRefGoogle Scholar
  42. Koch, K.R.: Gibbs sampler by sampling–importance-resampling. J. Geod. 81, 581–591 (2007a)CrossRefMATHGoogle Scholar
  43. Koch, K.R.: Introduction to Bayesian Statistics, 2nd edn. Springer, Berlin (2007b)MATHGoogle Scholar
  44. Koch, K.R.: Determining uncertainties of correlated measurements by Monte Carlo simulations applied to laserscanning. J. Appl. Geod. 2, 139–147 (2008a)Google Scholar
  45. Koch, K.R.: Evaluation of uncertainties in measurements by Monte Carlo simulations with an application for laserscanning. J. Appl. Geod. 2, 67–77 (2008b)Google Scholar
  46. Koch, K.R.: Minimal detectable outliers as measures of reliability. J. Geod. 89, 483–490 (2015)CrossRefGoogle Scholar
  47. Koch, K.R.: Bayesian statistics and Monte Carlo methods. J. Geod. Sci. 8 (in preparation) (2018)Google Scholar
  48. Koch, K.R., Brockmann, J.: Systematic effects in laser scanning and visualization by confidence regions. J. Appl. Geod. 10(4), 247–257 (2016)Google Scholar
  49. Koch, K.R., Kargoll, B.: Outlier detection by the EM algorithm for laser scanning in rectangular and polar coordinate systems. J. Appl. Geod. 9, 162–173 (2015)Google Scholar
  50. Koch, K.R., Schmidt, M.: Deterministische und stochastische Signale. Dümmler, Bonn (1994). ftp://skylab.itg.uni-bonn.de/koch/00_textbooks/Determ_u_stoch_Signale.pdf
  51. Koch, K.R., Kusche, J., Boxhammer, C., Gundlich, B.: Parallel Gibbs sampling for computing and propagating large covariance matrices. ZfV-Z Geod. Geoinf. Landmanag. 129, 32–42 (2004)Google Scholar
  52. Kok, J.: Statistical analysis of deformation problems using Baarda’s testing procedures. In: “Forty Years of Thought”. Anniversary Volume on the Occasion of Prof. Baarda’s 65th Birthday 2, 470–488 (1982). DelftGoogle Scholar
  53. Kok, J.: On data snooping and multiple outlier testing. NOAA Technical Report NOS NGS 30. US Department of Commerce, National Geodetic Survey, Rockville (1984)Google Scholar
  54. Lehmann, R.: Improved critical values for extreme normalized and studentized residuals in Gauss–Markov models. J. Geod. 86, 1137–1146 (2012)CrossRefGoogle Scholar
  55. Lehmann, R.: On the formulation of the alternative hypothesis for geodetic outlier detection. J. Geod. 87, 373–386 (2013)CrossRefGoogle Scholar
  56. Leonard, T., Hsu, J.: Bayesian Methods. Cambridge University Press, Cambridge (1999)MATHGoogle Scholar
  57. Liu, J.: Monte Carlo Strategies in Scientific Computing. Springer, Berlin (2001)MATHGoogle Scholar
  58. Marsaglia, G., Bray, T.: A convenient method for generating normal variables. SIAM Rev. 6, 260–264 (1964)MathSciNetCrossRefMATHGoogle Scholar
  59. Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1092 (1953)CrossRefGoogle Scholar
  60. Nowel, K.: Application of Monte Carlo method to statistical testing in deformation analysis based on robust M-estimation. Surv. Rev. 48(348), 212–223 (2016)CrossRefGoogle Scholar
  61. O’Hagan, A.: Bayesian Inference, Kendall’s Advanced Theory of Statistics, vol. 2B. Wiley, New York (1994)MATHGoogle Scholar
  62. Pope, A.: The statistics of residuals and the detection of outliers. NOAA Technical Report NOS65 NGS1, US Department of Commerce, National Geodetic Survey, Rockville (1976)Google Scholar
  63. Proszynski, W.: Another approach to reliability measures for systems with correlated observations. J. Geod. 84, 547–556 (2010)CrossRefGoogle Scholar
  64. Roberts, G., Smith, A.: Simple conditions for the convergence of the Gibbs sampler and Metropolis–Hastings algorithms. Stoch. Process. Appl. 49, 207–216 (1994)MathSciNetCrossRefMATHGoogle Scholar
  65. Rubin, D.: Using the SIR algorithm to simulate posterior distributions. In: Bernardo, J., DeGroot, M., Lindley, D., Smith, A. (eds.) Bayesian Statistics 3, pp. 395–402. Oxford University Press, Oxford (1988)Google Scholar
  66. Rubinstein, R.: Simulation and the Monte Carlo Method. Wiley, New York (1981)CrossRefMATHGoogle Scholar
  67. Schader, M., Schmid, F.: Distribution function and percentage points for the central and noncentral F-distribution. Stat. Pap. 27, 67–74 (1986)MATHGoogle Scholar
  68. Siebert, B., Sommer, K.D.: Weiterentwicklung des GUM und Monte-Carlo-Techniken. Tech. Messen 71, 67–80 (2004)CrossRefGoogle Scholar
  69. Smith, A., Gelfand, A.: Bayesian statistics without tears: a sampling-resampling perspective. Am. Stat. 46, 84–88 (1992)MathSciNetGoogle Scholar
  70. Smith, A., Roberts, G.: Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods. J. R. Stat. Soc. B 55, 3–23 (1993)MathSciNetMATHGoogle Scholar
  71. Staff of the Geodetic Computing Center, S.: The Delft approach for the design and computation of geodetic networks. In: “Forty Years of Thought”, Anniversary Volume on the Occasion of Prof. Baarda’s 65th Birthday 1, 202–274 (1982). DelftGoogle Scholar
  72. Teunissen, P.: Adjusting and testing with the models of the affine and similarity transformation. Manuscr. Geod. 11, 214–225 (1986)Google Scholar
  73. Teunissen, P.: Testing theory: an introduction. MGP, Delft University of Technology, Department of Mathematical Geodesy and Positioning, Delft (2000)Google Scholar
  74. Teunissen, P., de Bakker, P.: Single-receiver single-channel multi-frequency GNSS integrity: outliers, slips, and ionospheric disturbances. J. Geod. 87, 161–177 (2013)CrossRefGoogle Scholar
  75. van Dorp, J., Kotz, S.: Generalized trapezoidal distributions. Metrika 58, 85–97 (2003)MathSciNetCrossRefMATHGoogle Scholar
  76. Wilks, S.: Mathematical Statistics. Wiley, New York (1962)MATHGoogle Scholar
  77. Xu, P.: Random simulation and GPS decorrelation. J. Geod. 75, 408–423 (2001)CrossRefMATHGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2017

Authors and Affiliations

  1. 1.Theoretical Geodesy Group, Institute for Geodesy and GeoinformationUniversity of BonnBonnGermany

Personalised recommendations