Skip to main content
SpringerLink
Log in

More on Estimation

  • Chapter
  • First Online:
Random Fields for Spatial Data Modeling

Part of the book series: Advances in Geographic Information Science ((AGIS))

  • 1734 Accesses

Abstract

This chapter discusses estimation methods that are less established and not as commonly used as those presented in the preceding chapter. For example, the method of normalized correlations is relatively new, and its statistical properties have not been fully explored. The method of maximum entropy was used by Edwin Thompson Jaynes to derive statistical mechanics based on information theory [406, 407]. Following the work of Jaynes, maximum entropy has found several applications in physics [674, 753], image processing [315, 754], and machine learning [521, 561].

In desperation I asked Fermi whether he was not impressed by the agreement between our calculated numbers and his measured numbers. He replied, “How many arbitrary parameters did you use for your calculations?” I thought for a moment about our cut-off procedures and said, “Four.” He said, “I remember my friend Johnny von Neumann used to say, with four parameters I can fit an elephant, and with five I can make him wiggle his trunk.”

Freeman Dyson

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 129.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    To review constrained optimization with Lagrange multipliers consider [673, Chap. 16.5] and [67].

References

  1. Ambikasaran, S., Foreman-Mackey, D., Greengard, L., Hogg, D.W., O’Neil, M.: Fast direct methods for Gaussian processes. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 252–265 (2016)

    Article  ADS  Google Scholar 

  2. Bertsekas, D.P., Nedi, A., Ozdaglar, A.E.: Convex Analysis and Optimization. Athena Scientific, Belmont, MA, USA (2003)

    Google Scholar 

  3. Besag, J.: Statistical analysis of non-lattice data. The Statistician 24(3), 179–195 (1975)

    Article  Google Scholar 

  4. Besag, J.: Efficiency of pseudolikelihood estimation for simple Gaussian fields. Biometrika 64(3), 616–618 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bouchaud, J., Georges, A.: Anomalous diffusion in disordered media: statistical mechanisms, models and physical applications. Phys. Rep. 195, 127–293 (1990)

    Article  ADS  MathSciNet  Google Scholar 

  6. Box, G.E.P., Jenkins, G.M., Reinsel, G.C., Ljung, G.M.: Time Series Analysis: Forecasting and Control, 4th edn. John Wiley & Sons, Hoboken, NJ, USA (2008)

    Book  MATH  Google Scholar 

  7. Caticha, A., Preuss, R.: Maximum entropy and Bayesian data analysis: entropic prior distributions. Phys. Rev. E 70(4), 046127 (2004)

    Article  ADS  Google Scholar 

  8. Chandler, D.: Introduction to Modern Statistical Mechanics. Oxford University Press, Oxford, UK (1987)

    Google Scholar 

  9. Chilès, J.P., Delfiner, P.: Geostatistics: Modeling Spatial Uncertainty, 2nd edn. John Wiley & Sons, New York, NY, USA (2012)

    Book  MATH  Google Scholar 

  10. Christakos, G.: Random Field Models in Earth Sciences. Academic Press, San Diego (1992)

    MATH  Google Scholar 

  11. Christakos, G., Hristopulos, D.T.: Spatiotemporal Environmental Health Modelling. Kluwer, Boston (1998)

    MATH  Google Scholar 

  12. Cover, T.M., Thomas, J.A.: Elements of Information Theory, 2nd edn. John Wiley & Sons, Hoboken, NJ, USA (2006)

    MATH  Google Scholar 

  13. Elogne, S.N., Hristopulos, D.T.: Geostatistical applications of Spartan spatial random fields. In: Soares, A., Pereira, M.J., Dimitrakopoulos, R. (eds.) geoENV VI-Geostatistics for Environmental Applications. Quantitative Geology and Geostatistics, vol. 15, pp. 477–488. Springer, Berlin, Germany (2008)

    Chapter  Google Scholar 

  14. Elogne, S.N., Hristopulos, D.T., Varouchakis, E.: An application of Spartan spatial random fields in environmental mapping: focus on automatic mapping capabilities. Stochastic Environ. Res. Risk Assess. 22(5), 633–646 (2008)

    Article  MathSciNet  Google Scholar 

  15. Erickson, G., Smith, C.R. (eds.): Maximum-Entropy and Bayesian Methods in Science and Engineering: Foundations, vol. 1. Kluwer, Dordrecht, Netherlands (1988)

    Google Scholar 

  16. Feynman, R.P.: Statistical Mechanics. Benjamin and Cummings, Reading, MA, USA (1982)

    Google Scholar 

  17. Gull, S.F., Skilling, J.: Maximum entropy method in image processing. IEE Proc. F (Commun. Radar Signal Process.) 131(6), 646–659 (1984)

    Article  MATH  Google Scholar 

  18. Hall, P., Fisher, N., Hoffman, B.: On the nonparametric estimation of covariance functions. Ann. Stat. 22(4), 2115–2134 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  19. Hristopulos, D.T.: Permissibility of fractal exponents and models of band-limited two-point functions for fGn and fBm random fields. Stoch. Environ. Res. Risk Assess. 17(3), 191–216 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  20. Hristopulos, D.T.: Spatial random field models inspired from statistical physics with applications in the geosciences. Physica A: Stat. Mech. Appl. 365(1), 211–216 (2006)

    Article  ADS  Google Scholar 

  21. Hristopulos, D.T.: Stochastic local interaction (SLI) model: bridging machine learning and geostatistics. Comput. Geosci. 85(Part B), 26–37 (2015)

    Article  ADS  Google Scholar 

  22. Hristopulos, D.T., Christakos, G.: Practical calculation of non-Gaussian multivariate moments in spatiotemporal Bayesian maximum entropy analysis. Math. Geol. 33(5), 543–568 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  23. Hristopulos, D.T., Elogne, S.N.: Computationally efficient spatial interpolators based on Spartan spatial random fields. IEEE Trans. Signal Process. 57(9), 3475–3487 (2009)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  24. Hristopulos, D.T., Žukovič, M.: Relationships between correlation lengths and integral scales for covariance models with more than two parameters. Stoch. Environ. Res. Risk Assess. 25(1), 11–19 (2011)

    Article  MATH  Google Scholar 

  25. Hyman, J.M., Steinberg, S.: The convergence of mimetic discretization for rough grids. Comput. Math. Appl. 47(10–11), 1565–1610 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  26. Hyvärinen, A.: Consistency of pseudolikelihood estimation of fully visible Boltzmann machines. Neural Comput. 18(10), 2283–2292 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  27. Jaynes, E.T.: Information theory and statistical mechanics. Phys. Rev. 106(4), 620–630 (1957)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  28. Jaynes, E.T.: Information theory and statistical mechanics. Phys. Rev. 108(2), 171–190 (1957)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  29. Jizba, P., Korbel, J.: Maximum Entropy Principle in statistical inference: case for non-Shannonian entropies. Phys. Rev. Lett. 122(12), 120601 (2019)

    Article  ADS  Google Scholar 

  30. Kullback, S.: Information Theory and Statistics. Dover Publications, Mineola, NY, USA (1997)

    MATH  Google Scholar 

  31. Lantuéjoul, C.: Ergodicity and integral range. J. Microsc. 161(3), 387–403 (1991)

    Article  Google Scholar 

  32. Liang, X.S.: Entropy evolution and uncertainty estimation with dynamical systems. Entropy 16(7), 3605–3634 (2014)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  33. Lutz, E., Ciliberto, S.: Information: from Maxwell’s demon to Landauer’s eraser. Phys. Today 68(9), 30–35 (2015)

    Article  Google Scholar 

  34. MacKay, D.J.C.: Information Theory, Inference, and Learning Algorithms. Cambridge University Press, Cambridge, UK (2003)

    MATH  Google Scholar 

  35. Mehta, P., Bukov, M., Wang, C.H., Day, A.G.R., Richardson, C., Fisher, C.K., Schwab, D.J.: A high-bias, low-variance introduction to machine learning for physicists. Phys. Rep. 810, 1–124 (2019)

    Article  ADS  MathSciNet  Google Scholar 

  36. Nocedal, J., Wright, S.: Numerical Optimization, 2nd edn. Springer Science & Business Media, New York, NY, USA (2006)

    MATH  Google Scholar 

  37. Press, W.H., Teukolsky, S.A., Vetterling, W.T., Flannery, B.P.: Numerical Recipes, The Art of Scientific Computing, 3rd edn. Cambridge University Press, Cambridge, UK (1997)

    MATH  Google Scholar 

  38. Pressé, S., Ghosh, K., Lee, J., Dill, K.A.: Principles of maximum entropy and maximum caliber in statistical physics. Rev. Mod. Phys. 85(3), 1115–1141 (2013)

    Article  ADS  Google Scholar 

  39. Reusken, A.: Approximation of the determinant of large sparse symmetric positive definite matrices. SIAM J. Matrix Anal. Appl. 23(3), 799–818 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  40. Ripley, B.D.: Spatial Statistics, vol. 575. John Wiley & Sons, Hoboken, NJ, USA (2005)

    MATH  Google Scholar 

  41. Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27(3), 379–423 (1948)

    Article  MathSciNet  MATH  Google Scholar 

  42. Shen, Y., Ng, A.Y., Seeger, M.: Fast Gaussian process regression using KD-trees. In: Advances in Neural Information Processing Systems 18 [Neural Information Processing Systems, NIPS 2005, December 5–8, 2005, Vancouver, British Columbia, Canada], pp. 1225–1232. MIT Press, Cambridge, MA, USA (2005)

    Google Scholar 

  43. Sivia, D., Skilling, J.: Data Analysis: A Bayesian Tutorial. Oxford University Press, Oxford, UK (2006)

    MATH  Google Scholar 

  44. Skilling, J. (ed.): Maximum Entropy and Bayesian Methods, vol. 36. Springer Science & Business Media, Dordrecht, Netherlands (2013)

    Google Scholar 

  45. Skilling, J., Bryan, R.K.: Maximum entropy image reconstruction: general algorithm. Mon. Not. R. Astron. Soc. 211(1), 111–124 (1984)

    Article  ADS  MATH  Google Scholar 

  46. Sobczyk, K., Trcebicki, J.: Approximate probability distributions for stochastic systems: maximum entropy method. Comput. Methods Appl. Mech. Eng. 168(1), 91–111 (1999)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  47. Sobczyk, K., Trebicki, J.: Analysis of stochastic systems via maximum entropy principle. In: Bellomo, N., Casciati, F. (eds.) Nonlinear Stochastic Mechanics, Proceedings of the IUTAM 1991 Symposium, Turin (Italy), pp. 485–497. Springer, Berlin, Germany (1992)

    Chapter  Google Scholar 

  48. Stein, M.L.: Interpolation of Spatial Data: Some Theory for Kriging. Springer, New York, NY, USA (1999)

    Book  MATH  Google Scholar 

  49. Stinis, P.: A maximum likelihood algorithm for the estimation and renormalization of exponential densities. J. Comput. Phys. 208(2), 691–703 (2005)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  50. Transtrum, M.K., Machta, B.B., Sethna, J.P.: Why are nonlinear fits to data so challenging? Phys. Rev. Lett. 104(6), 060201 (2010)

    Article  ADS  Google Scholar 

  51. Tsallis, C.: Introduction to Nonextensive Statistical Mechanics. Springer, New York, NY, USA (2009)

    MATH  Google Scholar 

  52. Walder, C., Kim, K.I., Schölkopf, B.: Sparse multiscale Gaussian process regression. In: Cohen, W.W., McCallum, A., Roweis, S.T. (eds.) Machine Learning, Proceedings of the Twenty-Fifth International Conference (ICML 2008), Helsinki, Finland 5–9 June 2008. ACM International Conference Proceeding Series, vol. 307, pp. 1112–1119. ACM, New York, NY, USA (2008)

    Google Scholar 

  53. Wellmann, J.F.: Information theory for correlation analysis and estimation of uncertainty reduction in maps and models. Entropy 15(4), 1464–1485 (2013)

    Article  ADS  MathSciNet  MATH  Google Scholar 

  54. Žukovič, M., Hristopulos, D.T.: The method of normalized correlations: a fast parameter estimation method for random processes and isotropic random fields that focuses on short-range dependence. Technometrics 51(2), 173–185 (2009)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Technical University of Crete, Chania, Greece

    Dionissios T. Hristopulos

Authors
  1. Dionissios T. Hristopulos
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature B.V.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Hristopulos, D.T. (2020). More on Estimation. In: Random Fields for Spatial Data Modeling. Advances in Geographic Information Science. Springer, Dordrecht. https://doi.org/10.1007/978-94-024-1918-4_13

Download citation

Publish with us

Policies and ethics

Search

Navigation