Mathematical Geology

, Volume 29, Issue 1, pp 131–152 | Cite as

On the equivalence of kriging and maximum entropy estimators

  • Yuh-Ming Lee
  • J. Hugh Ellis


This study compares kriging and maximum entropy estimators for spatial estimation and monitoring network design. For second-order stationary random fields (a subset of Gaussian fields) the estimators and their associated interpolation error variances are identical. Simple lognormal kriging differs from the lognormal maximum entropy estimator, however, in both mathematical formulation and estimation error variances. Two numerical examples are described that compare the two estimators. Simple lognormal kriging yields systematically higher estimates and smoother interpolation surfaces compared to those produced by the lognormal maximum entropy estimator. The second empirical comparison applies kriging and entropy-based models to the problem of optimizing groundwater monitoring network design, using six alternative objective functions. The maximum entropy-based sampling design approach is shown to be the more computationally efficient of the two.

Key Words

spatial estimation entropy kriging monitoring network design lognormal random fields 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Amorocho, J., and Espildora, B., 1973, Entropy in the assessment of uncertainty in hydrologic systems and models: Water Resources Res., v. 9, no. 6, p. 1511–1522.Google Scholar
  2. Andricevic, R., and Foufoula-Georgiou, E., 1991, A transfer function approach to sampling network design for ground water contamination: Water Resources Res., v. 27, no. 10, p. 2759–2770.CrossRefGoogle Scholar
  3. Bras, R. L., and Rodriguez-Iturbe, I., 1985, Random functions and hydrology: Addison-Wesley Publ. Co., Reading, Massachusetts, 559 p.Google Scholar
  4. Caselton, W. F., and Husain, T., 1980, Hydrologic networks: information transmission: Jour. Water Resource Planning and Management Division, ASCE, v. 106, no. WR2, p. 503–520.Google Scholar
  5. Chapman, T. G., 1986, Entropy as a measure of hydrologie data uncertainty and model performance: Jour. Hydrology, v. 85, no. 1/2, p. 111–126.CrossRefGoogle Scholar
  6. Christakos, G., 1990, A Bayesian/maximum-entropy view to the spatial estimation problem: Math. Geology, v. 22, no. 7, p. 763–777.CrossRefGoogle Scholar
  7. Christakos, G., 1991, Some applications of the Bayesian/maximum-entropy concepts in geostatistics,in Grandy, W. T., and Schick, L. H., eds., Maximum entropy and bayesian methods: Kluwer Acad. Publ., Dordrecht, The Netherlands, p. 215–229.Google Scholar
  8. Christakos, G., 1992, Random field models in earth sciences: Academic Press, San Diego, California, 474 p.Google Scholar
  9. Christakos, G., and Olea, R. A., 1992, Sampling design for spatially distributed hydrogeologic and environmental processes: Advances in Water Resources, v. 15, no. 14, p. 219–238.CrossRefGoogle Scholar
  10. Christensen, R., 1991, Linear models for multivariate time series, and spatial data: Springer-Verlag, New York, 317 p.Google Scholar
  11. Cressie, N., 1990, The origins of kriging: Math. Geology, v. 22, no. 3, p. 239–252.CrossRefGoogle Scholar
  12. Cressie, N., 1993, Statistics for spatial data (revised ed.): John Wiley & Sons, New York, 900 p.Google Scholar
  13. David, M., 1988, Handbook of applied advanced geostatistical ore reserve estimation: Elsevier Science Publ. Co., New York, 216 p.Google Scholar
  14. de Marsily, G., 1985, Quantitative hydrogeology: Groundwater hydrology for engineers: Academic Press, Orlando, Florida, 440 p.Google Scholar
  15. Dowd, P. A., 1982, Lognormal kriging—the general case: Math. Geology, v. 14, no. 15, p. 475–499.Google Scholar
  16. Dowd, P. A., 1992, A review of recent developments in geostatistics: Computers & Geosciences, v. 17, no. 10, p. 1481–1500.CrossRefGoogle Scholar
  17. Fedorov, V. V., 1972, Theory of optimal experiments: Academic Press, New York, 292 p. (Studden, W. J., and Klimko, E. M., translators).Google Scholar
  18. Fedorov, V. V., 1988, Experimental design technique in the optimization of a monitoring network,in Fedorov, V. V., and Lanter, H., eds., Model-oriented data analysis: Springer-Verlag, Berlin, 238 p.Google Scholar
  19. Fedorov, V. V., and Mueller, W., 1989, Comparison of two approaches in the optimal design of an observation network: Statistics, v. 20, no. 3, p. 339–351.CrossRefGoogle Scholar
  20. Harmancioglu, N., and Yevjevich, V., 1987, Transfer of hydrologic information among river points: Jour. Hydrology, v. 91, no. 1-2, p. 103–118.CrossRefGoogle Scholar
  21. Husain, T., 1989, Hydrologic uncertainty measure and network design: Water Resources Bull., v. 25, no. 3, p. 527–534.Google Scholar
  22. Jaynes, E. T., 1957, Information theory and statistical mechanics: Physical Review, v. 106, no. 4, p. 620–630.CrossRefGoogle Scholar
  23. Jaynes, E. T., 1982, On the rationale of maximum-entropy methods: Proc. IEEE, v. 70, no. 9, p. 939–952.Google Scholar
  24. Johnson, N. L., and Kotz, S., 1972, Distributions in statistics: continuous multivariate distributions: John Wiley & Sons, New York, 333 p.Google Scholar
  25. Jones, D. S., 1979, Elementary information theory: Oxford Univ. Press, New York, 182 p.Google Scholar
  26. Joumel, A. G., 1980, The lognormal approach to predicting local distribution of selective mining unit grades: Math. Geology, v. 12, no. 4, p. 285–303.CrossRefGoogle Scholar
  27. Joumel, A. G., 1989, Fundamentals of geostatistics in five lessons (short course presented at the 28th Intern. Geol. Congress): Am. Geophy. Union, Washington, D.C., 40 p.Google Scholar
  28. Joumel, A. G., and Deutsch, C. V., 1993, Entropy and spatial disorder: Math. Geology, v. 25, no. 3, p. 329–355.CrossRefGoogle Scholar
  29. Joumel, A. G., and Huijbregts, Ch. J., 1978, Mining geostatistics: Academic Press, London, 600 P.Google Scholar
  30. Kapur, J. N., 1989, Maximum-entropy models in science and engineering: John Wiley & Sons, New York, 635 p.Google Scholar
  31. Kapur, J. N., and Kesavan, H. K., 1992, Entropy optimization principles with applications: Academic Press, Boston, Massachusetts, 408 p.Google Scholar
  32. Kvålseth, T., 1982, Some information properties of the lognormal distribution: IEEE Trans. on Information Theory, v. IT-28, no. 6, p. 963–966.CrossRefGoogle Scholar
  33. Lindley, D. V., 1956, On the measure of the information provided by an experiment: Ann. Math. Statistics, v. 27, no. 4, p. 986–1005.Google Scholar
  34. Matheron, G., 1963, Principles of geostatistics: Econ. Geology, v. 58, no. 8, p. 1246–1266.CrossRefGoogle Scholar
  35. Matheron, G., 1973, The intrinsic random functions and their applications: Advances in Applied Probability, v. 5, p. 438–468.CrossRefGoogle Scholar
  36. Rendu, J.-M. M., 1979, Normal and lognormal estimation: Math. Geology, v. 11, no. 4, p. 407–422.CrossRefGoogle Scholar
  37. Rivoirard, J., 1994, Introduction to disjunctive kriging and non-linear geostatistics: Clarendon Press, Oxford, 180 p.Google Scholar
  38. Rouhani, S., 1985, Variance reduction analysis: Water Resources Res., v. 21, no. 6, p. 837–846.Google Scholar
  39. Rouhani, S., and Fiering, M. B., 1986, Resilience of a statistical sampling scheme: Jour. Hydrology, v. 89, no. 1-2, p. 1–11.CrossRefGoogle Scholar
  40. Rouhani, S., and Hall, T. J., 1988, Geostatistical scheme for groundwater sampling: Jour. Hydrology, v. 103, no. 1-2, p. 85–102.CrossRefGoogle Scholar
  41. Sacks, J., and Schiller, S., 1988, Spatial designs,in Gupta, S. S., and Berger, J. O., eds., Statistical design theory and related topics IV-2: Springer-Verlag, New York, p. 385–399.Google Scholar
  42. Shannon, C. E., 1948, A mathematical theory of communication: The Bell System Tech. Jour., v. 27, p. 379–423, 623–656. Reprintedin Claude Elwood Shannon Collected Papers (Sloane, N. J. and Wyner, A. D., eds., 1993): IEEE Press, p. 5–83.Google Scholar
  43. Shewry, M. C., and Wynn, H. P., 1987, Maximum entropy sampling: Jour. Applied Statistics, v. 14, no. 2, p. 165–170.CrossRefGoogle Scholar
  44. Singh, V. P., 1989, Hydrologic modeling using entropy: Jour. Engineers, Civil Engineering Division, v. 70, no. CV2, p. 55–60.Google Scholar
  45. Singh, V. P., Rajagopal, A. K., and Singh, K., 1986, Derivation of some frequency distributions using the principle of maximum entropy: Advances in Water Research, v. 9, no. 2, p. 91–106.CrossRefGoogle Scholar
  46. Singh, V. P., and Singh, K., 1985, Derivation of the gamma distribution by using the principle of maximum entropy: Water Resources Bull., v. 21, no. 6, p. 941–952.Google Scholar
  47. Theil, H., and Fiebig, D. G., 1984, Exploiting continuity, maximum entropy estimation of continuous distributions: Ballinger Publ. Co., Cambridge, Massachusetts, 246 p.Google Scholar
  48. Tribus, M., 1978, Thirty years of information theory,in Levine, R. D., and Tribus, M., eds., The maximum entropy formalism: The MIT Press, Cambridge, Massachusetts, 498 p.Google Scholar
  49. Trujillo-Ventura, A., and Ellis, J. H., 1991, Multiobjective air pollution monitoring network design: Atmospheric Environment, v. 25A, no. 2, p. 469–479.Google Scholar
  50. Vanmarcke, E., 1983, Random fields: analysis and synthesis: The MIT Press, Cambridge, Massachusetts, 382 p.Google Scholar
  51. Wu, S., and Zidek, J. V., 1992, An entropy-based analysis of data from selected NAPD/NTN network sites for 1983–1986: Atmospheric Environment, v. 26A, no. 11, p. 2089–2103.Google Scholar

Copyright information

© International Association for Mathematical Geology 1997

Authors and Affiliations

  • Yuh-Ming Lee
    • 1
  • J. Hugh Ellis
    • 2
  1. 1.Graduate Institute of Natural Resources ManagementNational Chung Hsing UniversityTaipeiTaiwan, Republic of China
  2. 2.Department of Geography and Environmental EngineeringThe Johns Hopkins UniversityBaltimore

Personalised recommendations