Issues, Design and Interpretation of Performance Evaluations: Ensuring the Emperor Has Clothes

  • Robin L. Dennis
Part of the NATO · Challenges of Modern Society book series (NATS, volume 10)


Air quality model performance evaluation has been moving toward a more refined and statistically based status, beginning with the 1980 AMS/EPA Woods Hole Workshop of Dispersion Model Performance (Fox, 1981). The goal is to obtain a reasonably objective and well-defined evaluation. The general approach has been to address this goal solely through the use of statistics. As a result, a wealth of statistics is often presented for any one evaluation (Smith, 1984; Wackter and Londergan, 1984; Rao and Visalli, 1981). However, upon closer examination of the statistical results, it appears that a paucity of understanding has been achieved. This is seen as a severe problem for air quality model evaluations because adequate understanding of a model’s performance is crucial.


Model Performance Monitoring Site Cinder Cone Longe Travel Distance Vertical Dispersion 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Delhorame, J.P., 1978, Kriging in the Hydrosciences, Advances in Water Resources, 1(5): 251.CrossRefGoogle Scholar
  2. Dennis, R.L., Downton, M.W., and Keil, R.S., 1983, Evaluation of Performance Measures for an Urban Photochemical Model, EPA-450/4–83–021, Office of Air Quality Planning and Standards, EPA, Research Triangle Park, NC, 27711.Google Scholar
  3. Dennis, R.L. and Downton, M.W., 1984, Evaluation of Urban Photochemical Models for Regulatory Use, Atmospheric Environment, 18: 2055.CrossRefGoogle Scholar
  4. Dennis, R.L. and Irwin, J.S., 1985, Current Views of Model Performance Evaluation, in “Proceedings of the DOE/AMS Model Evaluation Workshop (Oct. 23–26, 1984 at Kiawah, S.C.), Vol. I: Participants and Invited Speakers Papers,” A.H. Weber and A.J. Garret, eds, E.I. DuPont de Nemours and Co., Savannah River Laboratory, Aiken, S.C., 29808.Google Scholar
  5. Downton M.W. and Dennis, R.L., 1985, Evaluation of Urban Air Quality Models for Regulatory Use: Refinement of an Approach, Journal of Climate and Applied Meteorology, 24(1).Google Scholar
  6. Fox, D.G., 1981, Judging Air Quality Model Performance, Bulletin American Meteorological Society, 62: 599.CrossRefGoogle Scholar
  7. Irwin, J.S., 1983, Estimating Plume dispersion—A Comparison of Several Sigma Schemes, Journal of Climate and Applied Meteorology, 22: 92.CrossRefGoogle Scholar
  8. Rao, S.T. and Visalli, J.R., 1981, One the Comparative Assessment of the Performance of Air Quality Models, J. of the Air Pollution Control Association, 31: 851.CrossRefGoogle Scholar
  9. Ruff, R.E., Nitz, K.C., Ludwig, F.L., Bhumralker, C.M., Shannon, J.S., Sheih, C.M., Lee, I.Y., Humar R., McNaughton, D.J., Assessment and Evaluation of Three Regional Air Quality Models, paper 84–46.1, 77th air Pollution Control Association Meeting, San Francisco, Calif. June 24–29, 1984, pp. 13.Google Scholar
  10. Smith, M.E., 1984, Review of the Attributes and Performance of 10 Rural Diffusion Models, Bulletin American Meteorological Society, 65: 554.CrossRefGoogle Scholar
  11. Wackter D.J. and Londergan, R.J., 1984, Evaluation of Complex Terrain Air Quality Simulation Models, EPA-450/4–84–017, Office of Air Quality Planning and Standards, EPA, Research Triangle Park, NC 27711.Google Scholar
  12. Willmott, C.J., 1982, Some Comments on the Evaluation of Model Performance, Bulletin American Meteorological Society, 63: 1309.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 1986

Authors and Affiliations

  • Robin L. Dennis
    • 1
  1. 1.Meteorology and Assessment Division, Atmospheric Sciences Research LaboratoryU.S. Environmental Protection AgencyResearch Triangle ParkUSA

Personalised recommendations