A Comprehensive Performance Evaluation of the Next Generation of the Canadian Operational Regional Air Quality Deterministic Prediction System

Abstract

The core of the Environment and Climate Change Canada (ECCC) operational Regional Air Quality Deterministic Prediction System (RAQDPS) is the GEM-MACH air quality model, which consists of an on-line chemical transport model embedded within the GEM model, ECCC’s multi-scale operational weather forecast model. A new version of GEM-MACH, version 2, which is based on the next-generation version of GEM, became operational earlier this year (2016) after 4 years of development and testing. A comprehensive evaluation of the performance of GEM-MACH version 2 for a 2010 annual simulation on a 10-km North American continental grid was performed as part of this implementation effort using measurements from multiple Canadian and U.S. air-chemistry and precipitation-chemistry surface networks. One evaluation metric considered was skill in predicting annual mean values of a number of gas- and particle-phase species, including PM_2.5 chemical components such as elemental carbon and crustal material. Such an analysis of time-averaged spatial fields is useful to check for systematic errors in input emissions fields, in chemical lateral boundary conditions, and in the representation of atmospheric dispersion, chemistry, and removal processes by the model. Spatial R values for NO₂, O₃, and PM_2.5 mean annual concentrations in air for all networks were 0.84, 0.76, and 0.58, and for PM_2.5 chemical components SO₄, NO₃, NH₄, EC, OM, and CM the corresponding R values were 0.95, 0.88, 0.78, 0.77, 0.54, and 0.41. For SO ⁼₄ , NO₃ ⁻, and NH₄ ⁺ mean annual concentrations in precipitation the R values were 0.79, 0.80, and 0.92.

Questions and Answers

Questioner: Sebnem Aksoyoglu

Question: The performance for annual nitrate looks quite good. Is it good also in winter?

Answer: The winter performance for PM_2.5-NO₃ was actually slightly better than the annual performance (n = 208, R = 0.90, NMB = 5%). By contrast, autumn performance was the least good (n = 212, R = 0.87, NMB = 80%).

Questioner: Jeff Weil

Question: Many of the plots showing predicted versus observed concentrations showed very large scatter (factor of 10–100) and thus the concentration variance is a very big problem. Do you, your group, or others in the modeling community attempt to model or predict this concentration variance? Such modeling would seem to be a problem worthy of attention.

Answer: Large scatter between model predictions and measurements is always of concern, but for regional chemical transport models (CTMs) there are many sources of such statistical dispersion, including measurement errors and uncertainties in numerical representations of emissions, meteorology, transport and diffusion, chemical transformation, and removal processes. An additional source of scatter arises from the inherent incommensurability between point measurements and model grid-volume predictions. I believe that the focus of the CTM community has been to reduce such scatter rather than trying to model or predict it.