Operational Dust Prediction

Abstract

Over the last few years, numerical prediction of dust aerosol concentration has become prominent at several research and operational weather centres due to growing interest from diverse stakeholders, such as solar energy plant managers, health professionals, aviation and military authorities and policymakers. Dust prediction in numerical weather prediction-type models faces a number of challenges owing to the complexity of the system. At the centre of the problem is the vast range of scales required to fully account for all of the physical processes related to dust. Another limiting factor is the paucity of suitable dust observations available for model, evaluation and assimilation. This chapter discusses in detail numerical prediction of dust with examples from systems that are currently providing dust forecasts in near real-time or are part of international efforts to establish daily provision of dust forecasts based on multi-model ensembles. The various models are introduced and described along with an overview on the importance of dust prediction activities and a historical perspective. Assimilation and evaluation aspects in dust prediction are also discussed.

Appendix A: Technical Aspects of Data Assimilation for Dust Prediction

10.1.1 A10.1 Assimilation Techniques

10.1.1.1 Variational Methods (CMA, ECMWF, FNMOC/NRL, Met Office, NASA GMAO)

The variational method is a well-established approach that combines model background information with observations to obtain the “best” initial conditions possible. In the 2D- and 3D-Var versions, the fields are adjusted at the analysis time whereas in 4D-Var, a short-term forecast is run over the selected time window (usually 12 h) to provide a so-called first guess. In 4D-Var, the dynamical model is then used as a strong constraint to minimise the difference between the model background and the observations. This approach is widely used in many NWP centres. The fundamental idea of the variational methods is based on minimisation of a cost function which measures the distance between observations and their model equivalent, subject to a background constraint usually provided by the model itself. Optimisation of this cost function is performed with respect to selected control variables (e.g. the initial conditions). Adjustments to these control variables allow for the updated model trajectory to match the observations more closely. Assuming the update to the initial condition is small, an incremental formulation can be adopted to ensure a good compromise between operational feasibility and physical consistency in the analysis (Courtier et al. 1994). This so-called “incremental” approach is employed at ECMWF. Another key aspect of the variational methods is the use of the adjoint model to calculate the gradient of the cost function needed in the minimisation. Coding an adjoint of highly nonlinear parameterisations can be involved, and the parameterisations may need to be linearised before an adjoint can be constructed.

10.1.1.2 Kalman Filter and Ensemble Kalman Filter Methods (MRI/JMA, NRL)

Another data assimilation method, the Kalman filter (KF), has been well known since the 1960s (Kalman 1960). KF, which is based on the linear minimum variance estimation approach, evolves the error covariance matrix temporally. The KF calculation requires neither tangent linear models nor adjoint models. Despite these advantages, KF requires the inverse calculation of the matrices with the dimensions of the model state space. The size of the model state space in geosciences is often of the order of millions: for such large systems, KF cannot be adopted. In order to exploit the advantages of KF and reduce the computational burden, the ensemble Kalman filter (EnKF) was developed (Evensen 1994, 2007). The basic concept of EnKF is that the ensemble of the forward model forecasts is able to represent the probability distribution function (PDF) of the system state and approximate the error covariance distribution. The EnKF is mathematically equivalent to the original Kalman filter, under the ideal conditions where the simulation model is linear, and the EnKF employs an infinite ensemble size. In the MRI/JMA aerosol assimilation system, a 4D expansion of the EnKF (4D-EnKF) is adopted to assimilate asynchronous observations at the appropriate times. Using the 4D-EnKF aerosol assimilation system, the surface emission intensity distribution of dust aerosol is estimated (Sekiyama et al. 2010, 2011). The vector augmentation mentioned above enables EnKF to estimate the parameters through the background error covariance between dust emissions and observations. Consequently, EnKF simultaneously estimated the aerosol concentrations (as model variables) together with the dust aerosol emission intensity (as model parameters). The MRI/JMA aerosol assimilation system employs the local ensemble transform Kalman filter (LETKF), which is one of the EnKF implementation schemes (Hunt et al. 2007). The LETKF uses the ensemble transform approach (Bishop et al. 2001) to obtain the analysis ensemble as a linear combination of the background ensemble forecasts. The LETKF handles observations locally in space, where all the observations are assimilated simultaneously.

It is important to note that 4D-Var and ensemble Kalman filter methods approximately converge, when 4D-Var is run over a long assimilation window (e.g. 24 h) and model error is included, as they are both based on the Bayes theorem which postulates that the probability distribution of the analysis errors is a linear combination of the probability distribution of the observations and background errors (Fisher et al. 2005).

10.1.2 A10.2 Observations Used for the Dust Analyses

10.1.2.1 Main Products

The MODIS AOD product is used most widely due to its reliability and availability in near real-time (Kaufman et al. 1997; Remer et al. 2005). Two separate retrievals with different accuracies are applied over land and ocean. The former suffers from higher uncertainties due to the impact of the surface reflectance. Several other factors affect the accuracy of the retrievals both over land and ocean: cloud contamination, assumptions about the aerosol types and size distribution, near-surface wind speed, radiative transfer model biases and instrumental uncertainties (Zhang and Reid 2006). The MODIS product provides the total AOD, such that the partitioning between dust and other aerosol species is driven by the particular analysis system and its underlying model. However, the standard MODIS Dark Target method does not deliver data over bright surfaces where there is not enough contrast between the surface and overlying aerosol layer. However, iron in desert soils absorbs at blue wavelengths, and albedo in the blue part of the solar spectrum is considerably darker than the mid-visible and red. This allowed the development of the MODIS Deep Blue product (Hsu et al. 2004, 2006). Deep Blue is not currently assimilated at NRL, NASA or ECMWF, but it is expected to be incorporated into their systems now that an error matrix has been established (Shi et al. 2012).

At the Met Office, the standard MODIS and MODIS Deep Blue (Hsu et al. 2004, 2006; Ginoux et al. 2010) products are being assimilated, and the AOD products at 550 nm wavelength from SEVIRI from Brindley and Ignatov (2006) and Brindley and Russell (2009) are being monitored prior to being assimilated in the near future. However, only a subset of observations can be used, as the forecast model contains only dust rather than a full suite of aerosols. This restriction is achieved by geographic filtering of the SEVIRI AOD and by using the MODIS standard product aerosol-type flags over land and preferentially using the MODIS Deep Blue product over bright desert surfaces. The presence of other aerosols in these regions of high dust loading introduces uncertainty into the assimilation process.

CALIPSO (e.g. Winker et al. 2007) is the first satellite mission to have made aerosol lidar observations routinely available from space. At MRI/JMA, the CALIPSO Level 1B data have been successfully assimilated into the JMA dust forecast model with a positive impact on the prediction of aeolian dust. A derived CALIPSO product is also assimilated at NRL (Campbell et al. 2010; Zhang et al. 2011). In particular, Zhang et al. (2011) found that assimilation of lidar data had a beneficial impact on the 48 h forecast. The same product is under study for assimilation at ECMWF.

10.1.2.2 Data Quality Aspects and Bias Correction

Perhaps the most pressing issue for satellite data assimilation is the development of appropriate satellite error models. Indeed, a key assumption in data assimilation is that the observation errors are uncorrelated spatially. For satellite aerosol products, and dust products in particular, there is considerable spatially correlated bias. Such bias is formed from a number of factors, including biases in the algorithm’s lower boundary condition/surface reflectance, microphysical bias in the assumed optical model of the aerosol particles and the cloud mask. These biases can lead to unphysical analysis fields, which in turn can lead to positive or negative perturbation “plumes” in forecast fields. Currently, satellite data providers do not generate prognostic error models, and it has fallen on the data assimilation community to modify the products for their own purposes. Debiasing data products and developing reliable point-by-point uncertainties are time-consuming. Further, aerosol product algorithms update frequently, leaving previous error analyses obsolete.

Each centre’s development team has approached satellite data quality and bias correction differently. Development for FNMOC systems at NRL and the University of North Dakota has favoured extensive error analysis at the expense of sophistication in the data assimilation technology. MODIS over ocean, land and Deep Blue products have had extensive debiasing based on comparison with AERONET observations and error modelling applied (Zhang and Reid 2006; Shi et al. 2011a, 2012; Hyer et al. 2011). In addition, the spatial covariance of the MODIS and MISR products has also been undertaken (Shi et al. 2011b). Internal studies at NRL have shown that, overall, the assimilation of raw satellite aerosol products boosts model verification scores. After a set of quality assurance steps were taken with the satellite products, NAAPS root-mean-square error (RMSE) improved by more than 40 %. Lidar assimilation has taken a similar method, with considerable quality assurance (QA) checks (Campbell et al. 2010).

At ECMWF, a variational bias correction is implemented based on the operational set-up for assimilated radiances following the developments by Dee and Uppala (2009). The bias model for the MODIS data consists of a global constant that is adjusted variationally in the minimisation based on the first-guess departures. Although simple, this bias correction works well in the sense that the MACC analysis is not biased with respect to MODIS observations. Moreover, this approach has the advantage of being tied to the optimisation of the cost function, and as such it is estimated online, not requiring previous preprocessing of the observations. The bias error model allows more complex treatment with the addition of other bias predictors that are relevant for AOD, for example, instrument geometry, viewing angle, cloud cover, wind speed, etc. Improvements to the bias model are currently being undertaken.

10.1.3 A10.3 Definitions of Background and Observational Errors

Since the relative weight between the background and the observations is decided by the error statistics prescribed for both, in areas that are data-limited such as the deserts, the aerosol analysis is severely under-constrained relative to the observations and relies almost entirely on the background. Also, the background matrix is responsible for the redistribution of the aerosol information from the observations to the model fields. This is again especially true for dust due to the already-mentioned paucity of observations over bright surfaces.

10.1.3.1 Background Error Covariance Matrices

The aerosol background error covariance matrix used for aerosol analyses at ECMWF was derived using the Parrish and Derber method (also known as NMC method; Parrish and Derber 1992) as detailed by Benedetti and Fisher (2007). This method was long used for the definition of the background error statistics for the meteorological variables and is based on the assumption that the forecast differences between the 48 h and the 24 h forecasts are a good statistical proxy to estimate the model background errors. The advantage in using the model to define the errors is the grid-point availability of the statistics over a long period. This leads to a satisfactory background error covariance matrix without the need to prescribe the vertical and horizontal correlation length as shown in Kahnert (2008). However, a shortcoming of this method consists in the static definition of the background error covariance matrix, which can lead to suboptimal analysis in the case of unusual situations such as intense storms. This is addressed by the ensemble methods with flow-dependent error estimates which suit the specific situation (“errors of the day”).

For the FNMOC/NRL NAAPS global model, background error covariances were estimated in a number of methods, all converging to the same number for the error covariance length (250 km, the same as is commonly assumed for water vapour). This length was determined from experiments from the MODIS data set. As a check, error covariances were also estimated from a 3-month simulation from the 20-member NAAPS ensemble driven purely from the NOGAPS meteorological ensemble.

10.1.3.2 Flow-Dependent Background Error Covariance Matrix

“Errors of the day” can be estimated in the context of the ensemble methods, where at each analysis time, a series of forecasts is run starting from perturbed conditions, and these forecasts provide an estimate of the model errors. However, the EnKF tends to be easily influenced by sampling errors at long distances because the available ensemble size is too small to estimate the background error covariance of the atmospheric system. Therefore, a covariance localisation must be applied for all the EnKF implementation schemes to reduce the spurious impact of distant observations. The LETKF permits a flexible choice of observations to be assimilated at each grid point. For example, the MRI/JMA system employs the covariance localisation with a Gaussian weighting function that depends on the physical distance between the grid location and the observation. The limited ensemble size causes both sampling errors at long distances and filter divergence. To compensate for the error underestimation and avoid the filter divergence, it is necessary to increase the ensemble spread every data assimilation cycle. This technique is called covariance inflation. The MRI/JMA system utilises a multiplicative inflation method, in which the ensemble spread is uniformly multiplied by a constant value larger than one; it is common to tune this inflation factor empirically. Furthermore, adding random perturbations to the initial state of each ensemble member is sometimes necessary to maintain the diversity of the ensemble members and not to lose the error covariance among the model variables. In the MRI/JMA system, random perturbations are added to dust emission intensity. This type of flow-dependent background error definition is very promising, and it has also been progressively adopted for standard meteorological applications in variational systems through the so-called hybrid approach (Buehner et al. 2010a, b; Clayton et al. 2012), in which the assimilation framework is variational, but the background errors of the day are defined through ensemble methods. This approach should work well for dust initialisation, where the errors on the dust prediction are both associated to emission uncertainties and transport.

10.1.3.3 Observation Errors

The problem of defining appropriate errors for the observations when those are retrieval products is very complex. Observation errors for these products are comprised of measurements errors that depend on instrument calibration and characteristics and a priori and representativeness errors that depend on the retrieval assumptions regarding the parameters that are not directly observed but that affect the retrieval output, such as the optical properties assumed for the aerosols, as well as on the overall quality of the forward model used in the retrieval. Most satellite data providers do not provide errors at the pixel level, but rather provide regression parameters derived from comparison of the satellite products with ground-based equivalent products like AERONET retrievals of AOD which are deemed to have high accuracy. This type of regression-based error estimate does not faithfully represent the accuracy of the retrieved product at the level of individual pixels, which is what is needed in data assimilation. Often the developers end up assigning their own errors to the observations to be able to fit the needs of their system. For example, at ECMWF, the observation error covariance matrix is assumed to be diagonal, to simplify the problem. The errors are also chosen ad hoc and prescribed as fixed values over land and ocean for the assimilated observations (MODIS AOD at 550 nm). This was decided after investigation revealed that biases were introduced in the analysis due to the observation error assumptions when those were specified as relative rather than absolute errors. While this might be a specific characteristic of the ECMWF system, the problem of a correct specification of the pixel-level errors on aerosol-retrieved products is a topic of ongoing research (Kolmonen et al. 2013).