Data Assimilation for Atmospheric, Oceanic and Hydrologic Applications (Vol. II) pp 465-488 | Cite as

# All-Sky Satellite Radiance Data Assimilation: Methodology and Challenges

## Abstract

Assimilation of satellite radiances is the backbone of today’s operational data assimilation. Satellites can cover all parts of globe and provide information in areas not accessible by any other observation type. Of special interest are high-impact weather areas, such as tropical cyclones and severe weather outbreaks, which are mostly covered by clouds. Unfortunately, in current operational practice only clear-sky satellite radiances are assimilated, with only few exceptions. This effectively filters out a potentially useful information from all-sky radiances related to clouds and microphysics, and consequently limits the utility of satellite data. In this paper we will address numerous challenges related to the use of all-sky satellite radiances.All-sky satellite radiances present a formidable challenge for data assimilation as they relate to numerous technical aspects of data assimilation such as: (1) forecast error covariance, (2) correlated observation errors, (3) nonlinearity and non-differentiability, and (4) non-Gaussian errors. Assimilation of all-sky radiances is also challenging from a dynamical/physical point of view, since observing clouds implies a need for better understanding and ultimately simulation of cloud microphysical processes. Given that a reliable prediction of clouds requires a high-resolution cloud-resolving model, assimilation of all-sky radiancesis also a high-dimensional problem that requires addressing computational challenges.

## Keywords

Data Assimilation Observation Error Satellite Radiance Cloud Microphysical Process Forecast Error Covariance## Notes

### Acknowledgements

This work was supported by the National Science Foundation Collaboration in Mathematical Geosciences Grant ATM-0930265, and by NOAA NESDIS Grant NA10NES4400012.We would also like to acknowledge high-performance computing support provided by NCAR’s Computational and Information Systems Laboratory, sponsored by the National Science Foundation.

## References

- Abramov RF, Majda AJ (2004) Quantifying uncertainty for non-Gaussian ensembles in complex systems. SIAM J Sci Comput 26:411–447CrossRefGoogle Scholar
- Anderson JL (2003) A local least squares framework for ensemble filtering. Mon Wea Rev 131:634–642CrossRefGoogle Scholar
- Auligne T, Lorenc A, Michel Y, Montmerle T, Jones A, Hu M, Dudhia J (2011) Toward a new cloud analysis and prediction system. Bull Am Meteorol Soc 92:207–210CrossRefGoogle Scholar
- Axelsson O (1994) Iterative solution methods. Cambridge University Press, Cambridge, p 668CrossRefGoogle Scholar
- Axelsson O, Barker VA (1984) Finite-element solution of boundary-layer problems. Theory and computations. Academic Press, Orlando, p 432Google Scholar
- Bauer P, Geer A, Lopez P, Salmond D (2010) Direct 4D-Var assimilation of all- sky radiances. Part I: implementation. Q J Roy Meteorol Soc 136A:1868–1885CrossRefGoogle Scholar
- Bauer P, Lopez P, Salmond D, Benedetti A, Saarinen S, Moreau E (2006) Implementation of 1D + 4D-Var assimilation of precipitation-affected microwave radiances at ECMWF. II: 4D-Var. Q J Roy Meteorol Soc 132:2307–2332CrossRefGoogle Scholar
- Bauer P, Ohring G, Kummerow C, Auligne T (2011) Assimilating satellite observations of clouds and precipitation into NWP models. Bull Am Meteorol Soc 92:ES25–ES28CrossRefGoogle Scholar
- Bocquet M, Pires CA, Wu L (2010) Beyond Gaussian statistical modeling in geophysical data assimilation. Mon Wea Rev 138:2997–3023CrossRefGoogle Scholar
- Cover TM, Thomas JA (2006) Elements of information theory, 2nd edn. Wiley, Hoboken, 776 ppGoogle Scholar
- Daley R (1993) Atmospheric data analysis. Cambridge University Press, Cambridge, 472 ppGoogle Scholar
- Dee DP, Uppala S (2009) Variational bias correction of satellite radiance data in the ERA-Interim reanalysis. Q J Roy Meteorol Soc 135:1830–1841CrossRefGoogle Scholar
- Errico RM, Ohring G, Bauer P, Ferrier B, Mahfouf J-F, Turk J, Weng F (2007a) Assimilation of satellite cloud and precipitation observations in numerical weather prediction models: introduction to JAS special collection. J Atmos Sci 64:3737–3741CrossRefGoogle Scholar
- Errrico RM, Bauer P, Mahfouf JF (2007b) Issues regarding the assimilation of cloud and precipitation data. J Atmos Sci 64:3785–3798CrossRefGoogle Scholar
- Evensen G (2003) The ensemble Kalman filter: theoretical formulation and practical implementation. Ocean Dyn 53:343–367CrossRefGoogle Scholar
- Evensen G (2009) Data assimilation: the ensemble Kalman filter. Springer, Berlin/Heideberg, p 307Google Scholar
- Evensen G, van Leeuwen PJ (2000) An ensemble Kalman smoother for nonlinear dynamics. Mon Wea Rev 128:1852–1867CrossRefGoogle Scholar
- Fertig EJ, Hunt BR, Ott E, Szunyogh I (2007) Assimilating non-local observations with a local ensemble Kalman filter. Tellus 59A:719–730Google Scholar
- Fletcher SJ, Zupanski M (2006a) A hybrid normal and lognormal distribution for data assimilation. Atmos Sci Lett 7:43–46CrossRefGoogle Scholar
- Fletcher SJ, Zupanski M (2006b) A data assimilation method for log-normally distributed observational errors. Q J Roy Meteorol Soc 132:2505–2519CrossRefGoogle Scholar
- Fletcher SJ, Zupanski M (2008) Implications and impacts of transforming lognormal variables into normal variables in VAR. Met Zeit 16:755–765CrossRefGoogle Scholar
- Gaspari G, Cohn SE (1999) Construction of correlation functions in two and three dimensions. Q J Roy Meteorol Soc 125:723–757CrossRefGoogle Scholar
- Geer A, Bauer P, Geer A, Lopez P (2010) Direct 4D-Var assimilation of all- sky radiances. Part II: assessment. Q J Roy Meteorol Soc 136A:1886–1905CrossRefGoogle Scholar
- Geer A, Bauer P (2010) Enhanced use of all-sky microwave observations sensitive to water vapour, cloud and precipitation. ECMWF Tech Memorandum 620:41Google Scholar
- Golub GH, and van Loan CF (1989) Matrix computations. The John Hopkins University Press, Baltimore, p 642Google Scholar
- Gordon NJ, Salmond DJ, Smith AFM (1993) Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc 140:107–113Google Scholar
- Gu Y, Oliver DS (2007) An iterative ensemble Kalman filter for multiphase fluid flow data assimilation. SPE J 12:438–446Google Scholar
- Haarala M, Miettinen K, Makela MM (2004) New limited memory bundle method for large-scale nonsmooth optimization. Optim Meth Softw 19:673–692CrossRefGoogle Scholar
- Hamill T, Whitaker J, Snyder C (2001) Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon Wea Rev 129:2776–2790CrossRefGoogle Scholar
- Harris BA, Kelly G (2001) A satellite radiance-bias correction scheme for data assimilation. Q J Roy Meteorol Soc 127:1453–1468CrossRefGoogle Scholar
- Houtekamer P, Mitchell H (2001) A sequential ensemble Kalman filter for atmospheric data assimilation. Mon Wea Rev 129:123–136CrossRefGoogle Scholar
- Huang X-Y, et al (2009) Four-dimensional variational data assimilation for WRF: formulation and preliminary results. Mon Wea Rev 137:299–314CrossRefGoogle Scholar
- Jazwinski AH (1970) Stochastic processes and filtering theory. Academic, San Diego, 376 ppGoogle Scholar
- Kalnay E (2003) Atmospheric modeling, data assimilation and predictability. Cambridge University Press, Cambridge, 341 pGoogle Scholar
- Karmitsa N, Bagirov A, Makela MM (2012) Comparing different nonsmooth minimization methods and software. Optim Meth Softw 27:131–153CrossRefGoogle Scholar
- Kullback S, Leibler RA (1951) On information and sufficiency. Annals Math Stat 22:79–86CrossRefGoogle Scholar
- Lewis JM, Lakshmivarahan S, Dhall SK (2006) Dynamic data assimilation: a least squares approach. Cambridge University Press, Cambridge, 680 pGoogle Scholar
- Li Z, Navon IM (2001) Optimality of 4D-Var and its relationship with the Kalman filter and Kalman smoother. Q J Roy Meteorol Soc 127:661–684CrossRefGoogle Scholar
- Lorenc AC (1986) Analysis methods for numerical weather prediction. Q J Roy Meteorol Soc 112:1177–1194CrossRefGoogle Scholar
- Luenberger DL (1989) Linear and non-linear programming. Addison-Wesley, Reading, 491 ppGoogle Scholar
- Navon IM (1986) A review of variational and optimization methods in meteorology. In: Sasaki YK (ed) Variational methods in geosciences. Developments in Geomathematics, vol 5. Elsevier Science Publishers, Amsterdam, pp 29–35Google Scholar
- Nocedal J (1980) Updating quasi-Newton matrices with limited storage. Math Comp 35:773–782CrossRefGoogle Scholar
- Okamoto K, Derber JC (2006) Assimilatioon of SSM/I radiances in the NCEP global data assimilation system. Mon Wea Rev 134:2612–2631CrossRefGoogle Scholar
- Parrish DF, Cohn SE (1985) A Kalman filter for a two-dimensional shallow- water model: formulation and preconditioning experiments. Office Note 304, National Meteorological Center, Washington, DCGoogle Scholar
- Parrish DP, Derber JC (1992) The national meteorological center’s spectral statistical interpolation analysis system. Mon Wea Rev 120:1747–1763CrossRefGoogle Scholar
- Polkinghorne R, Vukicevic T (2011) Data assimilation of cloud-affected radiances in a cloud-resolving model. Mon Wea Rev 139:755–773CrossRefGoogle Scholar
- Shannon CE, Weaver W (1949) The mathematical theory of communication. University of Illinois Press, Urbana, 144 ppGoogle Scholar
- Simon E, Bertino L (2009) Application of the Gaussian anamorphosis to assimilation in a 3-D coupled physical-ecosystem model of the North Atlantic with the EnKF: a twin experiment. Ocean Sci 5:495–510CrossRefGoogle Scholar
- Skamarock WC, Klemp JB, Dudhia J, Gill DO, Barker DM, Wang W, Powers JG (2005) A description of the advanced research WRF version 2. NCAR Techinal Note 468, 88 pp. http://www.mmm.ucar.edu/wrf/users/docs/arw_v2.pdf
- Stephens GL (1994) Remote sensing of the lower atmosphere. Oxford University Press, New York, p 544Google Scholar
- Steward JL (2012) On a unifying interpretation of empirically-determined square-root background error covariance matrices for variational and ensemble data assimilation. JPL Earth Science Seminar, PasadenaGoogle Scholar
- Steward JL, Navon IM, Zupanski M, Karmitsa N (2012) Impact of non-smooth observation operators on variational and sequential data assimilation for a limited-area shallow-water equation model. Q J Roy Meteorol Soc 138:323–339CrossRefGoogle Scholar
- Thepaut J-N, Courtier P, Belaud G, Lemaitre G (1996) Dynamical structure functions in a four-dimensional variational assimilation: a case study. Q J Roy Meteorol Soc 122:535–561CrossRefGoogle Scholar
- van Leeuwen PJ (2009) Particle filtering in geophysical systems. Mon Wea Rev 137:4089–4114CrossRefGoogle Scholar
- Vukicevic T, Greenwald T, Zupanski M, Zupanski D, VonderHaar T, Jones AS (2004) Mesoscale cloud state estimation from visible and infrared satellite radiances. Mon Wea Rev 132:3066–3077CrossRefGoogle Scholar
- Wang X, Hamill TM, Whitaker JS, Bishop CH (2007) A comparison of hybrid ensemble transform Kalman filter-OI and ensemble square-root filter analysis schemes. Mon Wea Rev 135
**:**1055–1076Google Scholar - Wu W-S, Purser RJ, Parrish DF (2002) Three-dimensional variational analysis with spatially inhomogeneous covariances. Mon Wea Rev 130:2905–2916CrossRefGoogle Scholar
- Xiong X, Navon IM, Uzunoglu B (2006) A note on the particle filter with posterior Gaussian resampling. Tellus 58A:456–460Google Scholar
- Yang W, Navon IM, Courtier P (1996) A new Hessian preconditioning method applied to variational data assimilation experiments using adiabatic version of NASA/GEOS-1 GCM. Mon Wea Rev 124:1000–1017CrossRefGoogle Scholar
- Zhang S, Zupanski M, Hou A, Lin X, Cheung S (2012) Assimilation of precipitation- affected radiances in a cloud-resolving WRF ensemble data assimilation system. Mon Weather Rev. doi:10.1175/MWR-D-12-00055.1 (in press)Google Scholar
- Zupanski M (1993) A preconditioning algorithm for large scale minimization problems. Tellus 45A:578–592Google Scholar
- Zupanski M (1995) A preconditionin algorithm for four-dimensional variational data assimilation. Mon Wea Rev 124:2562–2573CrossRefGoogle Scholar
- Zupanski M (2005) Maximum likelihood ensemble filter: theoretical aspects. Mon Wea Rev 133:1710–1726CrossRefGoogle Scholar
- Zupanski D, Mesinger F (1995) Four-dimensional variational assimilation of precipitation data. Mon Wea Rev 123:1112–1127CrossRefGoogle Scholar
- Zupanski M, Navon IM, Zupanski D (2008) The maximum likelihood ensemble filter as a non-differentiable minimization algorithm. Quart J Roy Meteorol Soc 134:1039–1050CrossRefGoogle Scholar
- Zupanski D, Zupanski M, Grasso LD, Brummer R, Jankov I, Lindsey D, Sengupta M, DeMaria M (2011a) Assimilating synthetic GOES-R radiances in cloudy conditions using an ensemble-based method. Int J Remote Sens 32:9637–9659CrossRefGoogle Scholar
- Zupanski D, Zhang SQ, Zupanski M, Hou AY, Cheung SH (2011b) A prototype WRF-based ensemble data assimilation system for downscaling satellite precipitation observations. J Hydromet 12:118–134CrossRefGoogle Scholar