All-Sky Satellite Radiance Data Assimilation: Methodology and Challenges

  • Milija ZupanskiEmail author


Assimilation of satellite radiances is the backbone of today’s operational data assimilation. Satellites can cover all parts of globe and provide information in areas not accessible by any other observation type. Of special interest are high-impact weather areas, such as tropical cyclones and severe weather outbreaks, which are mostly covered by clouds. Unfortunately, in current operational practice only clear-sky satellite radiances are assimilated, with only few exceptions. This effectively filters out a potentially useful information from all-sky radiances related to clouds and microphysics, and consequently limits the utility of satellite data. In this paper we will address numerous challenges related to the use of all-sky satellite radiances.All-sky satellite radiances present a formidable challenge for data assimilation as they relate to numerous technical aspects of data assimilation such as: (1) forecast error covariance, (2) correlated observation errors, (3) nonlinearity and non-differentiability, and (4) non-Gaussian errors. Assimilation of all-sky radiances is also challenging from a dynamical/physical point of view, since observing clouds implies a need for better understanding and ultimately simulation of cloud microphysical processes. Given that a reliable prediction of clouds requires a high-resolution cloud-resolving model, assimilation of all-sky radiancesis also a high-dimensional problem that requires addressing computational challenges.


Data Assimilation Observation Error Satellite Radiance Cloud Microphysical Process Forecast Error Covariance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported by the National Science Foundation Collaboration in Mathematical Geosciences Grant ATM-0930265, and by NOAA NESDIS Grant NA10NES4400012.We would also like to acknowledge high-performance computing support provided by NCAR’s Computational and Information Systems Laboratory, sponsored by the National Science Foundation.


  1. Abramov RF, Majda AJ (2004) Quantifying uncertainty for non-Gaussian ensembles in complex systems. SIAM J Sci Comput 26:411–447CrossRefGoogle Scholar
  2. Anderson JL (2003) A local least squares framework for ensemble filtering. Mon Wea Rev 131:634–642CrossRefGoogle Scholar
  3. Auligne T, Lorenc A, Michel Y, Montmerle T, Jones A, Hu M, Dudhia J (2011) Toward a new cloud analysis and prediction system. Bull Am Meteorol Soc 92:207–210CrossRefGoogle Scholar
  4. Axelsson O (1994) Iterative solution methods. Cambridge University Press, Cambridge, p 668CrossRefGoogle Scholar
  5. Axelsson O, Barker VA (1984) Finite-element solution of boundary-layer problems. Theory and computations. Academic Press, Orlando, p 432Google Scholar
  6. Bauer P, Geer A, Lopez P, Salmond D (2010) Direct 4D-Var assimilation of all- sky radiances. Part I: implementation. Q J Roy Meteorol Soc 136A:1868–1885CrossRefGoogle Scholar
  7. Bauer P, Lopez P, Salmond D, Benedetti A, Saarinen S, Moreau E (2006) Implementation of 1D + 4D-Var assimilation of precipitation-affected microwave radiances at ECMWF. II: 4D-Var. Q J Roy Meteorol Soc 132:2307–2332CrossRefGoogle Scholar
  8. Bauer P, Ohring G, Kummerow C, Auligne T (2011) Assimilating satellite observations of clouds and precipitation into NWP models. Bull Am Meteorol Soc 92:ES25–ES28CrossRefGoogle Scholar
  9. Bocquet M, Pires CA, Wu L (2010) Beyond Gaussian statistical modeling in geophysical data assimilation. Mon Wea Rev 138:2997–3023CrossRefGoogle Scholar
  10. Cover TM, Thomas JA (2006) Elements of information theory, 2nd edn. Wiley, Hoboken, 776 ppGoogle Scholar
  11. Daley R (1993) Atmospheric data analysis. Cambridge University Press, Cambridge, 472 ppGoogle Scholar
  12. Dee DP, Uppala S (2009) Variational bias correction of satellite radiance data in the ERA-Interim reanalysis. Q J Roy Meteorol Soc 135:1830–1841CrossRefGoogle Scholar
  13. Errico RM, Ohring G, Bauer P, Ferrier B, Mahfouf J-F, Turk J, Weng F (2007a) Assimilation of satellite cloud and precipitation observations in numerical weather prediction models: introduction to JAS special collection. J Atmos Sci 64:3737–3741CrossRefGoogle Scholar
  14. Errrico RM, Bauer P, Mahfouf JF (2007b) Issues regarding the assimilation of cloud and precipitation data. J Atmos Sci 64:3785–3798CrossRefGoogle Scholar
  15. Evensen G (2003) The ensemble Kalman filter: theoretical formulation and practical implementation. Ocean Dyn 53:343–367CrossRefGoogle Scholar
  16. Evensen G (2009) Data assimilation: the ensemble Kalman filter. Springer, Berlin/Heideberg, p 307Google Scholar
  17. Evensen G, van Leeuwen PJ (2000) An ensemble Kalman smoother for nonlinear dynamics. Mon Wea Rev 128:1852–1867CrossRefGoogle Scholar
  18. Fertig EJ, Hunt BR, Ott E, Szunyogh I (2007) Assimilating non-local observations with a local ensemble Kalman filter. Tellus 59A:719–730Google Scholar
  19. Fletcher SJ, Zupanski M (2006a) A hybrid normal and lognormal distribution for data assimilation. Atmos Sci Lett 7:43–46CrossRefGoogle Scholar
  20. Fletcher SJ, Zupanski M (2006b) A data assimilation method for log-normally distributed observational errors. Q J Roy Meteorol Soc 132:2505–2519CrossRefGoogle Scholar
  21. Fletcher SJ, Zupanski M (2008) Implications and impacts of transforming lognormal variables into normal variables in VAR. Met Zeit 16:755–765CrossRefGoogle Scholar
  22. Gaspari G, Cohn SE (1999) Construction of correlation functions in two and three dimensions. Q J Roy Meteorol Soc 125:723–757CrossRefGoogle Scholar
  23. Geer A, Bauer P, Geer A, Lopez P (2010) Direct 4D-Var assimilation of all- sky radiances. Part II: assessment. Q J Roy Meteorol Soc 136A:1886–1905CrossRefGoogle Scholar
  24. Geer A, Bauer P (2010) Enhanced use of all-sky microwave observations sensitive to water vapour, cloud and precipitation. ECMWF Tech Memorandum 620:41Google Scholar
  25. Golub GH, and van Loan CF (1989) Matrix computations. The John Hopkins University Press, Baltimore, p 642Google Scholar
  26. Gordon NJ, Salmond DJ, Smith AFM (1993) Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proc 140:107–113Google Scholar
  27. Gu Y, Oliver DS (2007) An iterative ensemble Kalman filter for multiphase fluid flow data assimilation. SPE J 12:438–446Google Scholar
  28. Haarala M, Miettinen K, Makela MM (2004) New limited memory bundle method for large-scale nonsmooth optimization. Optim Meth Softw 19:673–692CrossRefGoogle Scholar
  29. Hamill T, Whitaker J, Snyder C (2001) Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter. Mon Wea Rev 129:2776–2790CrossRefGoogle Scholar
  30. Harris BA, Kelly G (2001) A satellite radiance-bias correction scheme for data assimilation. Q J Roy Meteorol Soc 127:1453–1468CrossRefGoogle Scholar
  31. Houtekamer P, Mitchell H (2001) A sequential ensemble Kalman filter for atmospheric data assimilation. Mon Wea Rev 129:123–136CrossRefGoogle Scholar
  32. Huang X-Y, et al (2009) Four-dimensional variational data assimilation for WRF: formulation and preliminary results. Mon Wea Rev 137:299–314CrossRefGoogle Scholar
  33. Jazwinski AH (1970) Stochastic processes and filtering theory. Academic, San Diego, 376 ppGoogle Scholar
  34. Kalnay E (2003) Atmospheric modeling, data assimilation and predictability. Cambridge University Press, Cambridge, 341 pGoogle Scholar
  35. Karmitsa N, Bagirov A, Makela MM (2012) Comparing different nonsmooth minimization methods and software. Optim Meth Softw 27:131–153CrossRefGoogle Scholar
  36. Kullback S, Leibler RA (1951) On information and sufficiency. Annals Math Stat 22:79–86CrossRefGoogle Scholar
  37. Lewis JM, Lakshmivarahan S, Dhall SK (2006) Dynamic data assimilation: a least squares approach. Cambridge University Press, Cambridge, 680 pGoogle Scholar
  38. Li Z, Navon IM (2001) Optimality of 4D-Var and its relationship with the Kalman filter and Kalman smoother. Q J Roy Meteorol Soc 127:661–684CrossRefGoogle Scholar
  39. Lorenc AC (1986) Analysis methods for numerical weather prediction. Q J Roy Meteorol Soc 112:1177–1194CrossRefGoogle Scholar
  40. Luenberger DL (1989) Linear and non-linear programming. Addison-Wesley, Reading, 491 ppGoogle Scholar
  41. Navon IM (1986) A review of variational and optimization methods in meteorology. In: Sasaki YK (ed) Variational methods in geosciences. Developments in Geomathematics, vol 5. Elsevier Science Publishers, Amsterdam, pp 29–35Google Scholar
  42. Nocedal J (1980) Updating quasi-Newton matrices with limited storage. Math Comp 35:773–782CrossRefGoogle Scholar
  43. Okamoto K, Derber JC (2006) Assimilatioon of SSM/I radiances in the NCEP global data assimilation system. Mon Wea Rev 134:2612–2631CrossRefGoogle Scholar
  44. Parrish DF, Cohn SE (1985) A Kalman filter for a two-dimensional shallow- water model: formulation and preconditioning experiments. Office Note 304, National Meteorological Center, Washington, DCGoogle Scholar
  45. Parrish DP, Derber JC (1992) The national meteorological center’s spectral statistical interpolation analysis system. Mon Wea Rev 120:1747–1763CrossRefGoogle Scholar
  46. Polkinghorne R, Vukicevic T (2011) Data assimilation of cloud-affected radiances in a cloud-resolving model. Mon Wea Rev 139:755–773CrossRefGoogle Scholar
  47. Shannon CE, Weaver W (1949) The mathematical theory of communication. University of Illinois Press, Urbana, 144 ppGoogle Scholar
  48. Simon E, Bertino L (2009) Application of the Gaussian anamorphosis to assimilation in a 3-D coupled physical-ecosystem model of the North Atlantic with the EnKF: a twin experiment. Ocean Sci 5:495–510CrossRefGoogle Scholar
  49. Skamarock WC, Klemp JB, Dudhia J, Gill DO, Barker DM, Wang W, Powers JG (2005) A description of the advanced research WRF version 2. NCAR Techinal Note 468, 88 pp.
  50. Stephens GL (1994) Remote sensing of the lower atmosphere. Oxford University Press, New York, p 544Google Scholar
  51. Steward JL (2012) On a unifying interpretation of empirically-determined square-root background error covariance matrices for variational and ensemble data assimilation. JPL Earth Science Seminar, PasadenaGoogle Scholar
  52. Steward JL, Navon IM, Zupanski M, Karmitsa N (2012) Impact of non-smooth observation operators on variational and sequential data assimilation for a limited-area shallow-water equation model. Q J Roy Meteorol Soc 138:323–339CrossRefGoogle Scholar
  53. Thepaut J-N, Courtier P, Belaud G, Lemaitre G (1996) Dynamical structure functions in a four-dimensional variational assimilation: a case study. Q J Roy Meteorol Soc 122:535–561CrossRefGoogle Scholar
  54. van Leeuwen PJ (2009) Particle filtering in geophysical systems. Mon Wea Rev 137:4089–4114CrossRefGoogle Scholar
  55. Vukicevic T, Greenwald T, Zupanski M, Zupanski D, VonderHaar T, Jones AS (2004) Mesoscale cloud state estimation from visible and infrared satellite radiances. Mon Wea Rev 132:3066–3077CrossRefGoogle Scholar
  56. Wang X, Hamill TM, Whitaker JS, Bishop CH (2007) A comparison of hybrid ensemble transform Kalman filter-OI and ensemble square-root filter analysis schemes. Mon Wea Rev 135:1055–1076Google Scholar
  57. Wu W-S, Purser RJ, Parrish DF (2002) Three-dimensional variational analysis with spatially inhomogeneous covariances. Mon Wea Rev 130:2905–2916CrossRefGoogle Scholar
  58. Xiong X, Navon IM, Uzunoglu B (2006) A note on the particle filter with posterior Gaussian resampling. Tellus 58A:456–460Google Scholar
  59. Yang W, Navon IM, Courtier P (1996) A new Hessian preconditioning method applied to variational data assimilation experiments using adiabatic version of NASA/GEOS-1 GCM. Mon Wea Rev 124:1000–1017CrossRefGoogle Scholar
  60. Zhang S, Zupanski M, Hou A, Lin X, Cheung S (2012) Assimilation of precipitation- affected radiances in a cloud-resolving WRF ensemble data assimilation system. Mon Weather Rev. doi:10.1175/MWR-D-12-00055.1 (in press)Google Scholar
  61. Zupanski M (1993) A preconditioning algorithm for large scale minimization problems. Tellus 45A:578–592Google Scholar
  62. Zupanski M (1995) A preconditionin algorithm for four-dimensional variational data assimilation. Mon Wea Rev 124:2562–2573CrossRefGoogle Scholar
  63. Zupanski M (2005) Maximum likelihood ensemble filter: theoretical aspects. Mon Wea Rev 133:1710–1726CrossRefGoogle Scholar
  64. Zupanski D, Mesinger F (1995) Four-dimensional variational assimilation of precipitation data. Mon Wea Rev 123:1112–1127CrossRefGoogle Scholar
  65. Zupanski M, Navon IM, Zupanski D (2008) The maximum likelihood ensemble filter as a non-differentiable minimization algorithm. Quart J Roy Meteorol Soc 134:1039–1050CrossRefGoogle Scholar
  66. Zupanski D, Zupanski M, Grasso LD, Brummer R, Jankov I, Lindsey D, Sengupta M, DeMaria M (2011a) Assimilating synthetic GOES-R radiances in cloudy conditions using an ensemble-based method. Int J Remote Sens 32:9637–9659CrossRefGoogle Scholar
  67. Zupanski D, Zhang SQ, Zupanski M, Hou AY, Cheung SH (2011b) A prototype WRF-based ensemble data assimilation system for downscaling satellite precipitation observations. J Hydromet 12:118–134CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.Cooperative Institute for Research in the AtmosphereColorado State UniversityFort CollinsUSA

Personalised recommendations