Langevin and Kalman Importance Sampling for Nonlinear Continuous-Discrete State-Space Models

  • Hermann SingerEmail author


The likelihood function of a nonlinear continuous-discrete state-space model with state dependent diffusion function is computed by integrating out the latent variables with the help of Langevin sampling. The continuous-time paths are discretized on a time grid in order to obtain a finite-dimensional integration and densities w.r.t. Lebesgue measure. We use importance sampling, where the exact importance density is the conditional density of the latent states, given the measurements. This unknown density is either estimated from the sampler data or approximated by an estimated normal density. Then, new trajectories are drawn from this Gaussian measure. Alternatively, a Gaussian importance density is directly derived from an extended Kalman smoother with subsequent sampling of independent trajectories (extended Kalman sampling (EKS)). We compare the Monte Carlo results with numerical methods based on extended, unscented, and Gauss-Hermite Kalman filtering (EKF, UKF, GHF) and a grid-based solution of the Fokker-Planck equation between measurements. This comprises the repeated multiplication of transition matrices based on Euler transition kernels, finite differences, and discretized integral operators. The methods are illustrated for the geometrical Brownian motion and the Ginzburg-Landau model for phase transitions.


  1. Aït-Sahalia, Y. (2002). Maximum likelihood estimation of discretely sampled diffusions: A closed-form approximation approach. Econometrica, 70(1), 223–262. MathSciNetCrossRefGoogle Scholar
  2. Aït-Sahalia, Y. (2008). Closed-form likelihood expansions for multivariate diffusions. Annals of Statistics, 36(2), 906–937. MathSciNetCrossRefGoogle Scholar
  3. Apte, A., Hairer, M., Stuart, A. M., & Voss, J. (2007). Sampling the posterior: An approach to non-Gaussian data assimilation. Physica D: Nonlinear Phenomena, 230(1–2), 50–64. MathSciNetCrossRefGoogle Scholar
  4. Apte, A., Jones, C. K. R. T., Stuart, A. M., & Voss, J. (2008). Data assimilation: Mathematical and statistical perspectives. International Journal for Numerical Methods in Fluids, 56(8), 1033–1046. MathSciNetCrossRefGoogle Scholar
  5. Arasaratnam, I., Haykin, S., & Hurd, T. (2010). Cubature Kalman filtering for continuous-discrete systems: Theory and simulations. IEEE Transactions on Signal Processing, 58, 4977–4993. MathSciNetCrossRefGoogle Scholar
  6. Arnold, L. (1974). Stochastic differential equations. New York: Wiley.zbMATHGoogle Scholar
  7. Åström, K. J. (1970). Introduction to stochastic control theory. Mineola, NY: Courier Corporation.zbMATHGoogle Scholar
  8. Bagchi, A. (2001). Onsager-Machlup function. In Encyclopedia of mathematics. Berlin: Springer.
  9. Ballreich, D. (2017). Stable and efficient cubature-based filtering in dynamical systems. Berlin: Springer International Publishing.CrossRefGoogle Scholar
  10. Bartlett, M. S. (1946). On the theoretical specification and sampling properties of autocorrelated time-series. Journal of the Royal Statistical Society (Supplement), 7, 27–41. MathSciNetCrossRefGoogle Scholar
  11. Basawa, I. V., & Prakasa Rao, B. L. S. (1980). Statistical inference for stochastic processes. London: Academic Press.zbMATHGoogle Scholar
  12. Bergstrom, A. R. (1976a). Non-recursive models as discrete approximations to systems of stochastic differential equations. In A. R. Bergstrom (Ed.), Statistical inference in continuous time models (pp. 15–26). Amsterdam: North Holland.zbMATHGoogle Scholar
  13. Bergstrom, A. R. (Ed.). (1976b). Statistical inference in continuous time economic models. Amsterdam: North Holland.zbMATHGoogle Scholar
  14. Bergstrom, A. R. (1983). Gaussian estimation of structural parameters in higher order continuous time dynamic models. Econometrica: Journal of the Econometric Society, 51(1), 117–152. MathSciNetCrossRefGoogle Scholar
  15. Bergstrom, A. R. (1988). The history of continuous-time econometric models. Econometric Theory, 4, 365–383. MathSciNetCrossRefGoogle Scholar
  16. Beskos, A., Papaspiliopoulos, O., Roberts, G. O., & Fearnhead, P. (2006). Exact and efficient likelihood-based inference for discretely observed diffusion processes (with discussion). Journal of the Royal Statistical Society Series B, 68, 333–382. CrossRefGoogle Scholar
  17. Black, F., & Scholes, M. (1973). The pricing of options and corporate liabilities. Journal of Political Economy, 81, 637–654. MathSciNetCrossRefGoogle Scholar
  18. Chang, J., & Chen, S. X. (2011). On the approximate maximum likelihood estimation for diffusion processes. The Annals of Statistics, 39(6), 2820–2851. MathSciNetCrossRefGoogle Scholar
  19. Chow, S.-M., Ferrer, E., & Nesselroade, J. R. (2007). An unscented Kalman filter approach to the estimation of nonlinear dynamical systems models. Multivariate Behavioral Research, 42(2), 283–321. CrossRefGoogle Scholar
  20. Chow, S.-M., Lu, Z., Sherwood, A., & Zhu, H. (2016). Fitting nonlinear ordinary differential equation models with random effects and unknown initial conditions using the stochastic approximation expectation–maximization (SAEM) algorithm. Psychometrika, 81(1), 102–134. MathSciNetCrossRefGoogle Scholar
  21. Da Prato, G. (2004). Kolmogorov equations for stochastic PDEs. Basel: Birkhäuser. CrossRefGoogle Scholar
  22. Da Prato, G., & Zabczyk, J. (1992). Stochastic equations in infinite dimensions. New York: Cambridge University Press.
  23. Doreian, P., & Hummon, N. P. (1976). Modelling social processes. New York, Oxford, Amsterdam: Elsevier.Google Scholar
  24. Durham, G. B., & Gallant, A. R. (2002). Numerical techniques for simulated maximum likelihood estimation of stochastic differential equations. Journal of Business and Economic Statistics, 20, 297–316. MathSciNetCrossRefGoogle Scholar
  25. Elerian, O., Chib, S., & Shephard, N. (2001). Likelihood inference for discretely observed nonlinear diffusions. Econometrica, 69(4), 959–993. MathSciNetCrossRefGoogle Scholar
  26. Feller, W. (1951). Two singular diffusion problems. Annals of Mathematics, 54, 173–182. MathSciNetCrossRefGoogle Scholar
  27. Flury, T., & Shephard, N. (2011). Bayesian inference based only on simulated likelihood: particle filter analysis of dynamic economic models. Econometric Theory, 27(5), 933–956. MathSciNetCrossRefGoogle Scholar
  28. Fuchs, C. (2013). Inference for diffusion processes: With applications in life sciences. Berlin: Springer. CrossRefGoogle Scholar
  29. Gard, T. C. (1988). Introduction to stochastic differential equations. New York: Dekker.zbMATHGoogle Scholar
  30. Girolami, M., & Calderhead, B. (2011). Riemann manifold Langevin and Hamiltonian Monte Carlo methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73(2), 123–214. MathSciNetCrossRefGoogle Scholar
  31. Hairer, M., Stuart, A. M., & Voss, J. (2007). Analysis of SPDEs arising in path sampling, part II: The nonlinear case. Annals of Applied Probability, 17(5), 1657–1706. MathSciNetCrossRefGoogle Scholar
  32. Hairer, M., Stuart, A. M., & Voss, J. (2009). Sampling conditioned diffusions. In J. Blath, P. Morters, & M. Scheutzow (Eds.), Trends in stochastic analysis (pp. 159–186). Cambridge: Cambridge University Press. CrossRefGoogle Scholar
  33. Hairer, M., Stuart, A. M., & Voss, J. (2011). Signal processing problems on function space: Bayesian formulation, stochastic PDEs and effective MCMC methods. In D. Crisan & B. Rozovsky (Eds.), The Oxford handbook of nonlinear filtering (pp. 833–873). Oxford: Oxford University Press.Google Scholar
  34. Hairer, M., Stuart, A. M., Voss, J., & Wiberg, P. (2005). Analysis of SPDEs arising in path sampling, part I: The Gaussian case. Communications in Mathematical Sciences, 3(4), 587–603. MathSciNetCrossRefGoogle Scholar
  35. Haken, H. (1977). Synergetics. Berlin: Springer.zbMATHGoogle Scholar
  36. Haken, H., Kelso, J. A. S., & Bunz, H. (1985). A theoretical model of phase transitions in human hand movements. Biological Cybernetics, 51(5), 347–356. MathSciNetCrossRefGoogle Scholar
  37. Hamerle, A., Nagl, W., & Singer, H. (1991). Problems with the estimation of stochastic differential equations using structural equations models. Journal of Mathematical Sociology, 16(3), 201–220. CrossRefGoogle Scholar
  38. Harvey, A. C., & Stock, J. (1985). The estimation of higher order continuous time autoregressive models. Econometric Theory, 1, 97–112. CrossRefGoogle Scholar
  39. Hastings, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57(1), 97–109. MathSciNetCrossRefGoogle Scholar
  40. Herings, J. P. (1996). Static and dynamic aspects of general disequilibrium theory. Boston: Springer. CrossRefGoogle Scholar
  41. Jazwinski, A. H. (1970). Stochastic processes and filtering theory. New York: Academic Press.Google Scholar
  42. Jetschke, G. (1986). On the equivalence of different approaches to stochastic partial differential equations. Mathematische Nachrichten, 128, 315–329. MathSciNetCrossRefGoogle Scholar
  43. Jetschke, G. (1991). Lattice approximation of a nonlinear stochastic partial differential equation with white noise. In International series of numerical mathematics (Vol. 102, pp. 107–126). Basel: Birkhäuser. Google Scholar
  44. Jones, R. H. (1984). Fitting multivariate models to unequally spaced data. In E. Parzen (Ed.), Time series analysis of irregularly observed data (pp. 158–188). New York: Springer. CrossRefGoogle Scholar
  45. Jones, R. H., & Tryon, P. V. (1987). Continuous time series models for unequally spaced data applied to modeling atomic clocks. SIAM Journal of Scientific and Statistical Computing, 8, 71–81. MathSciNetCrossRefGoogle Scholar
  46. Kac, M. (1980). Integration in function spaces and some of its applications. Pisa: Scuola normale superiore.zbMATHGoogle Scholar
  47. Kloeden, P. E., & Platen, E. (1992). Numerical solution of stochastic differential equations. Berlin: Springer. CrossRefGoogle Scholar
  48. Kloeden, P. E., & Platen, E. (1999). Numerical solution of stochastic differential equations. Berlin: Springer. (corrected third printing)Google Scholar
  49. Langevin, P. (1908). Sur la théorie du mouvement brownien [On the theory of Brownian motion]. Comptes Rendus de l’Academie des Sciences (Paris), 146, 530–533.zbMATHGoogle Scholar
  50. Li, C. (2013). Maximum-likelihood estimation for diffusion processes via closed-form density expansions. The Annals of Statistics, 41(3), 1350–1380. MathSciNetCrossRefGoogle Scholar
  51. Lighthill, M. J. (1958). Introduction to Fourier analysis and generalised functions. Cambridge: Cambridge University Press.
  52. Liptser, R. S., & Shiryayev, A. N. (2001). Statistics of random processes (Vols. I and II, 2nd ed.). New York: SpringerGoogle Scholar
  53. Lorenz, E. (1963). Deterministic nonperiodic flow. Journal of the Atmospheric Sciences, 20, 130. CrossRefGoogle Scholar
  54. Louis, T. A. (1982). Finding the observed information matrix when using the EM algorithm. Journal of the Royal Statistical Association B, 44(2), 226–233.MathSciNetzbMATHGoogle Scholar
  55. Malik, S., & Pitt, M. K. (2011). Particle filters for continuous likelihood evaluation and maximisation. Journal of Econometrics, 165(2), 190–209. MathSciNetCrossRefGoogle Scholar
  56. Mardia, K. V., Kent, J. T., & Bibby, J. M. (1979). Multivariate analysis. London: Academic Press.zbMATHGoogle Scholar
  57. Molenaar, P., & Newell, K. M. (2003). Direct fit of a theoretical model of phase transition in oscillatory finger motions. British Journal of Mathematical and Statistical Psychology, 56(2), 199–214. MathSciNetCrossRefGoogle Scholar
  58. Moler, C., & Van Loan, C. (2003). Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later. SIAM Review, 45(1), 1–46. MathSciNetCrossRefGoogle Scholar
  59. Newell, K. M., & Molenaar, P. C. M. (2014). Applications of nonlinear dynamics to developmental process modeling. Hove: Psychology Press.CrossRefGoogle Scholar
  60. Okano, K., Schülke, L., & Zheng, B. (1993). Complex Langevin simulation. Progress of Theoretical Physics Supplement, 111, 313–346. CrossRefGoogle Scholar
  61. Onsager, L., & Machlup, S. (1953). Fluctuations and irreversible processes. Physical Review, 91(6), 1505–1515. MathSciNetCrossRefGoogle Scholar
  62. Oud, J. H. L., & Jansen, R. A. R. G. (2000). Continuous time state space modeling of panel data by means of SEM. Psychometrika, 65, 199–215. MathSciNetCrossRefGoogle Scholar
  63. Oud, J. H. L., & Singer, H. (2008). Continuous time modeling of panel data: SEM versus filter techniques. Statistica Neerlandica, 62(1), 4–28.MathSciNetCrossRefGoogle Scholar
  64. Ozaki, T. (1985). Nonlinear time series and dynamical systems. In E. Hannan (Ed.), Handbook of statistics (pp. 25–83). Amsterdam: North Holland.Google Scholar
  65. Parisi, G., & Wu, Y.-S. (1981). Perturbation theory without gauge fixing. Scientia Sinica, 24, 483.MathSciNetGoogle Scholar
  66. Pedersen, A. R. (1995). A new approach to maximum likelihood estimation for stochastic differential equations based on discrete observations. Scandinavian Journal of Statistics, 22, 55–71.MathSciNetzbMATHGoogle Scholar
  67. Pitt, M. K. (2002). Smooth particle filters for likelihood evaluation and maximisation (Warwick economic research papers No. 651). University of Warwick.
  68. Reznikoff, M. G., & Vanden-Eijnden, E. (2005). Invariant measures of stochastic partial differential equations and conditioned diffusions. Comptes Rendus Mathematiques, 340, 305–308. MathSciNetCrossRefGoogle Scholar
  69. Risken, H. (1989). The Fokker-Planck equation (2nd ed.). Berlin: Springer. CrossRefGoogle Scholar
  70. Roberts, G. O., & Stramer, O. (2001). On inference for partially observed nonlinear diffusion models using the Metropolis–Hastings algorithm. Biometrika, 88(3), 603–621. MathSciNetCrossRefGoogle Scholar
  71. Roberts, G. O., & Stramer, O. (2002). Langevin diffusions and Metropolis-Hastings algorithms. Methodology and Computing in Applied Probability, 4(4), 337–357. MathSciNetCrossRefGoogle Scholar
  72. Rümelin, W. (1982). Numerical treatment of stochastic differential equations. SIAM Journal of Numerical Analysis, 19(3), 604–613. MathSciNetCrossRefGoogle Scholar
  73. Särkkä, S. (2013). Bayesian filtering and smoothing (Vol. 3). Cambridge: Cambridge University Press. CrossRefGoogle Scholar
  74. Särkkä, S., Hartikainen, J., Mbalawata, I. S., & Haario, H. (2013). Posterior inference on parameters of stochastic differential equations via non-linear Gaussian filtering and adaptive MCMC. Statistics and Computing, 25(2), 427–437. MathSciNetCrossRefGoogle Scholar
  75. Schiesser, W. E. (1991). The numerical method of lines. San Diego: Academic Press.zbMATHGoogle Scholar
  76. Schuster, H. G., & Just, W. (2006). Deterministic chaos: An introduction. New York: Wiley.zbMATHGoogle Scholar
  77. Shoji, I., & Ozaki, T. (1997). Comparative study of estimation methods for continuous time stochastic processes. Journal of Time Series Analysis, 18(5), 485–506. MathSciNetCrossRefGoogle Scholar
  78. Shoji, I., & Ozaki, T. (1998a). Estimation for nonlinear stochastic differential equations by a local linearization method 1. Stochastic Analysis and Applications, 16(4), 733–752. MathSciNetCrossRefGoogle Scholar
  79. Shoji, I., & Ozaki, T. (1998b). A statistical method of estimation and simulation for systems of stochastic differential equations. Biometrika, 85(1), 240–243. MathSciNetCrossRefGoogle Scholar
  80. Silverman, B. W. (1986). Density estimation for statistics and data analysis. London: Chapman and Hall. CrossRefGoogle Scholar
  81. Singer, H. (1986). Depressivität und gelernte Hilflosigkeit als Stochastischer Prozeß [Depression and learned helplessness as stochastic process] (Unpublished master’s thesis). Universität Konstanz, Konstanz, Germany.
  82. Singer, H. (1990). Parameterschätzung in zeitkontinuierlichen dynamischen Systemen [Parameter estimation in continuous time dynamical systems]. Konstanz: Hartung-Gorre-Verlag.Google Scholar
  83. Singer, H. (1993). Continuous-time dynamical systems with sampled data, errors of measurement and unobserved components. Journal of Time Series Analysis, 14(5), 527–545. MathSciNetCrossRefGoogle Scholar
  84. Singer, H. (1995). Analytical score function for irregularly sampled continuous time stochastic processes with control variables and missing values. Econometric Theory, 11, 721–735. MathSciNetCrossRefGoogle Scholar
  85. Singer, H. (1998). Continuous panel models with time dependent parameters. Journal of Mathematical Sociology, 23, 77–98. CrossRefGoogle Scholar
  86. Singer, H. (2002). Parameter estimation of nonlinear stochastic differential equations: Simulated maximum likelihood vs. extended Kalman filter and Itô-Taylor expansion. Journal of Computational and Graphical Statistics, 11(4), 972–995. MathSciNetCrossRefGoogle Scholar
  87. Singer, H. (2003). Simulated maximum likelihood in nonlinear continuous-discrete state space models: Importance sampling by approximate smoothing. Computational Statistics, 18(1), 79–106. MathSciNetCrossRefGoogle Scholar
  88. Singer, H. (2005). Continuous-discrete unscented Kalman filtering (Diskussionsbeiträge Fachbereich Wirtschaftswissenschaft No. 384). FernUniversität in Hagen.
  89. Singer, H. (2008a). Generalized Gauss-Hermite filtering. Advances in Statistical Analysis, 92(2), 179–195. MathSciNetCrossRefGoogle Scholar
  90. Singer, H. (2008b). Nonlinear continuous time modeling approaches in panel research. Statistica Neerlandica, 62(1), 29–57.MathSciNetzbMATHGoogle Scholar
  91. Singer, H. (2010). SEM modeling with singular moment matrices. Part I: ML-estimation of time series. Journal of Mathematical Sociology, 34(4), 301–320. CrossRefGoogle Scholar
  92. Singer, H. (2011). Continuous-discrete state-space modeling of panel data with nonlinear filter algorithms. Advances in Statistical Analysis, 95, 375–413. MathSciNetCrossRefGoogle Scholar
  93. Singer, H. (2012). SEM modeling with singular moment matrices. Part II: ML-estimation of sampled stochastic differential equations. Journal of Mathematical Sociology, 36(1), 22–43. MathSciNetCrossRefGoogle Scholar
  94. Singer, H. (2014). Importance sampling for Kolmogorov backward equations. Advances in Statistical Analysis, 98(4), 345–369. MathSciNetCrossRefGoogle Scholar
  95. Singer, H. (2016). Simulated maximum likelihood for continuous-discrete state space models using Langevin importance sampling (Diskussionsbeiträge Fakultät Wirtschaftswissenschaft No. 497). Paper presented at the 9th International Conference on Social Science Methodology (RC33), 11–16 September 2016, Leicester, UK.
  96. Stramer, O., Bognar, M., & Schneider, P. (2010). Bayesian inference for discretely sampled Markov processes with closed-form likelihood expansions. Journal of Financial Econometrics, 8(4), 450–480. CrossRefGoogle Scholar
  97. Stratonovich, R. L. (1971). On the probability functional of diffusion processes. In Selected translations in mathematical statistics and probability, Vol. 10 (pp. 273–286).Google Scholar
  98. Stratonovich, R. L. (1989). Some Markov methods in the theory of stochastic processes in nonlinear dynamic systems. In F. Moss & P. McClintock (Eds.), Noise in nonlinear dynamic systems (pp. 16–71). Cambridge: Cambridge University Press. CrossRefGoogle Scholar
  99. Stuart, A. M., Voss, J., & Wiberg, P. (2004). Conditional path sampling of SDEs and the Langevin MCMC method. Communications in Mathematical Sciences, 2(4), 685–697. MathSciNetCrossRefGoogle Scholar
  100. Thomas, E. A. C., & Martin, J. A. (1976). Analyses of parent-infant-interaction. Psychological Review, 83(2), 141–156. CrossRefGoogle Scholar
  101. Van Kampen, N. G. (1981). Itô vs. Stratonovich. Journal of Statistical Physics, 24, 175–187. MathSciNetCrossRefGoogle Scholar
  102. Wei, G. W., Zhang, D. S., Kouric, D. J., & Hoffman, D. K. (1997). Distributed approximating functional approach to the Fokker-Planck equation: Time propagation. Journal of Chemical Physics, 107(8), 3239–3246. CrossRefGoogle Scholar
  103. Weidlich, W., & Haag, G. (1983). Quantitative sociology. Berlin: Springer.zbMATHGoogle Scholar
  104. Wong, E., & Hajek, B. (1985). Stochastic processes in engineering systems. New York: Springer. CrossRefGoogle Scholar
  105. Yoo, H. (2000). Semi-discretization of stochastic partial differential equations on R1 by a finite-difference method. Mathematics of Computation, 69(230), 653–666. MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of EconomicsFernUniversität in HagenHagenGermany

Personalised recommendations