Computational Statistics

, Volume 18, Issue 1, pp 79–106 | Cite as

Simulated Maximum Likelihood in Nonlinear Continuous-Discrete State Space Models: Importance Sampling by Approximate Smoothing

  • Hermann Singer


The likelihood function of a continuous-discrete state space model is computed recursively by Monte Carlo integration, using importance sampling techniques. A functional integral representation of the transition density is utilized and importance densities are obtained by smoothing. Examples are the likelihood surfaces of an AR(2) process, a Ginzburg-Landau model and stock price models with stochastic volatilities.


Stochastic differential equations Nonlinear filtering Discrete noisy measurements Maximum likelihood estimation Monte Carlo simulation Importance sampling 


  1. Alspach, D. & Sorenson, H. (1972), ‘Nonlinear Bayesian estimation using Gaussian sum approximations’, IEEE Transactions on Automatic Control 17, 439–448.CrossRefzbMATHGoogle Scholar
  2. Anderson, B. & Moore, J. (1979), Optimal Filtering, Prentice Hall, Englewood Cliffs.zbMATHGoogle Scholar
  3. Apple Computer (2001), Macintosh Programmer’s Workshop,
  4. Arnold, L. (1974), Stochastic Differential Equations, John Wiley, New York.zbMATHGoogle Scholar
  5. Arnold, V. (1973), Ordinary Differential Equations, MIT Press, Cambridge (Mass.), London.Google Scholar
  6. Arnold, V. (1986), Catastrophy Theory, Springer, Berlin.CrossRefGoogle Scholar
  7. Bergstrom, A. (1990), Continuous Time Econometric Modelling, Oxford University Press, Oxford.Google Scholar
  8. Carlin, B., Poison, N. & Stoffer, D. (1992), ‘A Monte Carlo Approach to Non-normal and Nonlinear State-Space Modeling’, Journal of the American Statistical Association 87, 493–500.CrossRefGoogle Scholar
  9. Durbin, J. & Koopman, S. (1997), ‘Monte Carlo maximum likelihood estimation for non-Gaussian state space models’, Biometrika 84,3, 669–684.CrossRefMathSciNetzbMATHGoogle Scholar
  10. Durbin, J. & Koopman, S. (2000), ‘Time series analysis of non-Gaussian observations based on state space models from both classical and Bayesian perspectives.’, Journal of the Royal Statistical Association B 62, 1, 3–56.CrossRefMathSciNetzbMATHGoogle Scholar
  11. Engle, R. & Bollerslev, L. (1986), ‘Modelling the persistence of conditional variances’, Econometric Reviews 5, 1–50.CrossRefMathSciNetzbMATHGoogle Scholar
  12. Fahrmeir, L. & Kaufmann, H. (1991), ‘On Kalman Filtering, Posterior Mode Estimation and Fisher Scoring in Dynamic Exponential Family Regression’, Metrika 38, 37–60.CrossRefMathSciNetzbMATHGoogle Scholar
  13. Frey, M. (1996), ‘A Wiener Filter, State-Space Flux-Optimal Control Against Escape from a Potential Well’, IEEE Transactions on Automatic Control 41, 2, 216–223.CrossRefMathSciNetzbMATHGoogle Scholar
  14. Gelb, A., ed. (1974), Applied Optimal Estimation, MIT Press, Cambridge, Mass.Google Scholar
  15. Gordon, N., Salmond, D. & Smith, A. (1993), ‘Novel approach to nonlinear/non-Gaussian Bayesian state estimation’, IEEE Transactions on Radar and Signal Procesing 140, 2, 107–113.CrossRefGoogle Scholar
  16. Haken, H. (1977), Synergetics, Springer, Berlin.CrossRefzbMATHGoogle Scholar
  17. Hammersley, J. & Handscomb, D. (1964), Monte Carlo Methods, Methuen, London.CrossRefzbMATHGoogle Scholar
  18. Harvey, A. (1989), Forecasting, structural time series models and the Kalman filter, Cambridge University Press, Cambridge.Google Scholar
  19. Harvey, A. & Stock, J. (1985), ‘The estimation of higher order continuous time autoregressive models’, Econometric Theory 1, 97–112.CrossRefGoogle Scholar
  20. Herings, J. (1996), Static and Dynamic Aspects of General Disequilibrium Theory, Kluwer, Boston, London, Dordrecht.CrossRefzbMATHGoogle Scholar
  21. Holmes, P. J. (1981), ‘Center manifolds, normal forms and bifurcations of vector fields’, Physica D 2, 449–481.CrossRefMathSciNetzbMATHGoogle Scholar
  22. Hull, J. & White, A. (1987), ‘The Pricing of Options with Stochastic Volatilities’, Journal of Finance XLII,2, 281–300.CrossRefGoogle Scholar
  23. Hürzeler, M. & Künsch, H. (1998), ‘Monte Carlo Approximations for General State-Space Models’, Journal of Computational and Graphical Statistics 7,2, 175–193.MathSciNetzbMATHGoogle Scholar
  24. Jazwinski, A. (1970), Stochastic Processes and Filtering Theory, Academic Press, New York.zbMATHGoogle Scholar
  25. Jones, R. (1984), Fitting multivariate models to unequally spaced data, in E. Parzen, ed., ‘Time Series Analysis of Irregularly Observed Data’, Springer, New York, pp. 158–188.CrossRefGoogle Scholar
  26. Jones, R. & Ackerson, L. (1990), ‘Serial correlation in unequally spaced longitudinal data’, Biometrika 77, 721–731.CrossRefMathSciNetGoogle Scholar
  27. Jones, R. & Tryon, P. (1987), ‘Continuous time series models for unequally spaced data applied to modeling atomic clocks’, SIAM J. Sci. Stat. Comput. 8, 71–81.CrossRefMathSciNetzbMATHGoogle Scholar
  28. Kim, S., Shephard, N. & Chib, S. (1998), ‘Stochastic Volatility: Likelihood Inference and Comparision with ARCH Models’, Review of Economic Studies 45, 361–393.CrossRefzbMATHGoogle Scholar
  29. Kitagawa, G. (1987), ‘Non-Gaussian state space modeling of nonstationary time series’, Journal of the American Statistical Association 82, 1032–1063.MathSciNetzbMATHGoogle Scholar
  30. Kitagawa, G. (1996), ‘Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models’, Journal of Computational and Graphical Statistics 5,1, 1–25.MathSciNetGoogle Scholar
  31. Kloeden, P. & Platen, E. (1992), Numerical Solution of Stochastic Differential Equations, Springer, Berlin.CrossRefzbMATHGoogle Scholar
  32. Liptser, R. & Shiryayev, A. (1977, 1978), Statistics of Random Processes, Volumes I and II, Springer, New York, Heidelberg, Berlin.CrossRefzbMATHGoogle Scholar
  33. Lo, A. (1988), ‘Maximum Likelihood Estimation of Generalized Itô Processes with Discretely Sampled Data’, Econometric Theory 4, 231–247.CrossRefMathSciNetGoogle Scholar
  34. Nelson, D. (1990), ‘ARCH models a diffusion approximations’, Journal of Econometrics 45, 7–38.CrossRefMathSciNetzbMATHGoogle Scholar
  35. Nelson, D. (1992), ‘Filtering and forecasting with misspecified ARCH models I’, Journal of Econometrics 52, 61–90.CrossRefMathSciNetzbMATHGoogle Scholar
  36. Press, W., Teukolsky, S., Vetterling, W. & Flannery, B. (1992), Numerical Recipes in C, second edn, Cambridge University Press, Cambridge.zbMATHGoogle Scholar
  37. Risken, H. (1989), The Fokker-Planck Equation, second edn, Springer, Berlin, Heidelberg, New York.CrossRefzbMATHGoogle Scholar
  38. Scott, L. (1987), ‘Option pricing when the variance changes randomly: Theory, estimation, and an application’, Journal of Financial and Quantitative Analysis 22, 419–438.CrossRefGoogle Scholar
  39. Shoji, I. & Ozaki, T. (1997), ‘Comparative Study of Estimation Methods for Continuous Time Stochastic Processes’, Journal of Time Series Analysis 18, 5, 485–506.CrossRefMathSciNetzbMATHGoogle Scholar
  40. Shoji, I. & Ozaki, T. (1998), ‘A statistical method of estimation and simulation for systems of stochastic differential equations’, Biometrika 85, 1, 240–243.CrossRefMathSciNetzbMATHGoogle Scholar
  41. Silverman, B. (1986), Density estimation for statistics and data analysis, Chapman and Hall, London.CrossRefzbMATHGoogle Scholar
  42. Singer, H. (1993b), ‘Continuous-time dynamical systems with sampled data, errors of measurement and unobserved components’, Journal of Time Series Analysis 14, 5, 527–545.CrossRefMathSciNetzbMATHGoogle Scholar
  43. Singer, H. (1995), ‘Analytical score function for irregularly sampled continuous time stochastic processes with control variables and missing values’, Econometric Theory 11, 721–735.CrossRefMathSciNetGoogle Scholar
  44. Singer, H. (1997a), Nonlinear Continuous-Discrete Filtering and ML Estimation using Kernel Density Estimates and Functional Integrals, Regensburger Beiträge zur Statistik und Ökonometrie 40, Universität Regensburg.Google Scholar
  45. Singer, H. (1999b), Parameter Estimation of Nonlinear Stochastic Differential Equations: Simulated Maximum Likelihood vs. Extended Kalman Filter and Itô-Taylor Expansion, Regensburger Beiträge zur Statistik und Ökonometrie 41, Universität Regensburg.Google Scholar
  46. Stratonovich, R. (1989), Some Markov methods in the theory of stochastic processes in nonlinear dynamic systems, in F. Moss & P. McClintock, eds, ‘Noise in nonlinear dynamic systems’, Cambridge University Press, pp. 16–71.Google Scholar
  47. Tanizaki, H. (1996), Nonlinear filters: estimation and applications, second edn, Springer, Berlin.CrossRefzbMATHGoogle Scholar
  48. Tanizaki, H. & Mariano, R. (1995), Prediction, Filtering and Smoothing in Nonlinear and Non-normal Cases using Monte-Carlo Integration, in H. Van Dijk, A. Monfort & B. Brown, eds, ‘Econometric Inference using Simulation Techniques’, John Wiley, pp. 245–261.Google Scholar
  49. Wagner, W. (1988), ‘Monte Carlo Evaluation of Functionals of Solutions of Stochastic Differential Equations. Variance Reduction and Numerical Examples’, Stochastic Analysis and Applications 6, 447–468.CrossRefMathSciNetzbMATHGoogle Scholar
  50. Wiggins, J. (1987), ‘Option values under stochastic volatility’, Journal of Financial Economics 19, 351–372.CrossRefGoogle Scholar
  51. Wolfram, S. (1992), Mathematica, 2nd edn, Addison-Wesley, Redwood City.zbMATHGoogle Scholar
  52. Wong, E. & Hajek, B. (1985), Stochastic Processes in Engineering Systems, Springer, New York.CrossRefzbMATHGoogle Scholar
  53. Zadrozny, P. (1988), ‘Gaussian likelihood of continuous-time armax models when data are stocks and flows at different frequencies’, Econometric Theory 4, 108–124.CrossRefGoogle Scholar

Copyright information

© Physica-Verlag 2003

Authors and Affiliations

  • Hermann Singer
    • 1
  1. 1.Lehrstuhl für angewandte Statistik und Methoden der empirischen SozialforschungFernUniversität HagenHagenGermany

Personalised recommendations