First- and Higher-Order Continuous Time Models for Arbitrary N Using SEM

  • Johan H. L. OudEmail author
  • Manuel C. Voelkle
  • Charles C. Driver


In this chapter we review continuous time series modeling and estimation by extended structural equation models (SEM) for single subjects and N > 1. First-order as well as higher-order models will be dealt with. Both will be handled by the general state space approach which reformulates higher-order models as first-order models. In addition to the basic model, the extensions of exogenous variables and traits (random intercepts) will be introduced. The connection between continuous time and discrete time for estimating the model by SEM will be made by the exact discrete model (EDM). It is by the EDM that the exact estimation procedure in this chapter differentiates from many approximate procedures found in the literature. The proposed analysis procedure will be applied to the well-known Wolfer sunspot data, an N = 1 time series that has been analyzed by several continuous time analysts in the past. The analysis will be carried out by ctsem, an R-package for continuous time modeling that interfaces to OpenMx, and the results will be compared to those reported in the previous studies.


  1. Arnold, L. (1974). Stochastic differential equations. New York: Wiley.zbMATHGoogle Scholar
  2. Bergstrom, A. R. (1984). Continuous time stochastic models and issues of aggregation over time. In Z. Griliches & M. D. Intriligator (Eds.), Handbook of econometrics (Vol. 2, pp. 1145–1212). Amsterdam: North-Holland. Google Scholar
  3. Boker, S. M. (2001). Differential structural equation modeling of intraindividual variability. In L. M. Collins & A. G. Sayer (Eds.), New methods for the analysis of change (pp. 5–27). Washington, DC: American Psychological Association. CrossRefGoogle Scholar
  4. Boker, S. M., Neale, M., Maes, H., Wilde, M., Spiegel, M., Brick, T., …Fox, J. (2011). OpenMx: An open source extended structural equation modeling framework. Psychometrika, 76(2), 306–317. MathSciNetCrossRefGoogle Scholar
  5. Boker, S. M., Neale, M., & Rausch, J. (2004). Latent differential equation modeling with multivariate multi-occasion indicators. In K. van Montfort, J. H. L. Oud, & A. Satorra (Eds.), Recent developments on structural equation models (pp. 151–174). Amsterdam: Kluwer.CrossRefGoogle Scholar
  6. Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley. CrossRefGoogle Scholar
  7. Box, G. E. P., & Jenkins, G. M. (1970). Time serie analysis: Forecasting and control. Oakland, CA: Holden-Day.zbMATHGoogle Scholar
  8. Brockwell, P. J. (2004). Representations of continuous-time ARMA processes. Journal of Applied Probability, 41(a), 375–382. MathSciNetCrossRefGoogle Scholar
  9. Caines, P. E. (1988). Linear stochastic systems. New York: Wiley.zbMATHGoogle Scholar
  10. Chan, K., & Tong, H. (1987). A note on embedding a discrete parameter ARMA model in a continuous parameter ARMA model. Journal of Time Series Analysis, 8, 277–281. MathSciNetCrossRefGoogle Scholar
  11. Deistler, M. (1985). General structure and parametrization of arma and state-space systems and its relation to statistical problems. In E. J. Hannan, P. R. Krishnaiah, & M. M. Rao (Eds.), Handbook of statistics: Volume 5. Time series in the time domain (pp. 257–277). Amsterdam: North Holland. Google Scholar
  12. Driver, C. C., Oud, J. H., & Voelkle, M. C. (2017). Continuous time structural equation modeling with R package ctsem. Journal of Statistical Software, 77, 1–35. CrossRefGoogle Scholar
  13. Durbin, J., & Koopman, S. J. (2001). Time series analysis by state space methods. Oxford: Oxford University Press.
  14. Gasimova, F., Robitzsch, A., Wilhelm, O., Boker, S. M., Hu, Y., & Hülür, G. (2014). Dynamical systems analysis applied to working memory data. Frontiers in Psychology, 5, 687 (Advance online publication).
  15. Goodrich, R. L., & Caines, P. (1979). Linear system identification from nonstationary cross-sectional data. IEEE Transactions on Automatic Control, 24, 403–411. MathSciNetCrossRefGoogle Scholar
  16. Gottman, J. M. (1981). Time-series analysis: A comprehensive introduction for social scientists. Cambridge: Cambridge University Press.zbMATHGoogle Scholar
  17. Hamaker, E. L., Dolan, C. V., & Molenaar, C. M. (2003). ARMA-based SEM when the number of time points T exceeds the number of cases N: Raw data maximum likelihood. Structural Equation Modeling, 10, 352–379. MathSciNetCrossRefGoogle Scholar
  18. Hamerle, A., Nagl, W., & Singer, H. (1991). Problems with the estimation of stochastic differential equations using structural equations models. Journal of Mathematical Sociology, 16(3), 201–220. CrossRefGoogle Scholar
  19. Hannan, E. J., & Deistler, M. (1988). The statistical theory of linear systems. New York: Wiley.zbMATHGoogle Scholar
  20. Hansen, L. P., & Sargent, T. J. (1983). The dimensionality of the aliasing problem. Econometrica, 51, 377–388. MathSciNetCrossRefGoogle Scholar
  21. Harvey, A. C. (1981). Time series models. Oxford: Philip Allen.zbMATHGoogle Scholar
  22. Harvey, A. C. (1989). Forecasting, structural time series models and the Kalman filter. Cambridge: Cambridge University Press.Google Scholar
  23. Higham, N. J. (2009). The scaling and squaring method for the matrix exponential revisited. SIAM Review, 51, 747–764. MathSciNetCrossRefGoogle Scholar
  24. Jones, R. H. (1993). Longitudinal data with serial correlation; a state space approach. London: Chapman & Hall. CrossRefGoogle Scholar
  25. Jöreskog, K. G. (1973). A general method for estimating a linear structural equation system. In A. S. Goldberger & O. D. Duncan (Eds.), Structural equation models in the social sciences (pp. 85–112). New York: Seminar Press.Google Scholar
  26. Jöreskog, K. G. (1977). Structural equation models in the social sciences: Specification, estimation and testing. In P. R. Krishnaiah (Ed.), Applications of statistics (pp. 265–287). Amsterdam: North Holland.Google Scholar
  27. Jöreskog, K. G., & Sörbom, D. (1976). LISREL III: Estimation of linear structural equation systems by maximum likelihood methods: A FORTRAN IV program. Chicago: National Educational Resources.Google Scholar
  28. Jöreskog, K. G., & Sörbom, D. (1996). LISREL 8: User’s reference guide. Chicago: Scientific Software International.Google Scholar
  29. Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Journal of Basic Engineering, 82, 35–45 (Trans. ASME, ser. D).CrossRefGoogle Scholar
  30. Kuo, H. H. (2006). Introduction to stochastic integration. New York: Springer.zbMATHGoogle Scholar
  31. Ljung, L. (1985). Estimation of parameters in dynamical systems. In E. J. Hannan, P. R. Krishnaiah, & M. M. Rao (Eds.), Handbook of statistics: Volume 5. Time series in the time domain (pp. 189–211). Amsterdam: North Holland. Google Scholar
  32. Lütkepohl, H. (1991). Introduction to multiple time series analysis. Berlin: Springer.CrossRefGoogle Scholar
  33. McCleary, R., & Hay, R. (1980). Applied time series analysis for the social sciences. Beverly Hills: Sage.Google Scholar
  34. Mills, T. C. (2012). A very British affair: Six Britons and the development of time series analysis during the 20th century. Basingstoke: Palgrave Macmillan.zbMATHGoogle Scholar
  35. Neale, M. C. (1997). Mx: Statistical modeling (4th ed.). Richmond, VA: Department of Psychiatry.Google Scholar
  36. Neale, M. C., Hunter, M. D., Pritikin, J. N., Zahery, M., Brick, T. R., Kirkpatrick, R. M., …Boker, S. M. (2016). OpenMx 2.0: Extended structural equation and statistical modeling. Psychometrika, 81, 535–549. MathSciNetCrossRefGoogle Scholar
  37. Oud, J. H. L. (1978). Systeem-methodologie in sociaal-wetenschappelijk onderzoek [Systems methodology in social science research]. Unpublished doctoral dissertation, Radboud University Nijmegen, Nijmegen.Google Scholar
  38. Oud, J. H. L. (2007). Comparison of four procedures to estimate the damped linear differential oscillator for panel data. In K. van Montfort, J. H. L. Oud, & A. Satorra (Eds.), Longitudinal models in the behavioral and related sciences (pp. 19–39). Mahwah, NJ: Erlbaum.Google Scholar
  39. Oud, J. H. L., & Delsing, M. J. M. H. (2010). Continuous time modeling of panel data by means of SEM. In K. van Montfort, J. H. L. Oud, & A. Satorra (Eds.), Longitudinal research with latent variables (pp. 201–244). New York: Springer. CrossRefGoogle Scholar
  40. Oud, J. H. L., & Jansen, R. A. R. G. (2000). Continuous time state space modeling of panel data by means of SEM. Psychometrika, 65, 199–215. MathSciNetCrossRefGoogle Scholar
  41. Oud, J. H. L., & Singer, H. (2008). Continuous time modeling of panel data: SEM versus filter techniques. Statistica Neerlandica, 62, 4–28. MathSciNetCrossRefGoogle Scholar
  42. Oud, J. H. L., van den Bercken J. H., & Essers, R. J. (1990). Longitudinal factor score estimation using the Kalman filter. Applied Psychological Measurement, 14, 395–418. CrossRefGoogle Scholar
  43. Oud, J. H. L., & Voelkle, M. C. (2014). Do missing values exist? Incomplete data handling in cross-national longitudinal studies by means of continuous time modeling. Quality & Quantity, 48, 3271–3288. CrossRefGoogle Scholar
  44. Oud, J. H. L., Voelkle, M. C., & Driver, C. C. (2018). SEM based CARMA time series modeling for arbitrary N. Multivariate Behavioral Research, 53(1), 36–56. CrossRefGoogle Scholar
  45. Phadke, M., & Wu, S. (1974). Modeling of continuous stochastic processes from discrete observations with application to sunspots data. Journal of the American Statistical Association, 69, 325–329. CrossRefGoogle Scholar
  46. Phillips, P. C. B. (1973). The problem of identification in finite parameter continuous time models. In A. R. Bergstrom (Ed.), Statistical inference in continuous time models (pp. 135–173). Amsterdam: North-Holland.Google Scholar
  47. Prado, R., & West, M. (2010). Time series: Modeling, computation, and inference. Boca Raton: Chapman & Hall.CrossRefGoogle Scholar
  48. R Core Team. (2015). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from
  49. Shumway, R. H., & Stoffer, D. (2000). Time series analysis and its applications. New York: Springer. CrossRefGoogle Scholar
  50. Singer, H. (1990). Parameterschätzung in zeitkontinuierlichen dynamischen Systemen [Parameter estimation in continuous time dynamic systems]. Konstanz: Hartung-Gorre.Google Scholar
  51. Singer, H. (1991). LSDE-A program package for the simulation, graphical display, optimal filtering and maximum likelihood estimation of linear stochastic differential equations: User’s guide. Meersburg: Author.Google Scholar
  52. Singer, H. (1992). The aliasing-phenomenon in visual terms. Journal of Mathematical Sociology, 17, 39–49. MathSciNetCrossRefGoogle Scholar
  53. Singer, H. (1998). Continuous panel models with time dependent parameters. Journal of Mathematical Sociology, 23, 77–98.CrossRefGoogle Scholar
  54. Singer, H. (2010). SEM modeling with singular moment part I: ML estimation of time series. Journal of Mathematical Sociology, 34, 301–320. CrossRefGoogle Scholar
  55. Singer, H. (2012). SEM modeling with singular moment part II: ML-estimation of sampled stochastic differential equations. Journal of Mathematical Sociology, 36, 22–43. MathSciNetCrossRefGoogle Scholar
  56. Steele, J. S., & Ferrer, E. (2011a). Latent differential equation modeling of self-regulatory and coregulatory affective processes. Multivariate Behavioral Research, 46, 956–984. CrossRefGoogle Scholar
  57. Steele, J. S., & Ferrer, E. (2011b). Response to Oud & Folmer: Randomness and residuals. Multivariate Behavioral Research, 46, 994–1003. CrossRefGoogle Scholar
  58. Tómasson, H. (2011). Some computational aspects of Gaussian CARMA modelling. Vienna: Institute for Advanced Studies.zbMATHGoogle Scholar
  59. Tómasson, H. (2015). Some computational aspects of Gaussian CARMA modelling. Statistical Computation, 25, 375–387. MathSciNetCrossRefGoogle Scholar
  60. Tsai, H., & Chan, K.-S. (2000). A note on the covariance structure of a continuous-time ARMA process. Statistica Sinica, 10, 989–998.MathSciNetzbMATHGoogle Scholar
  61. Voelkle, M. C., & Oud, J. H. L. (2013). Continuous time modelling with individually varying time intervals for oscillating and non-oscillating processes. British Journal of Mathematical and Statistical Psychology, 66, 103–126. MathSciNetCrossRefGoogle Scholar
  62. Voelkle, M. C., Oud, J. H. L., Davidov, E., & Schmidt, P. (2012a). An SEM approach to continuous time modeling of panel data: Relating authoritarianism and anomia. Psychological Methods, 17, 176–192. CrossRefGoogle Scholar
  63. Voelkle, M. C., Oud, J. H. L., Oertzen, T. von, & Lindenberger, U. (2012b). Maximum likelihood dynamic factor modeling for arbitrary N and T using SEM. Structural Equation Modeling: A Multidisciplinary Journal, 19, 329–350. MathSciNetCrossRefGoogle Scholar
  64. Wolf, E. J., Harrington, K. M., Clark, S. L., & Miller, M. W. (2013). Sample size requirements for structural equation models: An evaluation of power, bias, and solution propriety. Educational and Psychological Measurement, 73, 913–934. CrossRefGoogle Scholar
  65. Yu, J. (2014). Econometric analysis of continuous time models: A survey of Peter Philips’s work and some new results. Econometric Theory, 30, 737–774. MathSciNetCrossRefGoogle Scholar
  66. Zadeh, L. A., & Desoer, C. A. (1963). Linear system theory: The state space approach. New York: McGraw-Hill.zbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Johan H. L. Oud
    • 1
    Email author
  • Manuel C. Voelkle
    • 2
  • Charles C. Driver
    • 3
  1. 1.Behavioural Science InstituteUniversity of NijmegenNijmegenThe Netherlands
  2. 2.Department of PsychologyHumboldt-Universität zu BerlinBerlinGermany
  3. 3.Max Planck Institute for Human DevelopmentBerlinGermany

Personalised recommendations