Skip to main content
SpringerLink
Log in

Some Applications of the EM Algorithm to Analyzing Incomplete Time Series Data

  • Conference paper
Book cover Time Series Analysis of Irregularly Observed Data

Part of the book series: Lecture Notes in Statistics ((LNS,volume 25))

Abstract

One may encounter incompletely specified time series data in several distinct forms: (1) observations in time or space may be irregularly observed or (2) the underlying time series model may be incompletely observed, as in the case where one observes only the sum of a signal and a noise process. Maximum likelihood estimators for parameters in these missing data problems can be developed in a simple, heuristically appealing form by utilizing the EM (expectation-maximization) algorithm proposed by Dempster, et al. (1977) and others. Furthermore, the conditional expectations computed as a by-product of applying the algorithm are the empirical Bayes (in the sense of Efron and Morris (1973), (1975)) estimators for the unobserved components. The EM algorithm is reviewed here within the time series context and applied to (i) the parameter estimation and smoothing problem for missing data state-space models and (ii) linear estimation (deconvolution) in a frequency domain regression model.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Bibliography

  • Anderson, T.W. (1971). The Statistical Analysis of Time Series, Wiley, New York.

    MATH  Google Scholar 

  • Anderson, T.W. (1978). Repeated measurements on autoregressive processes. J. Amer. Statist. Assoc., 73, 371–378.

    MathSciNet  Google Scholar 

  • Backus, G. and Gilbert, F. (1970). Uniqueness in the inversion of inaccurate gross earth data. Trans, Phil. R. Soc. 266A, 123–192.

    Article  MathSciNet  Google Scholar 

  • Bloomfield, P. (1970). Spectral Analysis with randomly missing observations. Jour. Roy. Statist. Soc., Ser. B, 32, 369–380.

    MathSciNet  MATH  Google Scholar 

  • Borpujari, A. (1977). An empirical Bayes approach for estimating the mean of N stationary time series. J. Amer. Statist. Assoc., 72, 397–402.

    MathSciNet  MATH  Google Scholar 

  • Box, G.E. P. and Jenkins, G.M. (1970). Time Series Analysis, Forecasting, and Control, Holden Day, San Francisco.

    MATH  Google Scholar 

  • Boyles, R.A. (1980). Convergence results for the EM algorithm. Technical Report No. 13, Division of Statistics, University of California, Davis.

    Google Scholar 

  • — (1983). On the convergence of the EM algorithm. To appear, J. Royal Statist. Soc., Ser. B, 44.

    Google Scholar 

  • Brillinger, D.R. (1975). Time Series Data Analysis and Theory. Holt, Reinhart and Winston, New York.

    MATH  Google Scholar 

  • — (1980). Analysis of variance under time series models. P.R. Krishnaiah ed., Handbook of Statistics, Vol. 1, 237-278. North Holland, New York.

    Google Scholar 

  • Dempster, A.P., Laird, N.M. and Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm. J. of the Royal Statist. Soc, Ser. B, 39, 1–38.

    MathSciNet  MATH  Google Scholar 

  • Deregowski, S.M. (1971). Optimum digital filtering and inverse filtering in the frequency domain. Geophys. Prosp., 19, 729–768.

    Article  Google Scholar 

  • Donoho, D.L. (1981). On minimum entropy convolution. In D.F. Findley, ed., Applied Time Series Analysis II, 565–608. Academic Press, New York.

    Google Scholar 

  • Draper, N.R. and Van Nostrand, C. (1979). Ridge regression and James-Stein estimation: Review and Comments. Technometrics, 21, 451–566.

    MathSciNet  MATH  Google Scholar 

  • Dunsmuir, W. (1981). Estimation for stationary time series when data are irregularly spaced or missing. In Applied Time Series II, D.F. Findley, ed., Academic Press, New York.

    Google Scholar 

  • Dunsmuir, W. and Robinson, P.M. (1981). Estimation of time series models in the presence of missing data. J. Amer. Statist. Assoc., 76, 560–568.

    MATH  Google Scholar 

  • Efron, B. and Morris, C. (1973). Stein’s estimation rule and its competitors — An empirical Bayes approach. J. Amer. Statist. Assoc., 68, 117–130.

    MathSciNet  MATH  Google Scholar 

  • Efron, B. and Morris, C. (1975). Data analysis using Stein’s estimator and its generalizations. J. Amer. Statist. Assoc., 70, 311–319.

    MATH  Google Scholar 

  • Goodrich, R.L. and Caines, P.E. (1979). Linear system identification from nonstationary cross-sectional data. IEEE Trans. on Aut. Control AC-24, 403–411.

    Article  MathSciNet  Google Scholar 

  • Gupta, N.K. and Mehra, R.K. (1974). Computational aspects of maximum likelihood estimation and reduction in sensitivity function calculations. IEEE Trans. on Aut. Control AC-19, 774–783.

    Article  MathSciNet  Google Scholar 

  • Hannan, E.J. (1970). Multiple Time Series. Wiley, New York.

    Book  MATH  Google Scholar 

  • Harrison, P.J. and Stevens, C.F. (1976). Bayesian forecasting. Jour. Royal Statist. Soc., B, 38, 205–247.

    MathSciNet  MATH  Google Scholar 

  • Hartley, H.O. and Hocking, R.R. (1971). The analysis of incomplete data. Biometrics, 14, 174–194.

    Article  Google Scholar 

  • Harvey, A.C. and Phillips, G.D.A. (1979). Maximum likelihood estimation of regression models with autoregressive-moving average disturbances. Biometrika, 66, 49–58.

    MathSciNet  MATH  Google Scholar 

  • Harvey, A.C. and McKenzie, C.R. (1983). Missing observations in dynamic econometric models. Presented at Office of Naval Research Sponsored Symposium on Time Series Analysis of Irregularly Spaced Data. Feb. 10-13.

    Google Scholar 

  • Hoerl, A.E. and Kennard, R.W. (1970). Ridge regression: Biased estimation for some non-orthogonal problems. Technometrics, 12, 55–62.

    MATH  Google Scholar 

  • Hosoya, Y. (1979). Efficient estimation of a model with an autoregressive signal with white noise. Tech. Report No. 37, Dept. of Statistics, Stanford University.

    Google Scholar 

  • Hunt, B.R. (1971). Biased estimation for nonparametric identification of linear systems. Math. Biosci., 10, 215–237.

    Article  MathSciNet  MATH  Google Scholar 

  • Hunt, H.R. (1972). Deconvolution of linear systems by constrained regression and its relationship to the Wiener theory. IEEE Trans. Aut. Control, Oct., 703-705.

    Google Scholar 

  • Jazwinski, A.H. (1970). Stochastic Processes and Filtering Theory. Academic Press, New York.

    MATH  Google Scholar 

  • Jones, R.H. (1962). Spectral analysis with regularly missed observations. Ann. Math. Statist., 33, 455–461.

    Article  MathSciNet  MATH  Google Scholar 

  • Jones, R.H. (1966). Exponential smoothing for multivariate time series. J. Roy. Statist. Soc., Ser. B, 1, 241–251.

    Google Scholar 

  • Jones, R.H. (1980). Maximum likelihood fitting of ARMA models to time series with missing observations. Technometrics, 22, 389–395.

    MathSciNet  MATH  Google Scholar 

  • Kalman, R.E. (1960). A new approach to linear filtering and prediction problems. Trans. ASME J. of Basic Eng., 8, 35–45.

    Article  Google Scholar 

  • Kalman, R.E. and Bucy, R.S. (1961). New results in linear filtering and prediction theory. Trans. ASME J. of Basic Eng., 83, 95–108.

    Article  MathSciNet  Google Scholar 

  • Kendall, M.G. (1973). Time Series. Hafner Press, New York.

    Google Scholar 

  • Kitagawa, G. and Gersch, W. (1982). A smoothness priors approach to the modeling of time series with trend and seasonality. Submitted.

    Google Scholar 

  • Ledolter, J. (1979). A recursive approach to parameter estimation in regression and time series problems. Comm. Statist. Theor. Meth., A8, 1227–1245.

    Article  MathSciNet  MATH  Google Scholar 

  • Meltzer, A., Goodman, C., Langwell, K., Cosler, J., Baghelai, C., and Bobula, J. (1980). Develop Physician and Physician Extender Data Bases. G-155, Final Report, Applied Management Sciences, Inc., Silver Spring, MD 20910.

    Google Scholar 

  • Murray, G.D. (1977). Contribution to discussion of paper by Dempster, Laird and Rubin. J. Roy. Statist. Soc., Ser. B, 39, 27–28.

    Google Scholar 

  • Oldenburg, D.W. (1981). A comprehensive solution to the linear deconvolution problem. Geophys. J. R. Astr. Soc., 65, 331–357.

    Article  Google Scholar 

  • Oldenburg, D.W., Levy, S. and Whittall, K.P. (1981). Wavelet estimation and deconvolution. Geophysics, 46, 1528–1542.

    Google Scholar 

  • Orchard, T. and Woodbury, M.A. (1972). A missing information principle: Theory and applications. Proc. 6th Berkeley Symp. on Math. Statist. and Prob., 1, 697–715.

    MathSciNet  Google Scholar 

  • Pagano, A. (1974). Estimation of models of autoregressive signal plus white noise. Ann. Statist., 2, 99–108.

    Article  MathSciNet  MATH  Google Scholar 

  • Parzen, E. (1961). Spectral analysis of asymptotically stationary time series. Bull. I.S.I., 33rd Session, Paris.

    Google Scholar 

  • Parzen, E. (1963). On spectral analysis with missing observations and amplitude modulation. Sankhya, Series A, 25, 383–392.

    MathSciNet  MATH  Google Scholar 

  • Parzen, E. (1967). Time series analysis for models of signal plus white noise. Spectral Analysis of Time Series, B. Harris, ed., 233-258. Wiley

    Google Scholar 

  • Robinson, P.M. (1979). Distributed lag approximation to linear time-invariant systems. Ann. of Statist., 7, 507–515.

    Article  MATH  Google Scholar 

  • Scheinok, P.S. (1965). Spectral analysis with randomly missed observations: The binomial case. Ann. Math. Statist., 36, 971–977.

    Article  MathSciNet  MATH  Google Scholar 

  • Shumway, R.H. and Dean, W.C. (1968). Best linear unbiased estimation for multivariate stationary processes. Technometrics, 10, 523–534.

    MATH  Google Scholar 

  • Shumway, R.H. (1970). Applied regression and analysis of variance for stationary time series. J. Amer. Statist. Assoc., 65, 1527–1546.

    MATH  Google Scholar 

  • Shumway, R.H. and Blandford, R.R. (1978). On detecting and estimating multiple arrivals from underground nuclear explosions. Seismic Data Analysis Center Report No. SDAC-TR-77-8. Teledyne-Geotech, P.O. Box 334, Alexandria, Virginia 22313.

    Google Scholar 

  • Shumway, R.H., Olsen, D.E. and Levy, L.J. (1981). Estimation and tests of hypotheses for the initial mean and covariance in the Kalman filter model. Comm. of Statist., Theor. & Meth., A10(16), 1625–1641.

    Article  MathSciNet  MATH  Google Scholar 

  • Shumway, R.H. and Stoffer, D.S. (1982a). An approach to time series smoothing and forecasting using the EM algorithm. Jour. Time Series Anal., 3, 253–264.

    Article  MATH  Google Scholar 

  • — (1982b). An algorithm for parameter estimation and smoothing in space-time models with missing data. Final Report P.O. 82-ABA-1198, NOAA Pacific Environmental Group, National Marine Fisheries, Monterey, CA.

    Google Scholar 

  • Stoffer, D.S. (1982). Estimation of parameters in a linear dynamic system with missing observations. Ph.D. dissertation, University of California, Davis.

    Google Scholar 

  • Tan, Suan-Boon (1979). Maximum likelihood estimation in autoregressive processes with missing data. Ph.D. thesis, Univ. of Pittsburgh.

    Google Scholar 

  • Wahba, G. (1969). Estimation of the coefficients in a multidimensional distributed lag model. Econometrica, 37, 398–407.

    Article  MathSciNet  MATH  Google Scholar 

  • Webster, G.M. (1978). Deconvolution. Geophysics Reprint Series, No. 1, Society of Exploration Geophysics, P.O. Box 3098, Tulsa, Oklahoma 74101.

    Google Scholar 

  • Wiggins, R.A. (1978). Minimum entropy deconvolution. Geoexploration, 16, 21–35.

    Article  Google Scholar 

  • Wu, C.F. (1983). On the convergence properties of the EM algorithm. Ann. Statist., 11, 95–103.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division of Statistics, University of California, Davis, CA, 95616, USA

    R. H. Shumway

Authors
  1. R. H. Shumway
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Statistics, Texas A & M University, College Station, TX, 77843, USA

    Emanuel Parzen

Rights and permissions

Reprints and permissions

Copyright information

© 1984 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Shumway, R.H. (1984). Some Applications of the EM Algorithm to Analyzing Incomplete Time Series Data. In: Parzen, E. (eds) Time Series Analysis of Irregularly Observed Data. Lecture Notes in Statistics, vol 25. Springer, New York, NY. https://doi.org/10.1007/978-1-4684-9403-7_14

Download citation

  • DOI: https://doi.org/10.1007/978-1-4684-9403-7_14

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-0-387-96040-1

  • Online ISBN: 978-1-4684-9403-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation