Discontinuity Detection and Removal from Data Time Series

Abstract

The aim of time series analysis is to distinguish between stochastic and deterministic signals, which are generated by different sources and mixed in the data time series. Before analyzing long term linear trend and periodic effects, it is necessary to detect and remove time series discontinuities, often undocumented. Discontinuities can occur in the case of hardware change, data model change or even signal source and environmental variations.A data time series can be interpreted as a stochastic process plus a step function that represents the time series discontinuities or jumps. Modeling the process as a discrete-time linear system, it can be described by a finite state vector evolving with known dynamics, and by constant biases. The constant biases are described by a matrix of zeroes and ones, but generally the number and the position of jumps are unknown, and it cannot be defined univocally.Since it is not possible to build a bias model a priori, the null hypothesis H ₀ with no jump can be tested against a certain number of alternative hypotheses H _A, with a jump in a given epoch. An alternative hypothesis can be formulated for each observation epoch. The adequacy of the model can be verified using the ratio test, which is known to have the χ² distribution. After detecting the jumps, they can be estimated and removed. Simulated and real data examples will be given.