Change-Point Detection in Time-Series Data by Relative Density-Ratio Estimation

  • Song Liu
  • Makoto Yamada
  • Nigel Collier
  • Masashi Sugiyama
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7626)


The objective of change-point detection is to discover abrupt property changes lying behind time-series data. In this paper, we present a novel statistical change-point detection algorithm that is based on non-parametric divergence estimation between two retrospective segments. Our method uses the relative Pearson divergence as a divergence measure, and it is accurately and efficiently estimated by a method of direct density-ratio estimation. Through experiments on real-world human-activity sensing, speech, and Twitter datasets, we demonstrate the usefulness of the proposed method.


change-point detection distribution comparison relative density-ratio estimation kernel methods time-series data 


  1. 1.
    Basseville, M., Nikiforov, I.V.: Detection of Abrupt Changes: Theory and Application. Prentice-Hall, Inc., Upper Saddle River (1993)Google Scholar
  2. 2.
    Gustafsson, F.: The marginalized likelihood ratio test for detecting abrupt changes. IEEE Transactions on Automatic Control 41(1), 66–78 (1996)MathSciNetzbMATHCrossRefGoogle Scholar
  3. 3.
    Takeuchi, Y., Yamanishi, K.: A unifying framework for detecting outliers and change points from non-stationary time series data. IEEE Transactions on Knowledge and Data Engineering 18(4), 482–489 (2006)CrossRefGoogle Scholar
  4. 4.
    Moskvina, V., Zhigljavsky, A.: Change-point detection algorithm based on the singular-spectrum analysis. Communications in Statistics: Simulation and Computation 32, 319–352 (2003)MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Kawahara, Y., Yairi, T., Machida, K.: Change-point detection in time-series data based on subspace identification. In: Proceedings of the 7th IEEE International Conference on Data Mining, pp. 559–564 (2007)Google Scholar
  6. 6.
    Kawahara, Y., Sugiyama, M.: Sequential change-point detection based on direct density-ratio estimation. Statistical Analysis and Data Mining 5(2), 114–127 (2012)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Sugiyama, M., Suzuki, T., Kanamori, T.: Density Ratio Estimation in Machine Learning. Cambridge University Press, Cambridge (2012)zbMATHCrossRefGoogle Scholar
  8. 8.
    Sugiyama, M., Suzuki, T., Nakajima, S., Kashima, H., von Buenau, P., Kawanabe, M.: Direct importance estimation for covariate shift adaptation. Annals of the Institute of Statistical Mathematics 60(4), 699–746 (2008)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Desobry, F., Davy, M., Doncarli, C.: An online kernel change detection algorithm. IEEE Transactions on Signal Processing 53(8), 2961–2974 (2005)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Kanamori, T., Hido, S., Sugiyama, M.: A least-squares approach to direct importance estimation. Journal of Machine Learning Research 10, 1391–1445 (2009)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Yamada, M., Suzuki, T., Kanamori, T., Hachiya, H., Sugiyama, M.: Relative density-ratio estimation for robust distribution comparison. Advances in Neural Information Processing Systems 24, 594–602 (2011)Google Scholar
  12. 12.
    Ali, S.M., Silvey, S.D.: A general class of coefficients of divergence of one distribution from another. Journal of the Royal Statistical Society, Series B 28(1), 131–142 (1966)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Csiszár, I.: Information-type measures of difference of probability distributions and indirect observation. Studia Scientiarum Mathematicarum Hungarica 2, 229–318 (1967)Google Scholar
  14. 14.
    Liu, S., Yamada, M., Collier, N., Sugiyama, M.: Change-point detection in time-series data by relative density-ratio estimation. arXiv 1203.0453 (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Song Liu
    • 1
  • Makoto Yamada
    • 2
  • Nigel Collier
    • 3
  • Masashi Sugiyama
    • 1
  1. 1.Tokyo Institute of TechnologyMeguro-kuJapan
  2. 2.NTT Communication Science LaboratoriesSeika-choJapan
  3. 3.National Institute of InformaticsChiyoda-kuJapan

Personalised recommendations