Advertisement

Asymptotically Optimal Quickest Change Detection in Multistream Data—Part 1: General Stochastic Models

  • Alexander G. TartakovskyEmail author
Article
  • 6 Downloads

Abstract

Assume that there are multiple data streams (channels, sensors) and in each stream the process of interest produces generally dependent and non-identically distributed observations. When the process is in a normal mode (in-control), the (pre-change) distribution is known, but when the process becomes abnormal there is a parametric uncertainty, i.e., the post-change (out-of-control) distribution is known only partially up to a parameter. Both the change point and the post-change parameter are unknown. Moreover, the change affects an unknown subset of streams, so that the number of affected streams and their location are unknown in advance. A good changepoint detection procedure should detect the change as soon as possible after its occurrence while controlling for a risk of false alarms. We consider a Bayesian setup with a given prior distribution of the change point and propose two sequential mixture-based change detection rules, one mixes a Shiryaev-type statistic over both the unknown subset of affected streams and the unknown post-change parameter and another mixes a Shiryaev–Roberts-type statistic. These rules generalize the mixture detection procedures studied by Tartakovsky (IEEE Trans Inf Theory 65(3):1413–1429, 2019) in a single-stream case. We provide sufficient conditions under which the proposed multistream change detection procedures are first-order asymptotically optimal with respect to moments of the delay to detection as the probability of false alarm approaches zero.

Keywords

Asymptotic optimality Changepoint detection General non-i.i.d. models Hidden Markov models Moments of the delay to detection r-Complete convergence Statistical process control Surveillance 

Mathematics Subject Classification (2010)

MSC 62L10 MSC 62L15 MSC 60G40 MSC 62C10 MSC 62C20 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes

Acknowledgements

The work was supported in part by the Russian Ministry of Education and Science 5-100 excellence project, the Russian Federation Ministry of Education and Science Arctic program and the grant 18-19-00452 from the Russian Science Foundation at the Moscow Institute of Physics and Technology.

The author would like to thank two referees for useful comments.

References

  1. Bakut PA, Bolshakov IA, Gerasimov BM, Kuriksha AA, Repin VG, Tartakovsky GP, Shirokov VV (1963) Stat Radar Theory. In: Tartakovsky GP (ed). in Russian, vol 1. Sovetskoe Radio, MoscowGoogle Scholar
  2. Chan HP (2017) Optimal sequential detection in multi-stream data. Ann Stat 45(6):2736–2763MathSciNetCrossRefzbMATHGoogle Scholar
  3. Fellouris G, Sokolov G (2016) Second-order asymptotic optimality in multichannel sequential detection. IEEE Trans Inf Theory 62(6):3662–3675.  https://doi.org/10.1109/TIT.2016.2549042 CrossRefzbMATHGoogle Scholar
  4. Fellouris G, Tartakovsky AG (2017) Multichannel sequential detection—Part I: non-i.i.d. data. IEEE Trans Inf Theory 63(7):4551–4571.  https://doi.org/10.1109/TIT.2017.2689785 MathSciNetCrossRefzbMATHGoogle Scholar
  5. Lai TL (1995) Sequential changepoint detection in quality control and dynamical systems (with discussion). J R Stat Soc Ser B Methodol 57(4):613–658MathSciNetzbMATHGoogle Scholar
  6. Lai TL (1998) Information bounds and quick detection of parameter changes in stochastic systems. IEEE Trans Inf Theory 44(7):2917–2929MathSciNetCrossRefzbMATHGoogle Scholar
  7. Mei Y (2010) Efficient scalable schemes for monitoring a large number of data streams. Biometrika 97(2):419–433MathSciNetCrossRefzbMATHGoogle Scholar
  8. Pollak M, Tartakovsky AG (2009) Optimality properties of the Shiryaev–Roberts procedure. Stat Sin 19(4):1729–1739MathSciNetzbMATHGoogle Scholar
  9. Polunchenko AS, Tartakovsky AG (2010) On optimality of the Shiryaev–Roberts procedure for detecting a change in distribution. Ann Stat 38(6):3445–3457MathSciNetCrossRefzbMATHGoogle Scholar
  10. Tartakovsky AG (2005) Asymptotic performance of a multichart CUSUM test under false alarm probability constraint. In: Proceedings of the 44th IEEE conference decision and control and European control conference (CDC-ECC’05), Seville, SP, IEEE, Omnipress CD-ROM, pp 320–325Google Scholar
  11. Tartakovsky AG (2017) On asymptotic optimality in sequential changepoint detection: non-iid case. IEEE Trans Inf Theory 63(6):3433–3450.  https://doi.org/10.1109/TIT.2017.2683496 MathSciNetCrossRefzbMATHGoogle Scholar
  12. Tartakovsky AG (2019) Asymptotic optimality of mixture rules for detecting changes in general stochastic models. IEEE Trans Inf Theory 65(3):1413–1429.  https://doi.org/10.1109/TIT.2018.287686 MathSciNetCrossRefzbMATHGoogle Scholar
  13. Tartakovsky AG, Brown J (2008) Adaptive spatial-temporal filtering methods for clutter removal and target tracking. IEEE Trans Aerosp Electron Syst 44(4):1522–1537CrossRefGoogle Scholar
  14. Tartakovsky AG, Veeravalli VV (2005) General asymptotic Bayesian theory of quickest change detection. Theory of Probability and its Applications 49(3):458–497MathSciNetCrossRefzbMATHGoogle Scholar
  15. Tartakovsky AG, Rozovskii BL, Blaźek R B, Kim H (2006) Detection of intrusions in information systems by sequential change-point methods. Stat Methodology 3(3):252–293MathSciNetCrossRefzbMATHGoogle Scholar
  16. Tartakovsky AG, Pollak M, Polunchenko AS (2012) Third-order asymptotic optimality of the generalized Shiryaev–Roberts changepoint detection procedures. Theory of Probability and its Applications 56 (3):457–484.  https://doi.org/10.1137/S0040585X97985534 MathSciNetCrossRefzbMATHGoogle Scholar
  17. Tartakovsky AG, Nikiforov IV, Basseville M (2014) Sequential analysis: hypothesis testing and changepoint detection. Monographs on Statistics and Applied Probability, Chapman & Hall/CRC Press Boca Raton, London, New YorkGoogle Scholar
  18. Willsky AS, Jones HL (1976) A generalized likelihood ratio approach to the detection and estimation of jumps in linear systems. IEEE Trans Autom Control 21 (1):108–112MathSciNetCrossRefzbMATHGoogle Scholar
  19. Xie Y, Siegmund D (2013) Sequential multi-sensor change-point detection. Ann Stat 41(2):670–692MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Space Informatics LaboratoryMoscow Institute of Physics and TechnologyDolgoprudnyRussia
  2. 2.AGT StatConsultLos AngelesUSA

Personalised recommendations