Encyclopedia of Database Systems

2018 Edition
| Editors: Ling Liu, M. Tamer Özsu

Anomaly Detection on Streams

  • Spiros Papadimitriou
Reference work entry
DOI: https://doi.org/10.1007/978-1-4614-8265-9_18

Definition

Anomaly detection generally refers to the process of automatically detecting events or behaviors which deviate from those considered normal. It is an unsupervised process, and can thus detect anomalies which have not been previously encountered. It is based on estimating a model of typical behavior from past observations and consequently comparing current observations against this model. It can be performed either on a single stream or among multiple streams. Anomaly detection encompasses outlier detection as well as change detection and therefore is closely related to forecasting and clustering methods.

Historical Background

Anomaly detection in streams has close connections to traditional outlier detection, as well as to change detection. The former is a common and widely studied topic in statistics [11]. The latter emerged in the context of statistical monitoring and control for continuous processes and the widely used CUSUM algorithm was proposed as early as 1954 [9]....

This is a preview of subscription content, log in to check access.

Recommended Reading

  1. 1.
    Aggarwal CC, Han J, Wang J, and Yu PS. A Framework for clustering evolving data streams. In: Proceedings of the 29th International Conference on Very Large Data Bases; 2003. p. 81–92.CrossRefGoogle Scholar
  2. 2.
    Aggarwal CC, Yu PS. A survey of synopsis construction in data streams. In: Data streams: models and algorithms. New York: Springer; 2007.zbMATHCrossRefGoogle Scholar
  3. 3.
    Cormode G, Muthukrishnan S. What’s new: finding significant differences in network data streams. IEEE/ACM Trans Netw. 2005;13(6):1219–32.CrossRefGoogle Scholar
  4. 4.
    Guha S, Meyerson A, Mishra N, Motwani R, O’Callaghan L. Clustering data streams: theory and practice. IEEE Trans Knowl Data Eng. 2003;15(3):515–28.CrossRefGoogle Scholar
  5. 5.
    Hulten G, Spencer L, and Domingos P. Mining time-changing data streams. In: Proceedings of the 7th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2001. p. 97–106.Google Scholar
  6. 6.
    Jain AK, Murty MN, Flynn PJ. Data clustering: a review. ACM Comput Surv. 1999;31(3):264–323.CrossRefGoogle Scholar
  7. 7.
    Kleinberg J. Bursty and hierarchical structure in streams. In: Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2002. p. 91–101.Google Scholar
  8. 8.
    Lee W, Stolfo SJ, Mok KW. Adaptive intrusion detection: a data mining approach. Artif Intell Rev. 2000;14(6):533–67.zbMATHCrossRefGoogle Scholar
  9. 9.
    Page ES. Continuous inspection schemes. Biometrika. 1954;41(1):100–15.MathSciNetzbMATHCrossRefGoogle Scholar
  10. 10.
    Papadimitriou S, Sun J, and Faloutsos C. Streaming pattern discovery in multiple time-series. In: Proceedings of the 31st International Conference on Very Large Data Bases; 2005. p. 697–708.Google Scholar
  11. 11.
    Peter JR, Annick ML. Robust regression and outlier detection. New York: Wiley; 1987.zbMATHGoogle Scholar
  12. 12.
    Wang H, Fan W, Yu PS, and Han J. Mining concept-drifting data streams using ensemble classifiers. In: Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2003. p. 226–35.Google Scholar
  13. 13.
    Zhu Y and Shasha D. StatStream: statistical monitoring of thousands of data streams in real time. In: Proceedings of the 28th International Conference on Very Large Data Bases; 2002. p. 358–69.CrossRefGoogle Scholar
  14. 14.
    Zhu Y and Shasha D. Efficient elastic burst detection in data streams. In: Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; 2003. p. 336–45.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.IBM T.J. Watson Research CenterHawthorneUSA

Section editors and affiliations

  • Divesh Srivastava
    • 1
  1. 1.AT&T Labs - ResearchAT&TBedminsterUSA