Robustness of Change Detection Algorithms

Abstract

Stream mining is a challenging problem that has attracted considerable attention in the last decade. As a result there are numerous algorithms for mining data streams, from summarizing and analyzing, to change and anomaly detection. However, most research focuses on proposing, adapting or improving algorithms and studying their computational performance. For a practitioner of stream mining, there is very little guidance on choosing a technology suited for a particular task or application.

In this paper, we address the practical aspect of choosing a suitable algorithm by drawing on the statistical properties of power and robustness. For the purpose of illustration, we focus on change detection algorithms (CDAs). We define an objective performance measure, streaming power, and use it to explore the robustness of three different algorithms. The measure is comparable for disparate algorithms, and provides a common framework for comparing and evaluating change detection algorithms on any data set in a meaningful fashion. We demonstrate on real world applications, and on synthetic data.

In addition, we present a repository of data streams for the community to test change detection algorithms for streaming data.