Data Mining and Knowledge Discovery

, Volume 7, Issue 4, pp 349–371

On the Need for Time Series Data Mining Benchmarks: A Survey and Empirical Demonstration

  • Eamonn Keogh
  • Shruti Kasetty

DOI: 10.1023/A:1024988512476

Cite this article as:
Keogh, E. & Kasetty, S. Data Mining and Knowledge Discovery (2003) 7: 349. doi:10.1023/A:1024988512476


In the last decade there has been an explosion of interest in mining time series data. Literally hundreds of papers have introduced new algorithms to index, classify, cluster and segment time series. In this work we make the following claim. Much of this work has very little utility because the contribution made (speed in the case of indexing, accuracy in the case of classification and clustering, model accuracy in the case of segmentation) offer an amount of “improvement” that would have been completely dwarfed by the variance that would have been observed by testing on many real world datasets, or the variance that would have been observed by changing minor (unstated) implementation details.

To illustrate our point, we have undertaken the most exhaustive set of time series experiments ever attempted, re-implementing the contribution of more than two dozen papers, and testing them on 50 real world, highly diverse datasets. Our empirical results strongly support our assertion, and suggest the need for a set of time series benchmarks and more careful empirical evaluation in the data mining community.

time series data mining experimental evaluation 

Copyright information

© Kluwer Academic Publishers 2003

Authors and Affiliations

  • Eamonn Keogh
    • 1
  • Shruti Kasetty
    • 1
  1. 1.University of CaliforniaRiverside

Personalised recommendations