Data mining pipeline; Data mining process; KDD process
The KDD pipeline describes the complete process of knowledge discovery in databases (KDD), i.e. the process of deriving useful, valid and non-trivial patterns from a large amount of data. The pipeline consists of five consecutive steps:
The selection step identifies the goal of the current application and selects a data set that is likely to contain relevant patterns.
The preprocessing step increases the quality of the data set by supplementing missing attributes, removing duplicate instances and resolving data inconsistencies.
The transformation step deletes correlated and irrelevant attributes and derives new more meaningful attributes from the current data description.
This step selects a data mining algorithm with respect to the goal which was identified in the selection step and derives patterns or learns functions that are valid for the current data set.
- 1.Brachman R, Anand T. The process of knowledge discovery in databases: a human centered approach. In: Proceedings of the 10th National Conference on Artificial Intelligence; 1996. p. 37–8.Google Scholar
- 2.Fayyad U, Piatetsky-Shapiro G, Smyth P. From data mining to knowledge discovery in databases. In: Proceedings of the 10th National Conference on Artificial Intelligence; 1996. p. 1–30.Google Scholar
- 3.Fayyad U, Piatetsky-Shapiro G, Smyth P. Knowledge discovery and data mining: towards a unifying framework. In: Proceedings of the 2nd Internatinal Conference on Knowledge Discovery and Data Mining; 1996. p. 82–8.Google Scholar