Overview of Text Mining

Abstract

Text mining and data mining are contrasted relative to automated prediction. Models are constructed by training on samples of unstructured documents, and results are projected to new text. A standard data format for input to prediction methods is described. The key objective of data preparation is to transform text into a numerical format, eventually sharing a common representation with numerical data mining. Different text-mining tasks are introduced that fit within a predictive framework for machine-learning. These include document classification, information retrieval, clustering of documents, information extraction, and performance evaluation.