Combining Pattern Discovery and Probabilistic Modeling in Data Mining

Abstract

Data mining has in recent years emerged as an interesting area in the boundary between algorithms, probabilistic modeling, statistics, and databases. Data mining research has come from two different traditions. The global approach aims at modeling the joint distribution of the data, while the local approach aims at efficient discovery of frequent patterns from the data. Among the global modeling techniques, mixture models have emerged as a strong unifying theme, and methods exist for fitting such models on large data sets. For pattern discovery, the methods for finding frequently occurring positive conjunctions have been applied in various domains. An interesting open issue is how to combine the two approaches, e.g., by inferring joint distributions from pattern frequencies. Some promising results have been achieved using maximum entropy approaches. In the talk we describe some basic techniques in global and local approaches to data mining, and present a selection of open problems.