Using Genetic Algorithms for Parameter Optimization in Building Predictive Data Mining Models

Abstract

We present an application of genetic algorithms to search the space of model building parameters for optimizing the score function or accuracy of a predictive data mining model. The goal of predictive modeling is to build a classification or regression model that can accurately predict the value of a target column by observing the values of the input attributes. The process of finding an optimal algorithm and its control parameters for building a predictive model is a non-trivial process because of two reasons. The first reason is that the number of classification algorithms and its control parameters are very large. The second reason is that it can be quite time consuming to build a model for datasets containing a large number of records and attributes. These two reasons makes it impractical to enumerate through every algorithm and its possible control parameters for finding an optimal model. Genetic Algorithms are adaptive heuristic search algorithm and have been successfully applied to solve optimization problems in diverse domains. In this work, we formulate the problem of finding optimal predictive model building parameter as an optimization problem and examine the usefulness of genetic algorithms. We perform experiments on several datasets and report empirical results to show the applicability of genetic algorithms to the problem of finding optimal predictive model building parameters.