Sample Selection

Abstract

The databases available for model construction can be vast. Some customer databases contain tens of millions of records (observations) and thousands of predictor variables. Even with modern computing facilities it may not be practical to use all of the data available. There will also be cases that are not suitable for model construction, and these need to be identified and dealt with. The data used for model construction should also be as similar as possible to the data that will exist when the completed model is put into service — which usually means that the sample used to construct the model should be as recent as possible to mitigate against changes in the patterns of behaviour that accumulate over time. For these reasons it is common for models to be constructed using a sub-set (a sample) of the available data, rather than the full population.