Ranked Set Sampling as Data Reduction Tools

Abstract

Contrary to the traditional problem facing statisticians that sample sizes are small, the data size in data mining is tremendously huge. It is common in data mining to deal with data sets in gigabytes or even terabytes. It is simply impossible to store a whole data set of such size in the central memory of a computer. However, certain statistical procedures, for instance, the computation of a quantile, require the whole data set to be processed at the same time in the central memory. Therefore, data reduction becomes a necessary step in dealing with huge data sets for those procedures. A desirable data reduction procedure should discard those data with low information contents and retain the data with high information contents. The RSS is essentially a data selection procedure that selects only those data which have high information contents. Therefore the notion of RSS can well be applied for data reduction.