A clustering-based feature selection method for automatically generated relational attributes

Abstract

Although data mining problems require a flat mining table as input, in many real-world applications analysts are interested in finding patterns in a relational database. To this end, new methods and software have been recently developed that automatically add attributes (or features) to a target table of a relational database which summarize information from all other tables. When attributes are automatically constructed by these methods, selecting the important attributes is particularly difficult, because a large number of the attributes are highly correlated. In this setting, attribute selection techniques such as the Least Absolute Shrinkage and Selection Operator (lasso), elastic net, and other machine learning methods tend to under-perform. In this paper, we introduce a novel attribute selection procedure, where after an initial screening step, we cluster the attributes into different groups and apply the group lasso to select both the true attributes groups and then the true attributes. The procedure is particularly suited to high dimensional data sets where the attributes are highly correlated. We test our procedure on several simulated data sets and a real-world data set from a marketing database. The results show that our proposed procedure obtains a higher predictive performance while selecting a much smaller set of attributes when compared to other state-of-the-art methods.

Appendices

Appendix A: Attribute generation with Dataconda

Dataconda is a software, freely available online, that generates attributes in a relational database. The user chooses a target table, to which Dataconda automatically adds new attributes that contain information contained in the rest of the database. In the example of Fig. 1, the target table is Purchases.

Here, we briefly illustrate how Dataconda generates attributes; the details are reported in Samorani et al. (2016). An attribute is built in two steps. In the first step, the procedure generates a large number of paths that start from the target table (and end anywhere).

In the second step, many attributes are generated for each path. The procedure generates attributes by virtually adding attributes to each table of the path, starting from the second to last table of the path and finishing with the first table of the path; each virtual attribute has the purpose of summarizing the information contained in the tables that follow along the path. When the procedure reaches the target table, the algorithm has constructed an attribute which summarizes the information contained in the path.

Suppose that the path built in the first step is Purchases P1 $\rightarrow$ Clients C $\rightarrow$ Purchases P2. The second step starts by virtually adding to table C an attribute that summarizes P2. Since the relationship linking C to P2 is 1-to-n (i.e., one client may have many purchases), the attribute virtually added to C is obtained by summarizing the purchases made by each client. Examples of virtual attributes that can be added to C the “number of purchases” made by each client, or “the average value of the attribute Return among the purchases of each client”, or the “number of purchases of where Online = 1 made by each client”. It is clear that there are many choices in building attributes, both in terms of aggregate functions to use (average, sum, count, etc) and in terms of “where clauses” to use (where Online = 1, where Online = 0, where Return = 1, etc). Dataconda allows the user to decide which aggregate function and “where clauses” to adopt for each attribute. After virtually adding an attribute to table C, the algorithm proceeds by adding to table P1, the target table, an attribute from table C that summarizes the rest of the path. Since the relationship linking P1 to C is 1-to-1 (i.e., one purchase has exactly one client), this attribute could simply be the virtual attribute added tot able C. In this way, we could add to the target table attributes such as “number of purchases made by the same client prior to the current purchase, or “the average value of the attribute Return among the purchases of the same client made prior to the current purchase”, or the “number of purchases of where $\hbox {Online}=1$ made by the same client prior to the current purchase”.

Appendix B: Results for the six combinations with Shrinkage Method = elastic net on the simulated data set

See Tables 8 and 9.

Table 8 Results obtained with Shrinkage Method = elastic net—part 1 of 2—our proposed method is in bold

Table 9 Results obtained with Shrinkage Method = elastic net—part 2 of 2—our proposed method is in bold