A formal series-based unification of the frequent itemset mining approaches

Abstract

Over the last two decades, a great deal of work has been devoted to the algorithmic aspects of the frequent itemset (FI) mining problem, leading to a phenomenal number of algorithms and associated implementations, each of which claims supremacy. Meanwhile, it is generally well agreed that developing a unifying theory is one of the most important issues in data mining research. Hence, our primary motivation for this work is to introduce a high-level formalism for this basic problem, which induces a unified vision of the algorithmic approaches presented so far. The key distinctive feature of the introduced model is that it combines, in one fashion, both the qualitative and the quantitative aspects of this basic problem. In this paper, we propose a new model for the FI-mining task based on formal series. In fact, we encode the itemsets as words over a sorted alphabet and express this problem by a formal series over the counting semiring $(\mathbb N,+,\times ,0,1)$, whose range represents the itemsets, and the coefficients are their supports. The aim is threefold: First, to define a clear, unified and extensible theoretical framework through which we can state the main FI-approaches. Second, to prove a convenient connection between the determinization of the acyclic weighted automaton that represents a transaction dataset and the computation of the associated collection of FI. Finally, to devise a first algorithmic transcription, baptized Wafi, of our model by means of weighted automata, which we evaluate against representative leading algorithms. The obtained results show the suitability of our formalism.

Appendix

1.1 Proof of Proposition 3

Proof

We construct the automaton $\mathscr {C}$ by determinizing both automata $\mathscr {A}$ and $\mathscr {B}$ seen as one with two start states, that is, determinizing the automaton $\mathscr {A} \cup \mathscr {B}$.

Let $\mathscr {A}=(Q_A,A_1,\mu _A,\lambda _A,\gamma _A)$ be a PWA isomorphic, via $h_A$, to the automaton $\mathcal{P}_{X} = (Q_X,A_X,\mu _X,\lambda _X,\gamma _X)$ which realizes the polynomial $\mathop {\mathbb {P}_{X}}$ associated with the dataset X and $\mathscr {B}=(Q_B,A_2,\mu _B,\lambda _B,\gamma _B)$ the one isomorphic, via $h_B$, to the automaton $\mathcal{P}_{Y} =(Q_Y,A_Y,\mu _Y,\lambda _Y,\gamma _Y)$ which realizes the polynomial $\mathop {\mathbb {P}_{Y}}$ associated with the dataset Y.

We give below firstly the construction of the automaton $\mathscr {C}$ and then a mapping h from the set of states $Q_C$ to the set of states of the automaton $\mathcal{P}_{X\cup Y}$ which is the range of the polynomial $\mathop {\mathbb {P}_{X\cup Y}}$.

We define $\mathscr {C}=(Q_C,A_1\cup A_2,\mu _C,\lambda _C,\gamma _C)$ the prefixial weighted automaton as follows ($q_A$ and $q_B$ are elements from $Q_A$ and $Q_B$, respectively):

• 1. $Q_C = T\cup W\cup Z,$ where:

  • $T=\{ \{q_A,q_B\} \mid h_A(p) = h_B(q)\}$,

  • $W=\{\{q_A\} \mid h_A(q_A) \in h_A(Q_A)\setminus h_B(Q_B)\}$,

  • $Z= \{\{q_B\} \mid h_B(q_B) \in h_B(Q_B)\setminus h_A(Q_A)\}.$

• 2. $\mu _C(\{q_A,q_B\})={\left\{ \begin{array}{ll} \begin{array}{ll} 1 &{} \text{ for } \text{ the } \text{ start } \text{ state } ({q_A}_0,{q_B}_0) \text{ obtained } \text{ by } \text{ pairing } \text{ those } \text{ of } \mathscr {A} \text{ and } \mathscr {B},\\ 0 &{} \text{ otherwise }. \end{array} \end{array}\right. }$

• 3. $\lambda _C(q,a,q\prime ) \text{ is } \text{ a } \text{ binary } \text{ matrix, } \text{ for } q \text{ and } q\prime \in Q_C \text{ and } a\in A_1\cup A_2$, that is, the weight of each transition is either 0 or 1. We have five cases according to the subsets T, W or Z to which belong q and $q\prime$:

  • $q \in T$ and $q\prime \in T$: $\lambda _C(\{q_A,q_B\},a,\{q\prime _A,q\prime _B\}) = 1$, if both $\lambda _A(q_A,a,q\prime _A)$ and $\lambda _B(q_B,a,q\prime _B)$ are defined,

  • $q \in T$ and $q\prime \in W$: $\lambda _C(\{q_A,q_B\},a,\{q\prime _A\}) = 1$, if only $\lambda _A(q_A,a,q\prime _A)$ is defined,

  • $q \in T$ and $q\prime \in Z$: $\lambda _C(\{q_A,q_B\},a,\{q\prime _B\}) = 1$, if only $\lambda _B(q_B,a,q\prime _B)$ is defined,

  • $q\in W$: if defined $q\prime$ can only belong to W (closure of prefixial sets): $\lambda _C(\{q_A\},a,\{q\prime _A\}) = 1$ if $\lambda _A(q_A,a,q\prime _A)$ exists,

  • $q\in Z$: for the same reason, if defined $q\prime$ can only belong to Z: $\lambda _C(\{q_B\},a,\{q\prime _B\}) = 1$ if $\lambda _B(q_B,a,q\prime _B)$ exists,

  In order to simplify the proof, let $\Delta _C$ symbolizes the weight function $\lambda _C$, and let also $\delta _A$, $\delta _B$, $\delta _X$ and $\delta _Y$ denote, respectively, the functions $\lambda _A$, $\lambda _B$, $\lambda _X$ and $\lambda _Y$, and we can recapitulate the function $\lambda _C$ in the following cases: $\Delta _C(q,a)={\left\{ \begin{array}{ll} \begin{array}{lll} \{q\prime _A,q\prime _B\}\quad &{} \text{ if } \quad &{} q=\{q_A,q_B\}\in T \text{ and } \delta _A(q_A,a)=q\prime _A\text{, } \text{ and } \delta _B(q_B,a) =q\prime _B,\\ \{q\prime _A\}\quad &{} \text{ if } \quad &{} {\left\{ \begin{array}{ll} \begin{array}{ll} q=\{q_A,q_B\}\in T \quad &{} \text{ and } \delta _A(q_A,a)=q\prime _A\text{, } \text{ and } \delta _B(q_B,a) = \emptyset ,\\ q=\{q_A\}\in W \quad &{} \text{ and } \delta _A(q_A,a)=q\prime _A. \end{array} \end{array}\right. }\\ \{q\prime _B\}\quad &{} \text{ if } &{} {\left\{ \begin{array}{ll} \begin{array}{ll} q=\{q_A,q_B\}\in T \quad &{} \text{ and } \delta _B(q_B,a)=q\prime _B\text{, } \text{ and } \delta _A(q_A,a) = \emptyset ,\\ q=\{q_B\}\in Z \quad &{} \text{ and } \delta _B(q_B,a)=q\prime _B. \end{array} \end{array}\right. }\\ \end{array} \end{array}\right. }$

• 4. $\gamma _C(I)={\left\{ \begin{array}{ll} \begin{array}{lll} \gamma _A(q_A)+\gamma _B(q_B)\quad &{} \text{ if } \quad &{} I=\{q_A,q_B\} \in T ,\\ \gamma _A(q_A)\quad &{} \text{ if } \quad &{} I=\{q_A\}\in W ,\\ \gamma _B(q_B)\quad &{} \text{ if } \quad &{} I=\{q_B\} \in Z. \end{array} \end{array}\right. }$

Define also the mapping h from $Q_C$ to $\mathrm{range}(\mathop {\mathbb {P}_{X\cup Y}})$ as follows:

$$\begin{aligned} h(I)= {\left\{ \begin{array}{ll} \begin{array}{lll} h_A(\{q_A\})\quad &{} \text{ if } \quad &{} I=\{q_A,q_B\} \in T \text{, } \text{ or } I=\{q_A\} \in W,\\ h_B(\{q_B\})\quad &{} \text{ if } \quad &{} I =\{q_B\} \in Z. \end{array} \end{array}\right. } \end{aligned}$$

Now, we must verify that h defines well an automata isomorphism.

Let I be a state of $Q_C$:

• 1. It is clear from the definition of the vector $\mu _C$ that the start state of $\mathscr {C}$ is mapped by h to the start state of $\mathrm{range}(\mathop {\mathbb {P}_{X\cup Y}})$ which is the pair $(\varepsilon ,\varepsilon )$

• 2. $h(\Delta _C(q,a))={\left\{ \begin{array}{ll} \begin{array}{lll} h(\{q\prime _A,q\prime _B\})\quad &{} \text{ if } \quad &{} q=\{q_A,q_B\}\in T \text{ and } \delta _A(q_A,a)=q\prime _A\text{, } \text{ and } \delta _B(q_B,a) =q\prime _B,\\ h(\{q\prime _A\})\quad &{} \text{ if } &{} {\left\{ \begin{array}{ll} \begin{array}{ll} q=\{q_A,q_B\}\in T \quad &{} \text{ and } \delta _A(q_A,a)=q\prime _A\text{, } \text{ and } \delta _B(q_B,a) = \emptyset ,\\ q=\{q_A\}\in W \quad &{} \text{ and } \delta _A(q_A,a)=q\prime _A. \end{array} \end{array}\right. }\\ h(\{q\prime _B\})\quad &{} \text{ if } \quad &{} {\left\{ \begin{array}{ll} \begin{array}{ll} q=\{q_A,q_B\}\in T \quad &{} \text{ and } \delta _B(q_B,a)=q\prime _B\text{, } \text{ and } \delta _A(q_A,a) = \emptyset ,\\ q=\{q_B\}\in Z \quad &{} \text{ and } \delta _B(q_B,a)=q\prime _B. \end{array} \end{array}\right. }\\ \end{array} \end{array}\right. }$ According to the definition of the mapping h, we obtain:

  $$\begin{aligned}&h(\Delta _C(q,a))\\&\quad ={\left\{ \begin{array}{ll} \begin{array}{lll} h_A(\{q\prime _A\})\quad &{} \text{ if } \quad &{} q=\{q_A,q_B\}\in T \text{ and } \delta _A(q_A,a)=q\prime _A\text{, } \text{ and } \delta _B(q_B,a) =q\prime _B,\\ h_A(\{q\prime _A\})\quad &{} \text{ if } \quad &{} {\left\{ \begin{array}{ll} \begin{array}{ll} q=\{q_A,q_B\}\in T \quad &{} \text{ and } \delta _A(q_A,a)=q\prime _A\text{, } \text{ and } \delta _B(q_B,a) = \emptyset ,\\ q=\{q_A\}\in W \quad &{} \text{ and } \delta _A(q_A,a)=q\prime _A. \end{array} \end{array}\right. }\\ h_B(\{q\prime _B\})\quad &{} \text{ if } \quad &{} {\left\{ \begin{array}{ll} \begin{array}{ll} q=\{q_A,q_B\}\in T \quad &{} \text{ and } \delta _B(q_B,a)=q\prime _B\text{, } \text{ and } \delta _A(q_A,a) = \emptyset ,\\ q=\{q_B\}\in Z \quad &{} \text{ and } \delta _B(q_B,a)=q\prime _B 1. \end{array} \end{array}\right. }\\ \end{array} \end{array}\right. }\\&\quad ={\left\{ \begin{array}{ll} \begin{array}{lll} h_A(\delta _A(q_A,a))\quad &{} \text{ if } \quad &{} q=\{q_A,q_B\}\in T \text{ and } \delta _A(q_A,a)=q\prime _A\text{, } \text{ and } \delta _B(q_B,a) =q\prime _B,\\ h_A(\delta _A(q_A,a))\quad &{} \text{ if } \quad &{} {\left\{ \begin{array}{ll} \begin{array}{ll} q=\{q_A,q_B\}\in T \quad &{} \text{ and } \delta _A(q_A,a)=q\prime _A\text{, } \text{ and } \delta _B(q_B,a) = \emptyset ,\\ q=\{q_A\}\in W \quad &{} \text{ and } \delta _A(q_A,a)=q\prime _A. \end{array} \end{array}\right. }\\ h_B(\delta _B(q_B,a))\quad &{} \text{ if } \quad &{} {\left\{ \begin{array}{ll} \begin{array}{ll} q=\{q_A,q_B\}\in T \quad &{} \text{ and } \delta _B(q_B,a)=q\prime _B\text{, } \text{ and } \delta _A(q_A,a) = \emptyset ,\\ q=\{q_B\}\in Z \quad &{} \text{ and } \delta _B(q_B,a)=q\prime _B. \end{array} \end{array}\right. }\\ \end{array} \end{array}\right. } \end{aligned}$$

  Since both $h_A$ and $h_B$ are weighted automata isomorphisms:

  $$\begin{aligned}&h(\Delta _C(q,a))\\&\quad ={\left\{ \begin{array}{ll} \begin{array}{lll} \delta _X(h_A(\{q_A\}),a)\quad &{} \text{ if } \quad &{} q=\{q_A,q_B\}\in T \text{ and } \delta _A(q_A,a)=q\prime _A\text{, } \text{ and } \delta _B(q_B,a) =q\prime _B,\\ \delta _X(h_A(\{q_A\}),a)\quad &{} \text{ if } \quad &{} {\left\{ \begin{array}{ll} \begin{array}{ll} q=\{q_A,q_B\}\in T \quad &{} \text{ and } \delta _A(q_A,a)=q\prime _A\text{, } \text{ and } \delta _B(q_B,a) = \emptyset ,\\ q=\{q_A\}\in W \quad &{} \text{ and } \delta _A(q_A,a)=q\prime _A. \end{array} \end{array}\right. }\\ \delta _Y(h_B(\{q_B\}),a)\quad &{} \text{ if } &{} {\left\{ \begin{array}{ll} \begin{array}{ll} q=\{q_A,q_B\}\in T \quad &{} \text{ and } \delta _B(q_B,a)=q\prime _B\text{, } \text{ and } \delta _A(q_A,a) = \emptyset ,\\ q=\{q_B\}\in Z \quad &{} \text{ and } \delta _B(q_B,a)=q\prime _B. \end{array} \end{array}\right. }\\ \end{array} \end{array}\right. } \end{aligned}$$

  According to our defined mapping h, and since all our weight functions are binary we obtain for each case:  $h(\Delta _C(q,a))=\delta _{X\cup Y}(h(q),a)$

• 3. Now, we consider the output weight property, for only the first case (the other cases are simple to prove). Let $I\in Q_C$. When $I\in T$, we have:

  $$\begin{aligned} \gamma _C(I)= & {} \gamma _C(\{q_A,q_B\}) \\= & {} \gamma _A(q_A)+\gamma _B(q_B) \\= & {} \gamma _X(h_A(\{q_A\}))+\gamma _Y(h_B(\{q_B\})) \text{ since } h_A\text{, } \text{ and } h_B \text{ are } \text{ isomorphisms } \\= & {} \langle \mathop {\mathbb {P}_{X}},h_A(\{q_A\}) \rangle \!+ \!\langle \mathop {\mathbb {P}_{Y}},h_B(\{q_B\}) \!\rangle \text{: } \text{ by } \text{ the } \text{ definition } \text{ of } \text{ the } \text{ functions } \gamma _X \text{ and } \gamma _Y\\= & {} \langle \mathop {\mathbb {P}_{X}},h_A(\{q_A\}) \rangle + \langle \mathop {\mathbb {P}_{Y}},h_A(\{q_A\}) \rangle \text{: } \text{ because } I \in T\\= & {} \langle \mathop {\mathbb {P}_{X}}+\mathop {\mathbb {P}_{Y}},h_A(\{q_A\}) \rangle \\= & {} \langle \mathop {\mathbb {P}_{X\cup Y}},h_A(\{q_A\}) \rangle \\= & {} \langle \mathop {\mathbb {P}_{X\cup Y}},h(\{q_A,q_B\}) \rangle \\= & {} \langle \mathop {\mathbb {P}_{X\cup Y}},h(I) \rangle \\= & {} \gamma _{X\cup Y}(h(I)). \end{aligned}$$

• 4. Finally, it is not hard to see that h is bijective since it is derived from two weighted automata isomorphisms. The definition of h involves $h_A$ or $h_B$ which are both bijective.

$\square$

1.2 Proof of Proposition 5

Proof

Let us start by checking that Proposition 5 is true for one transaction $t_i$ taken from the dataset D of n transactions. So, let $t_i=a_{i_1}a_{i_2}\ldots a_{i_k}$ be a k-itemset. According to the definitions in Sects. 3 and 4, and the convention $\overline{a_i}=a_i+1$, we have:

$$\begin{aligned} \mathop {\mathbb {P}_{t_i}}= & {} 1+a_{i_1}+a_{i_1}a_{i_2}+\ldots +a_{i_1}a_{i_2}a_{i_3} \ldots a_{i_k}\\ \text{ So, } \quad \overline{\mathop {\mathbb {P}_{t_i}}}= & {} 1+a_{i_1}+\overline{a_{i_1}}a_{i_2}+\ldots +\overline{a_{i_1}a_{i_2} \ldots a_{i_{k-1}}}a_{i_k}\\ \quad \text{ Since } \quad&\overline{a_i}=&1+a_i, \text{ we } \text{ can } \text{ write: } \\ \overline{\mathop {\mathbb {P}_{t_i}}}= & {} \overline{a_{i_1}}+\overline{a_{i_1}}a_{i_2}+\ldots +\overline{a_{i_1}a_{i_2}a_{i_3} \ldots a_{i_{k-1}}}a_{i_k} \\= & {} \overline{a_{i_1}}(1+a_{i_2})+\ldots +\overline{a_{i_1}a_{i_2}a_{i_3} \ldots a_{i_{k-1}}}a_{i_k} \\= & {} \overline{a_{i_1}a_{i_2}}+\ldots +\overline{a_{i_1}a_{i_2}a_{i_3} \ldots a_{i_{k-1}}}a_{i_k} \\= & {} \overline{a_{i_1}a_{i_2}}(1+a_{i_3})+\ldots +\overline{a_{i_1}a_{i_2}a_{i_3} \ldots a_{i_{k-1}}}a_{i_k} \\&\ldots&\\= & {} \overline{{a_{i_1}a_{i_2}a_{i_3} \ldots a_{i_{k-1}}a_{i_k}}}\\= & {} \mathop {\mathbb {S}_{t_i}} \end{aligned}$$

Now let us verify also the equality between the sum of the prefixial-bar polynomials and the prefixial-bar polynomial of the whole dataset D.

$$\begin{aligned} \overline{\mathop {\mathbb {P}_{t_i}}}= & {} \langle \mathop {\mathbb {P}_{t_i}}, \varepsilon \rangle + \sum _{ \mathop {a \in A}\limits ^{u \in A^*}} \langle \mathop {\mathbb {P}_{t_i}}, ua \rangle \overline{u}a\\ \sum _{i = 1}^{n}\overline{\mathop {\mathbb {P}_{t_i}}}= & {} \sum _{i = 1}^{n} (\langle \mathop {\mathbb {P}_{t_i}}, \varepsilon \rangle + \sum _{ \mathop {a \in A}\limits ^{u \in A^*}} \langle \mathop {\mathbb {P}_{t_i}}, ua \rangle \overline{u}a)\\ \sum _{i = 1}^{n}\overline{\mathop {\mathbb {P}_{t_i}}}= & {} \sum _{i = 1}^{n} \langle \mathop {\mathbb {P}_{t_i}}, \varepsilon \rangle +\sum _{i=1}^{n} \sum _{\mathop {a \in A}\limits ^{u \in A^*}}\langle \mathop {\mathbb {P}_{t_i}}, ua \rangle \overline{u}a\\= & {} \sum _{i = 1}^{n} \langle \mathop {\mathbb {P}_{t_i}}, \varepsilon \rangle +\sum _{\mathop {a \in A}\limits ^{u \in A^*}} \sum _{i=1}^{n} \langle \mathop {\mathbb {P}_{t_i}}, ua \rangle \overline{u}a\\= & {} \langle \mathop {\mathbb {P}_{D}}, \varepsilon \rangle +\sum _{\mathop {a \in A}\limits ^{u \in A^*}} \langle \mathop {\mathbb {P}_{D}}, ua \rangle \overline{u}a\\= & {} \overline{\mathop {\mathbb {P}_{D}}} \end{aligned}$$

We have found that: $\overline{\mathop {\mathbb {P}_{t_i}}} = \mathop {\mathbb {S}_{t_i}} \text{, } \text{ so } \displaystyle \sum _{i=1}^{n}\overline{\mathop {\mathbb {P}_{t_i}}} = \sum _{i = 1}^{n}\mathop {\mathbb {S}_{t_i}},\, \mathrm{which \ leads \ to } \; \overline{\mathop {\mathbb {P}_{D}}} = \mathop {\mathbb {S}_{D}}$. $\square$