Accelerating pattern-based time series classification: a linear time and space string mining approach

Abstract

Subsequences-based time series classification algorithms provide interpretable and generally more accurate classification models compared to the nearest neighbor approach, albeit at a considerably higher computational cost. A number of discretized time series-based algorithms have been proposed to reduce the computational complexity of these algorithms; however, the asymptotic time complexity of the proposed algorithms is also cubic or higher-order polynomial. We present a remarkably fast and resource-efficient time series classification approach which employs a linear time and space string mining algorithm for extracting frequent patterns from discretized time series data. Compared to other subsequence or pattern-based classification algorithms, the proposed approach only requires a few parameters, which can be chosen arbitrarily and do not require any fine-tuning for different datasets. The time series data are discretized using symbolic aggregate approximation, and frequent patterns are extracted using a string mining algorithm. An independence test is used to select the most discriminative frequent patterns, which are subsequently used to create a transformed version of the time series data. Finally, a classification model can be trained using any off-the-shelf algorithm. Extensive empirical evaluations demonstrate the competitive classification accuracy of our approach compared to other state-of-the-art approaches. The experiments also show that our approach is at least one to two orders of magnitude faster than the existing pattern-based methods due to the extremely fast frequent pattern extraction, which is the most computationally intensive process in pattern-based time series classification approaches.

Appendix: Calculating independence test statistics

Section 3.3 provides the algorithmic details for selecting frequent patterns based on their discriminative power. $\chi ^2$ independence test or information gain values can be used to determine the discriminative power of a given pattern and find out how effectively it can identify the instances of a given class. This section explains the procedure of calculating these statistics based on the occurrence frequency of a pattern in the positive and negative class dataset splits. In this regard, the notation used for the following discussion is given below.

Symbol	Representation
$\widehat{P}$	Positive class dataset
$\widehat{N}$	Negative class dataset
$N_{\widehat{P}}$	Number of instances in $\widehat{P}$
$N_{\widehat{N}}$	Number of instances in $\widehat{N}$
p	Frequent pattern
$f_{\widehat{P}}$	Occurrence frequency of p in $\widehat{P}$
$f_{\widehat{N}}$	Occurrence frequency of p in $\widehat{N}$

1.1 Calculating the $\chi ^2$ test statistic

The $\chi ^2$ test statistic is calculated based on observed ($O_{ij}$) and expected ($E_{ij}$) values for the given categorical variables. The formula for calculating the $\chi ^2$ statistic is given below.

$$\begin{aligned} \chi ^2 = \sum \frac{(O_{ij} - E_{ij})^2}{E_{ij}} \end{aligned}$$

Observed values ($O_{ij}$) correspond to the number of instances observed as belonging to a certain categorical variable. In our case, it is the number of instances labeled as belonging to the positive or negative class given a particular frequent pattern. This can be determined using the instance counts of the positive and negative class datasets and occurrence frequency values of the given pattern in the respective dataset splits. Based on these values, a contingency table can be created as follows.

	Dataset splits
	Positive, $\widehat{P}$	Negative, $\widehat{N}$
$With (p)$	$O_{11}=\lfloor f_{\widehat{P}} \times N_{\widehat{P}} \rceil$	$O_{12}=\lfloor f_{\widehat{N}} \times N_{\widehat{N}} \rceil$	$RSum_{1.} = O_{11}+O_{12}$
$WithOut (p)$	$O_{21}=N_{\widehat{P}}-O_{11}$	$O_{22}=N_{\widehat{N}}-O_{12}$	$RSum_{2.} = O_{21}+O_{22}$
	$CSum_{.1}=O_{11}+O_{21}$	$CSum_{.2}=O_{12}+O_{22}$	$n=\sum _{i,j} O_{ij}$

The rows of this contingency table correspond to the number of instances containing or not containing the given pattern p, while the columns correspond to the positive and negative dataset splits, respectively. The combined total of row and column sums equals the total number of instances in the positive and negative dataset splits. Finally, the expected values ($E_{ij}$) are calculated using the following formula.

$$\begin{aligned} E_{ij} = \frac{RSum_{i.} \times CSum_{.j}}{n} \end{aligned}$$

The $\chi ^2$ test statistic determines whether any relationship between the positive and negative dataset splits exists given the frequent pattern. If the pattern occurs in both datasets, then the $\chi ^2$ value will be close to zero which signifies a relationship exists between the two dataset splits. We can order the frequent patterns based on their $\chi ^2$ statistic and select the ones for which the dataset splits do not exhibit any mutual relationship.

1.2 Calculating the information gain value

Entropy (H) is a measure for establishing whether a dataset has a uniform or varying distribution in terms of the different classes of instances. Given a dataset with positive and negative class instances, the entropy of the dataset can be calculated using the following formula.

$$\begin{aligned} H = -\Bigg (\frac{N_{\widehat{P}}}{N_{\widehat{P}}+N_{\widehat{N}}} \times \mathrm{log}_2 \frac{N_{\widehat{P}}}{N_{\widehat{P}}+N_{\widehat{N}}}\Bigg ) -\Bigg (\frac{N_{\widehat{N}}}{N_{\widehat{P}}+N_{\widehat{N}}} \times \mathrm{log}_2 \frac{N_{\widehat{N}}}{N_{\widehat{P}}+N_{\widehat{N}}}\Bigg ) \end{aligned}$$

If a pattern p occurs frequently in either class of instances in the dataset, we can create positive and negative class subsets based on the presence or absence of this pattern in each of the instances. The entropy of these subsets can then be calculated using the following equations.

$$\begin{aligned} H_{\widehat{P}}= & {} -\Bigg ( \frac{ f_{\widehat{P}} \times N_{\widehat{P}} }{ f_{\widehat{P}} \times N_{\widehat{P}} + f_{\widehat{N}} \times N_{\widehat{N}} } \times \mathrm{log}_2 \frac{ f_{\widehat{P}} \times N_{\widehat{P}} }{ f_{\widehat{P}} \times N_{\widehat{P}} + f_{\widehat{N}} \times N_{\widehat{N}} } \Bigg )\\&-\Bigg ( \frac{ f_{\widehat{N}} \times N_{\widehat{N}} }{ f_{\widehat{P}} \times N_{\widehat{P}} + f_{\widehat{N}} \times N_{\widehat{N}} } \times \mathrm{log}_2 \frac{ f_{\widehat{N}} \times N_{\widehat{N}} }{ f_{\widehat{P}} \times N_{\widehat{P}} + f_{\widehat{N}} \times N_{\widehat{N}} } \Bigg ) \\ H_{\widehat{N}}= & {} -\Bigg ( \frac{ (1-f_{\widehat{P}}) \times N_{\widehat{P}} }{ (1-f_{\widehat{P}}) \times N_{\widehat{P}} + (1-f_{\widehat{N}}) \times N_{\widehat{N}} } \times \mathrm{log}_2 \frac{ (1-f_{\widehat{P}}) \times N_{\widehat{P}} }{ (1-f_{\widehat{P}}) \times N_{\widehat{P}} + (1-f_{\widehat{N}}) \times N_{\widehat{N}} }\Bigg )\\&-\Bigg ( \frac{ (1-f_{\widehat{N}}) \times N_{\widehat{N}} }{ (1-f_{\widehat{P}}) \times N_{\widehat{P}} + (1-f_{\widehat{N}}) \times N_{\widehat{N}} } \times \mathrm{log}_2 \frac{ (1-f_{\widehat{N}}) \times N_{\widehat{N}} }{ (1-f_{\widehat{p}}) \times N_{\widehat{P}} + (1-f_{\widehat{N}}) \times N_{\widehat{N}} } \Bigg ) \end{aligned}$$

Using the entropy values of the source dataset and the positive and negative subsets, we can calculate the information gain value using the following formula.

$$\begin{aligned} IG = H - \Bigg ( \frac{f_{\widehat{P}} \times N_{\widehat{P}} + f_{\widehat{N}} \times N_{\widehat{N}}}{N_{\widehat{P}}+N_{\widehat{N}}} \times H_{\widehat{P}} + \frac{(1-f_{\widehat{P}}) \times N_{\widehat{P}} + (1-f_{\widehat{N}}) \times N_{\widehat{N}}}{N_{\widehat{P}}+N_{\widehat{N}}} \times H_{\widehat{N}} \Bigg ) \end{aligned}$$

If the frequent pattern effectively distinguishes between the two classes, the positive and negative class subsets will have very few or no instances of the other class resulting in a smaller value of entropy for the two subsets. This in turn will cause a higher information gain value indicating that the pattern is a good candidate for distinguishing between the two classes of instances. If, however, the converse is true, then the pattern is not a good candidate. This way the candidates can be selected on the basis of their discriminative power.