An Output-Sensitive Flexible Pattern Discovery Algorithm

Abstract

Given an input sequence of data, a motif is a repeating pat- tern, possibly interspersed with “dont care” characters and a flexible motif could have a variable (as opposed to fixed) number of “dont care” characters. Given a sequence of records with F fields each, an association rule is a common set of f fields, f ≤ F, with identical (or similar) re- peating values. The data in either case could be a sequence of characters or sets of characters or even real values. It is well known that the number of motifs or association rules, say N, could potentially be exponential in the size of the input sequence or number of records, say n. In this paper we present a new algorithm to discover all flexible motifs or association rules in the input. A novel feature of this algorithm is that its running time is linear in the size of the output (ignoring polylog factors). More precisely, the complexity of the algorithm is O(n ⁵ + N) log n). This is the first algorithm for motif discovery with a proven output sensitive complexity bound. The discovery algorithm works in two phases: in the first phase it detects a linear number of core motifs in time polynomial in the input size n and in the second phase it detects all the remaining motifs N′ in O(N′ log n) time. The core motifs of the first phase are also characterized as being those of “highest specificity”: loosely speaking, a pattern with higher specificity has less “dont care” characters. Some applications (for instance the ones that require the study of those por- tions of the input sequence that contribute to the non-gapped regions of motifs ) require only the core motifs. Hence for such applications, the first phase of the algorithm suffices. However, the general problem is of use in motif discovery tasks in gene or protein sequences, or discovery of association rules from gene expression data or in data mining.