Interactive Discovery of Statistically Significant Itemsets

  • Philippe Fournier-VigerEmail author
  • Xiang Li
  • Jie Yao
  • Jerry Chun-Wei Lin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10868)


Frequent Itemset Mining (FIM) is a fundamental data mining task, which consists of finding frequent sets of items in transaction databases. However, traditional FIM algorithms can find lot of spurious patterns. To address this issue, the OPUS-Miner algorithm was proposed to find statistically significant patterns, called productive itemsets. Though, this algorithm is useful, it cannot be used for interactive data mining, that is the user cannot guide the search toward items of interest using queries, and the database is assumed to be static. This paper addresses this issue by proposing a novel approach to process targeted queries to check if some itemsets of interest to the user are non redundant and productive. The approach relies on a novel structure called Query-Tree to efficiently process queries. An experimental evaluation on several datasets of various types shows that thousands of queries are processed per second on a desktop computer, making it suitable for interactive data mining, and that it is up to 22 times faster than a baseline approach.


Itemset mining Productive itemsets Query-Tree Pattern 


  1. 1.
    Han, J., Pei, J., Kamber, M.: Data Mining: Concepts and Techniques. Elsevier, Waltham (2011)zbMATHGoogle Scholar
  2. 2.
    Webb, G.I., Vreeken, J.: Efficient discovery of the most interesting associations. ACM Trans. Knowl. Discov. Data 8(3), 15 (2014)Google Scholar
  3. 3.
    Kubat, M., Hafez, A., Raghavan, V.V., Lekkala, J.R., Chen, W.K.: Itemset trees for targeted association querying. IEEE Trans. Knowl. Data Eng. 15(6), 1522–1534 (2003)CrossRefGoogle Scholar
  4. 4.
    Lavergne, J., Benton, R., Raghavan, V.V.: Min-max itemset trees for dense and categorical datasets. In: Chen, L., Felfernig, A., Liu, J., Raś, Z.W. (eds.) ISMIS 2012. LNCS (LNAI), vol. 7661, pp. 51–60. Springer, Heidelberg (2012). Scholar
  5. 5.
    Fournier-Viger, P., Mwamikazi, E., Gueniche, T., Faghihi, U.: MEIT: memory efficient itemset tree for targeted association rule mining. In: Motoda, H., Wu, Z., Cao, L., Zaiane, O., Yao, M., Wang, W. (eds.) ADMA 2013, Part II. LNCS (LNAI), vol. 8347, pp. 95–106. Springer, Heidelberg (2013). Scholar
  6. 6.
    Agrawal, R., Srikant, R.: Fast algorithms for mining association rules in large databases. In: Proceedings of 20th International Conference on Very Large Databases, pp. 487–499. Morgan Kaufmann, Santiago de Chile (1994)Google Scholar
  7. 7.
    Llinares-López, F., Sugiyama, M., Papaxanthos, L., Borgwardt, K.: Fast and memory-efficient significant pattern mining via permutation testing. In: Proceedings of 21th ACM International Conference on Knowledgs Discovery and Data Mining, pp. 725–734. ACM (2015)Google Scholar
  8. 8.
    Fournier-Viger, P., Lin, J.C.-W., Vo, B., Chi, T.T., Zhang, J., Le, H.B.: A survey of itemset mining. WIREs Data Mining Knowl. Discov. 7(4), e1207 (2017). Scholar
  9. 9.
    Nofong, V.M.: Discovering productive periodic frequent patterns in transactional databases. Ann. Data Sci. 3(3), 235–249 (2016)CrossRefGoogle Scholar
  10. 10.
    Petitjean, F., Li, T., Tatti, N., Webb, G.I.: Skopus: mining top-k sequential patterns under leverage. Data Mining Knowl. Discov. 30(5), 1086–1111 (2016)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Fournier-Viger, P., Wu, C.-W., Tseng, V.S.: Novel concise representations of high utility itemsets using generator patterns. In: Luo, X., Yu, J.X., Li, Z. (eds.) ADMA 2014. LNCS (LNAI), vol. 8933, pp. 30–43. Springer, Cham (2014). Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Philippe Fournier-Viger
    • 1
    Email author
  • Xiang Li
    • 2
  • Jie Yao
    • 1
  • Jerry Chun-Wei Lin
    • 2
  1. 1.School of Humanities and Social SciencesHarbin Institute of Technology (Shenzhen)ShenzhenChina
  2. 2.School of Computer Science and TechnologyHarbin Institute of Technology (Shenzhen)ShenzhenChina

Personalised recommendations