Advertisement

Map-Reduce Based Generic Basis of Association Rules Mining from Big Bata

  • Marwa BouraouiEmail author
  • Ines Bouzouita
  • Amel Grissa Touzi
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1075)

Abstract

Mining big data poses computational and memory challenges because of the astonishing rate of data generation when addressed by traditional mining methods. To deal with such problems we can take advantage of parallel programming such as MapReduce which permits parallel processing in massively distributed environment. In this paper, we address the issue of mining association rules from big datasets in such environments. For this, we introduce two contributions. The first one consists on exploiting irreducible paradigm for attributes reduction. The second one is to introduce a new generic parallel algorithm called DGARM for mining generic association rules from big data. We carried out exhaustive experiments over real world datasets to illustrate the efficiency of DGARM for large real world datasets.

Keywords

Generic association rules Distributed information systems Big data MapReduce Hadoop Irreducible elements 

References

  1. 1.
    Kryszkiewicz, J.: Concise representations of association rules. In: Proceedings of Exploratory Workshop on Pattern Detection and Discovery in Data Mining (ESF), LNAI, vol. 2447, pp. 92–109. Springer, London, UK (2002)Google Scholar
  2. 2.
    Han, J., Pei, J., Yin, Y.: Mining frequent patterns without candidate generation. In: ACM SIGMOD International Conference on Management of Data, no. 29, pp. 1–12 (2000)CrossRefGoogle Scholar
  3. 3.
    Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. Commun. ACM 1(51), 107–113 (2008)CrossRefGoogle Scholar
  4. 4.
    Riondato, M., DeBrabant, J.A., Fonseca, R., Upfal, E.: PARMA: “a parallel randomized algorithm for approximate association rules mining in MapReduce. In: Proceedings of the 21st ACM International Conference on Information and Knowledge Management, pp. 85–94 (2012)Google Scholar
  5. 5.
    Lin, K., Chung, S.-H.: A fast and resource efficient mining algorithm for discovering frequent patterns in distributed computing environments. Future Gener. Comput. Syst. 52, 49–58 (2015)CrossRefGoogle Scholar
  6. 6.
    Asha, P., Srinivasan, S.: Distributed association rule mining with load balancing in grid environment. J. Comput. Theor. Nanosci. 13(1), 33–42 (2016)CrossRefGoogle Scholar
  7. 7.
    Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: the IEEE 26th Symposium on Mass Storage, pp. 1–10 (2010)Google Scholar
  8. 8.
    Moens, S., Aksehirli, E., Goethals, B.: Frequent itemset mining for big data. In: IEEE International Conference on in Big Data, pp. 111–118 (2013)Google Scholar
  9. 9.
    Kovacs, F., Illes, J.: Frequent itemset mining on hadoop. In: Proceedings of IEEE 9th International Conference on Computational Cybernetics (ICCC), pp. 241–245 (2013)Google Scholar
  10. 10.
    Riondato, M., DeBrabant, J.A., Fonseca, R., Upfal, E.: PARMA: a parallel randomized algorithm for approximate association rules mining in MapReduce. In: Proceedings of the 21st ACM International Conference on Information and Knowledge Management, pp. 85–94 (2012)Google Scholar
  11. 11.
    Gasmi, G., BenYahia, S., Nguifo, E.M., Slimani, Y.: IGB: a new informative generic base of association rules. In: Proceedings of the Intl. Ninth Pacific-Asia Conference on Knowledge Data Discovery (PAKDD 2005). LNAI, vol. 3518, pp. 81–90. Spring, Hanoi, Vietnam (2005)Google Scholar
  12. 12.
    Bastide, Y., Pasquier, N., Taouil, R., Lakhal, L., Stumme, G.: Mining minimal non-redundant association rules using frequent closed itemsets. In: Proceedings of the International Conference DOOD 2000, LNAI, vol. 1, no. 861, pp. 972–986. Springer, London (2000)CrossRefGoogle Scholar
  13. 13.
    Wang, S.-Q., Yang, Y.-B., Gao, Y., Chen, G.-P., Zhang, Y.: Mapreduce based closed frequent itemset mining with efficient redundancy filtering. In: ICDM Workshop, pp. 449–453 (2012)Google Scholar
  14. 14.
    Li, H., Wang, Y., Zhang, D., Zhang, M., Chang, E.Y.: PFP: parallel FP growth for query recommendation. In: ACM Conference on Recommender Systems (RecSys), pp. 107–114 (2008)Google Scholar
  15. 15.
    Zitouni, M., Akbarinia, R., Yahia, S.B., Masseglia, F.: A prime number based approach for closed frequent itemset mining in big data. In: the 26th International conference on database and expert systems applications (DEXA’2015), vol. 9261, pp. 509–516 (2015)Google Scholar
  16. 16.
    Ines, B., Samir, E.: Integrated generic association rule based classifier. In: DEXA Workshops, pp. 514–515 (2007)Google Scholar
  17. 17.
  18. 18.

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Marwa Bouraoui
    • 1
    Email author
  • Ines Bouzouita
    • 1
  • Amel Grissa Touzi
    • 1
  1. 1.University of Tunis El Manar, LR-SITI, ENITTunisTunisia

Personalised recommendations