Mining Closed Colossal Frequent Patterns from High-Dimensional Dataset: Serial Versus Parallel Framework
Mining colossal patterns is one of the budding fields with a lot of applications, especially in the field of bioinformatics and genetics. Gene sequences contain inherent information. Mining colossal patterns in such sequences can further help in their study and improve prediction accuracy. The increase in average transaction length reduces the efficiency and effectiveness of existing closed frequent pattern mining algorithm. The traditional algorithms expend most of the running time in mining huge amount of minute and midsize patterns which do not enclose valuable information. The recent research focused on mining large cardinality patterns called as colossal patterns which possess valuable information. A novel parallel algorithm has been proposed to extract the closed colossal frequent patterns from high-dimensional datasets. The algorithm has been implemented on Hadoop framework to exploit its inherent distributed parallelism using MapReduce programming model. The experiment results highlight that the proposed parallel algorithm on Hadoop framework gives an efficient performance in terms of execution time compared to the existing algorithms.
KeywordsFrequent patterns Closed patterns Minimum support Closed colossal frequent patterns High-dimensional datasets Hadoop MapReduce
- 1.Chang, Rui, and Zhiyi Liu. “An improved apriori algorithm.” Electronics and Optoelectronics (ICEOE), 2011 International Conference on. Vol. 1. IEEE, 2011.Google Scholar
- 2.Han, Jiawei, Jian Pei, and Yiwen Yin. “Mining frequent patterns without candidate generation.” ACM Sigmod Record. Vol. 29. No. 2. ACM, 2000.Google Scholar
- 3.Pasquier, Nicolas, et al. “Discovering frequent closed itemsets for association rules.” Database Theory ICDT99. Springer Berlin Heidelberg, 1999. 398–416.Google Scholar
- 4.Pei, Jian, Jiawei Han, and Runying Mao. “CLOSET: An Efficient Algorithm for Mining Frequent Closed Itemsets.” ACM SIGMOD workshop on research issues in data mining and knowledge discovery. Vol. 4. No. 2. 2000.Google Scholar
- 5.Zaki, Mohammed J., and Ching-Jui Hsiao. “Efficient algorithms for mining closed itemsets and their lattice structure.” Knowledge and Data Engineering, IEEE Transactions on 17.4 (2005): 462–478.Google Scholar
- 6.Pan, Feng, et al. “Carpenter: Finding closed patterns in long biological datasets.” Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2003.Google Scholar
- 7.Zhu, Feida, et al. “Mining colossal frequent patterns by core pattern fusion.” Data Engineering, 2007. ICDE 2007. IEEE 23rd International Conference on. IEEE, 2007.Google Scholar
- 8.Sohrabi, Mohammad Karim, and Ahmad Abdollahzadeh Barforoush. “Efficient colossal pattern mining in high dimensional datasets.” Knowledge-Based Systems 33 (2012): 41–52.Google Scholar
- 9.Zulkurnain, Nurul F., David J. Haglin, and John A. Keane. “DisClose: discovering colossal closed itemsets via a memory efficient compact row-tree.” Emerging Trends in Knowledge Discovery and Data Mining. Springer Berlin Heidelberg, 2012. 141–156.Google Scholar
- 10.The Data Mining & Research Blog,. “An Introduction To Frequent Pattern Mining - The Data Mining & Research Blog”. N.p., 2013. Web. 6 Feb. 2016. http://data-mining.philippe-fournier-viger.com/introduction-frequent-pattern-mining/.
- 11.Howto.commetrics.com,. “How Raw Data Are Normalized Howto.Commetrics”. N.p., 2016. Web. 7 Feb. 2016. http://howto.commetrics.com/methodology/statistics/normalization/.
- 12.Normalization, Data. “Data Mining Blog: Data Preprocessing Normalization”. Intelligencemining.blogspot.in. N.p., 2009. Web. 7 Feb. 2016. http://intelligencemining.blogspot.in/2009/07/data-preprocessing-normalization.html.
- 13.Prekopcsk, Zoltn, et al. “Radoop: Analyzing big data with rapidminer and hadoop.” Proceedings of the 2nd RapidMiner community meeting and conference (RCOMM 2011). 2011.Google Scholar
- 14.Itkar, Suhasini A., and Uday V. Kulkarni. “Distributed Algorithm for Frequent Pattern Mining using HadoopMap Reduce Framework.” (2013).Google Scholar
- 15.Golub, Todd R., et al. “Molecular classification of cancer: class discovery and class prediction by gene expression monitoring.” Science 286.5439 (1999): 531–537.Google Scholar
- 16.Gordon, Gavin J., et al. “Translation of microarray data into clinically relevant cancer diagnostic tests using gene expression ratios in lung cancer and mesothelioma.” Cancer research 62.17 (2002): 4963–4967.Google Scholar