A MapReduce-Based Approach for Mining Embedded Patterns from Large Tree Data
Finding tree patterns hidden in large datasets is an important research area that has many practical applications. Unfortunately, previous contributions have focused almost exclusively on extracting patterns from a set of small trees on a centralized machine. The problem of mining embedded patterns from large data trees has been neglected. However, this pattern mining problem is also important for many modern applications that arise naturally and in particular with the explosion of big data. In this paper, we propose a novel MapReduce approach to mine embedded patterns from a single large tree which can handle situations when either the tree itself or intermediate mining results at low frequency thresholds cannot fit in the memory of any individual computer node. Furthermore, we come up with a set of optimizations to minimize inter-node communication. Experimental evaluation shows that our algorithm can scale well to trees with over ten million vertices.
KeywordsTree pattern MapReduce Holistic twig-join algorithm
- 2.Bruno, N., Koudas, N., Srivastava, D.: Holistic twig joins: optimal XML pattern matching. In: Proceedings of ACM SIGMOD International Conference on Management of Data, pp. 310–321 (2002)Google Scholar
- 4.Lin, W., Xiao, X., Ghinita, G.: Large-scale frequent subgraph mining in mapreduce. In: 2014 IEEE 30th International Conference on Data Engineering, pp. 844–855, March 2014. https://doi.org/10.1109/ICDE.2014.6816705
- 5.Lu, W., Chen, G., Tung, A.K.H., Zhao, F.: Efficiently extracting frequent subgraphs using mapreduce. In: 2013 IEEE International Conference on Big Data, pp. 639–647, October 2013. https://doi.org/10.1109/BigData.2013.6691633
- 6.Schmidt, A.: Xmark—an XML benchmark project. https://projects.cwi.nl/xmark/. Accessed 28 June 2003
- 8.Wu, X., Theodoratos, D.: Leveraging homomorphisms and bitmaps to enable the mining of embedded patterns from large data trees. In: Renz, M., Shahabi, C., Zhou, X., Cheema, M.A. (eds.) DASFAA 2015. LNCS, vol. 9049, pp. 3–20. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-18120-2_1CrossRefGoogle Scholar