Web Mining Accelerated with In-Memory and Column Store Technology
Current web mining approaches use massive amounts of commodity hardware and processing time to leverage analytics for today’s web. For a seamless application interaction, those approaches have to use pre-aggregated results and indexes to circumvent the slow processing on their data stores e.g. relational databases or document stores. The upcoming trend of in-memory, column-oriented databases is widely used to accelerate business analytics like financial reports, but the application on large text corpora remains unaffected. We argue that although in-memory, column-oriented stores are tailor-made for traditional data schemes, they are also applicable for web mining applications that mainly consists of raw text informations enriched with limited semantic meta data. Thus, we implement a web mining application that stores every information in a pure main memory data store. We experience an acceleration of current web mining queries and identify new opportunities for web mining applications. To evaluate the performance impact, we compare the run-time of general web mining tasks on a traditional row-oriented, disc-based database and a column-oriented, in-memory database using the example of BlogIntelligence, which serves exemplary for web mining applications.
Keywordsblog analysis web mining data mining in-memory column-layout
Unable to display preview. Download preview PDF.
- 1.Bahmani, B., Chakrabarti, K., Xin, D.: Fast personalized pagerank on mapreduce. In: Proceedings of the 37th SIGMOD International Conference on Management of Data, pp. 973–984 (2011)Google Scholar
- 3.Chang, F., Dean, J., Ghemawat, S., Hsieh, W.C., Wallach, D.A., Burrows, M., Chandra, T., Fikes, A., Gruber, R.E.: Bigtable: A distributed storage system for structured data. ACM Transactions on Computer Systems (TOCS) 26(2), 4 (2008)Google Scholar
- 5.Hewitt, E.: Cassandra: the definitive guide. O’Reilly Media, Incorporated (2010)Google Scholar
- 6.Bross, J., Kohnen, M., Richly, K., Kohnen, M., Meinel, C.: Identifying the top dogs of the blogosphere. Social Network Analysis and Mining. Springer LNSN (2011)Google Scholar
- 9.Momjian, B.: PostgreSQL: introduction and concepts, vol. 192. Addison-Wesley (2001)Google Scholar
- 10.Hennig, P., Berger, P., J.B.C.M.: Mapping the blogosphere - towards a universal and scalable blog-crawler. In: Proceedings of the Third IEEE International Conference on Social Computing (Social Com2011), pp. 672–677. IEEE CS, MIT, Boston, USA (2011)Google Scholar
- 11.Page, L., Brin, S., Motwani, R., Winograd, T.: The pagerank citation ranking: bringing order to the web (1999)Google Scholar
- 13.Sparck Jones, K.: A statistical interpretation of term specificity and its application in retrieval, pp. 132–142 (December 1988)Google Scholar
- 14.Widenius, M., Axmark, D., MySQL, A.: MySQL reference manual: documentation from the source. O’Reilly Media, Incorporated (2002)Google Scholar