Web Mining Accelerated with In-Memory and Column Store Technology

  • Patrick Hennig
  • Philipp Berger
  • Christoph Meinel
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8346)


Current web mining approaches use massive amounts of commodity hardware and processing time to leverage analytics for today’s web. For a seamless application interaction, those approaches have to use pre-aggregated results and indexes to circumvent the slow processing on their data stores e.g. relational databases or document stores. The upcoming trend of in-memory, column-oriented databases is widely used to accelerate business analytics like financial reports, but the application on large text corpora remains unaffected. We argue that although in-memory, column-oriented stores are tailor-made for traditional data schemes, they are also applicable for web mining applications that mainly consists of raw text informations enriched with limited semantic meta data. Thus, we implement a web mining application that stores every information in a pure main memory data store. We experience an acceleration of current web mining queries and identify new opportunities for web mining applications. To evaluate the performance impact, we compare the run-time of general web mining tasks on a traditional row-oriented, disc-based database and a column-oriented, in-memory database using the example of BlogIntelligence, which serves exemplary for web mining applications.


blog analysis web mining data mining in-memory column-layout 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bahmani, B., Chakrabarti, K., Xin, D.: Fast personalized pagerank on mapreduce. In: Proceedings of the 37th SIGMOD International Conference on Management of Data, pp. 973–984 (2011)Google Scholar
  2. 2.
    Bross, J., Richly, K., Kohnen, M., Meinel, C.: Identifying the top-dogs of the blogosphere. Social Netw. Analys. Mining 2(1), 53–67 (2012)CrossRefGoogle Scholar
  3. 3.
    Chang, F., Dean, J., Ghemawat, S., Hsieh, W.C., Wallach, D.A., Burrows, M., Chandra, T., Fikes, A., Gruber, R.E.: Bigtable: A distributed storage system for structured data. ACM Transactions on Computer Systems (TOCS) 26(2), 4 (2008)Google Scholar
  4. 4.
    Etzioni, O.: The world-wide web: quagmire or gold mine? Communications of the ACM 39(11), 65–68 (1996)CrossRefGoogle Scholar
  5. 5.
    Hewitt, E.: Cassandra: the definitive guide. O’Reilly Media, Incorporated (2010)Google Scholar
  6. 6.
    Bross, J., Kohnen, M., Richly, K., Kohnen, M., Meinel, C.: Identifying the top dogs of the blogosphere. Social Network Analysis and Mining. Springer LNSN (2011)Google Scholar
  7. 7.
    Kosala, R., Blockeel, H.: Web mining research: A survey. ACM Sigkdd Explorations Newsletter 2(1), 1–15 (2000)CrossRefGoogle Scholar
  8. 8.
    Maes, P., et al.: Agents that reduce work and information overload. Communications of the ACM 37(7), 30–40 (1994)CrossRefGoogle Scholar
  9. 9.
    Momjian, B.: PostgreSQL: introduction and concepts, vol. 192. Addison-Wesley (2001)Google Scholar
  10. 10.
    Hennig, P., Berger, P., J.B.C.M.: Mapping the blogosphere - towards a universal and scalable blog-crawler. In: Proceedings of the Third IEEE International Conference on Social Computing (Social Com2011), pp. 672–677. IEEE CS, MIT, Boston, USA (2011)Google Scholar
  11. 11.
    Page, L., Brin, S., Motwani, R., Winograd, T.: The pagerank citation ranking: bringing order to the web (1999)Google Scholar
  12. 12.
    Plattner, H.: A course in In-Memory Data Management. Springer, Berlin (2013)CrossRefGoogle Scholar
  13. 13.
    Sparck Jones, K.: A statistical interpretation of term specificity and its application in retrieval, pp. 132–142 (December 1988)Google Scholar
  14. 14.
    Widenius, M., Axmark, D., MySQL, A.: MySQL reference manual: documentation from the source. O’Reilly Media, Incorporated (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Patrick Hennig
    • 1
  • Philipp Berger
    • 1
  • Christoph Meinel
    • 1
  1. 1.Hasso-Plattner-InstitutUniversity of PotsdamGermany

Personalised recommendations