A Multi-Layered and Multi-Faceted Framework for Mining Evolving Web Clickstreams

Abstract

Data on the Web is noisy, huge, and dynamic. This poses enormous challenges to most data mining techniques that try to extract patterns from this data. While scalable data mining methods are expected to cope with the size challenge, coping with evolving trends in noisy data in a continuous fashion, and without any unnecessary stoppages and reconfigurations is still an open challenge. This dynamic and single pass setting can be cast within the framework of mining evolving data streams. Furthermore, the heterogeneity of the Web has required Web-based applications to more effectively integrate a variety of types of data across multiple channels and from different sources such as content, structure, and more recently, semantics. Most existing Web mining and personalization methods are limited to working at the level described to be the lowest and most primitive level, namely discovering models of the user profiles from the input data stream. However, in order to improve understanding of the real intention and dynamics of Web clickstreams, we need to extend reasoning and discovery beyond the usual data stream level. We propose a new multi-level framework for Web usage mining and personalization, consisting of knowledge discovery at different granularities: (i) session/user clicks, profiles, (ii) profile life events and profile communities, and (iii) sequential patterns and predicted shifts in the user profiles. One of the most promising features of the proposed framework address the challenging dynamic scenarios, including (i) defining and detecting events in the life of a synopsis profile, such as Birth, Death and Atavism, and (ii) identifying Node Communities that can later be used to track the temporal evolution of Web profile activity events and dynamic trends within communities, such as Expansion, Shrinking, and Drift.