Exploring Versus Exploiting when Learning User Models for Text Recommendation

Abstract

The text recommendation task involves delivering sets of documents to users on the basis of user models. These models are improved over time, given feedback on the delivered documents. When selecting documents to recommend, a system faces an instance of the exploration/exploitation tradeoff: whether to deliver documents about which there is little certainty, or those which are known to match the user model learned so far. In this paper, a simulation is constructed to investigate the effects of this tradeoff on the rate of learning user models, and the resulting compositions of the sets of recommended documents, in particular World-Wide Web pages. Document selection strategies are developed which correspond to different points along the tradeoff. Using an exploitative strategy, our results show that simple preference functions can successfully be learned using a vector-space representation of a user model in conjunction with a gradient descent algorithm, but that increasingly complex preference functions lead to a slowing down of the learning process. Exploratory strategies are shown to increase the rate of user model acquisition at the expense of presenting users with suboptimal recommendations; in addition they adapt to user preference changes more rapidly than exploitative strategies. These simulated tests suggest an implementation for a simple control that is exposed to users, allowing them to vary a system's document selection behavior depending on individual circumstances.