Market-Based Recommender Systems: Learning Users’ Interests by Quality Classification

Abstract

Recommender systems are widely used to cope with the problem of information overload and, consequently, many recommendation methods have been developed. However, no one technique is best for all users in all situations. To combat this, we have previously developed a market-based recommender system that allows multiple agents (each representing a different recommendation method or system) to compete with one another to present their best recommendations to the user. In our system, the marketplace encourages good recommendations by rewarding the corresponding agents according to the users’ ratings of their suggestions. Moreover, we have shown this incentivises the agents to bid in a manner that ensures only the best recommendations are presented. To do this effectively, however, each agent needs to classify its recommendations into different internal quality levels, learn the users’ interests and adapt its bidding behaviour for the various internal quality levels accordingly. To this end, in this paper, we develop a reinforcement learning and Boltzmann exploration strategy that the recommending agents can exploit for these tasks. We then demonstrate that this strategy helps the agents to effectively obtain information about the users’ interests which, in turn, speeds up the market convergence and enables the system to rapidly highlight the best recommendations.

This research is funded in part by QinetiQ and the EPSRC Magnitude project (reference GR/N35816).