Self-Selection Bias in Reputation Systems

  • Mark A. Kramer
Part of the IFIP International Federation for Information Processing book series (IFIPAICT, volume 238)


Reputation systems appear to be inherently biased towards better-than-average ratings. We explain this as a consequence of self-selection, where reviewers are drawn disproportionately from the subset of potential consumers favorably predisposed toward the resource. Inflated ratings tend to attract consumers with lower expected value, who have a greater chance of disappointment. Paradoxically, the more accurate the ratings, the greater the degree of self-selection, and the faster the ratings become biased. We derive sufficient conditions under which biased ratings occur. Finally, we outline a potential solution to this problem that involves stating expectations before interaction with the resource, and expressing subsequent ratings in terms of delight or disappointment.


Reputation System Resource Selection Rating Bias Prior Expectation Feedback Group 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    J.J. Heckerman, Sample Selection Bias as a Specification Error, Econometrica, 47(1), 153–162 (1997).CrossRefGoogle Scholar
  2. 2.
    Xinxin Li and L.M. Hitt, Self Selection and Information Role of Online Product Reviews, Working Paper, Wharton School of Management, University of Pennsylvania (2004);
  3. 3. Netflix, Inc. (January 17, 2007);
  4. 4.
    C. Piller, Everyone Is A Critic in Cyberspace, Los Angeles Times (December 3, 1999).Google Scholar
  5. 5.
    J. Chevalier and D. Mayzlin, The Effect of Word of Mouth on Sales: Online Book Reviews, Working Paper, Yale School of Management (2003).Google Scholar

Copyright information

© International Federation for Information Processing 2007

Authors and Affiliations

  • Mark A. Kramer
    • 1
  1. 1.MITRE CorporationBedfordUSA

Personalised recommendations