Abstract
We present a algorithm based on factor analysis for performing collaborative quality filtering (CQF). Unlike previous approaches to CQF, which estimate the consensus opinion of a group of reviewers, our algorithm uses a generative model of the review process to estimate the latent intrinsic quality of the items under reviews. We run several tests that demonstrate that consensus and intrinsic quality are, in fact different and unrelated aspects of quality. These results suggest that asymptotic consensus, which purports to model peer review, is, in fact, not recovering the ground truth quality of reviewed items.
This research was supported by the Digital Libraries Initiative under grant NSF CA98-17353.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Internet Movie Database: (Top 250 films), http://www.imdb.com/top_250_films
Riggs, T.: Collaborative quality filtering in open review systems. Master’s thesis, University of California, Berkeley, Computer Science Division (2001)
Wilensky, R., Riggs, T.: An algorithm for automatically rating reviewers. In: Proc. of the First Joint Conf. on Digital Libraries, Roanoke, Virginia (2001)
Katerattanakui, P., et al.: Objective quality ranking of computing journals. Communications of the ACM 46, 111–114 (2003)
Brown, L., Gardner, J.: Using citation analysis to assess the impact of journals and articles on contemporary accounting research. Journal of Accounting Research 23, 84–109 (1995)
Goldberg, D., et al.: Using collaborative filtering to weave and information tapestry. Communications of the ACM 35, 61–70 (1992)
Resnick, P., et al.: GroupLens: An Open Architecture for Collaborative Filtering of Netnews. In: Proc. of ACM 1994 Conf. on Computer Supported Cooperative Work, Chapel Hill, North Carolina, pp. 175–186. Chapel Hill, North Carolina, ACM (1994)
Hill, W., et al.: Recommending and evaluating choices in a virtual community of use. In: Proc. of the SIGCHI Conf. on Humand Factors in Computing Systems, pp. 194–201 (1995)
Shardanand, U., Maes, P.: Social information filtering: Algorithms for automating “word of mouth”. In: Proc. of ACM CHI 1995 Conf. on Human Factors in Computing Systems, vol. 1, pp. 210–217 (1995)
Herlocker, J.L., et al.: An algorithmic framework for performing collaborative filtering. In: SIGIR 1999: Proc. of the 22nd Annual International ACM SIGIR Conf. on Research and Development in Information Retrieval, Berkeley, CA, USA, August 15-19, 1999, pp. 230–237. ACM, New York (1999)
Hofmann, T.: Latent semantic models for collaborative filtering. ACM Transactions on Information Systems 22, 89–115 (2004)
Canny, J.: Collaborative filtering with privacy via factor analysis. In: Proc. of the 25th Annual International ACM SIGIR Conf. on Research and Development in Information Retrieval, pp. 238–245 (2002)
Resnick, P., Varian, H.R.: Recommender systems. Communications of the ACM (1997)
Konstan, J.A.: Introduction to recommender systems: Algorithms and evaluation. ACM Transactions on Information Systems 22, 1–4 (2004)
Middleton, S.E., et al.: Ontological user profiling in recommender systems. ACM Transactions on Information Systems 22, 54–88 (2004)
Zacharia, G., et al.: Collaborative reputation mechanisms in electronic marketplaces. In: HICSS (1999)
Rubin, D.B., Thayer, D.T.: EM algorithms for ML factor analysis. Psychometrika 47, 69–76 (1982)
Jordan, M.I.: An introduction to probabilistic graphic models (unpublished textbook manuscript)
Ghahramani, Z., Jordan, M.I.: Learning from incomplete data. Technical Report AIM-1509, Massachusettes Institute of Technology, Artificial Intelligence Laboratory (1994)
Jelinek, F., Mercer, R.L.: Interpolated estimation of markov source parameters from sparse data. In: Gelsema, E.S., Kanal, L.N. (eds.) Pattern Recognition in Practice, pp. 381–397. North-Holland Publishing Company, Amsterdam (1980)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Traupman, J., Wilensky, R. (2006). Collaborative Quality Filtering: Establishing Consensus or Recovering Ground Truth? . In: Mobasher, B., Nasraoui, O., Liu, B., Masand, B. (eds) Advances in Web Mining and Web Usage Analysis. WebKDD 2004. Lecture Notes in Computer Science(), vol 3932. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11899402_5
Download citation
DOI: https://doi.org/10.1007/11899402_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-47127-1
Online ISBN: 978-3-540-47128-8
eBook Packages: Computer ScienceComputer Science (R0)