Abstract
Peer-prediction [19] is a (meta-)mechanism which, given any proper scoring rule, produces a mechanism to elicit prie information from self-interested agents. Formally, truth-telling is a strict Nash equilibrium of the mechanism. Unfortunately, there may be other equilibria as well (including uninformative equilibria where all players simply report the same fixed signal, regardless of their true signal) and, typically, the truth-telling equilibrium does not have the highest expected payoff. The main result of this paper is to show that, in the symmetric binary setting, by tweaking peer-prediction, in part by carefully selecting the proper scoring rule it is based on, we can make the truth-telling equilibrium focal—that is, truth-telling has higher expected payoff than any other equilibrium.
Along the way, we prove the following: in the setting where agents receive binary signals we (1) classify all equilibria of the peer-prediction mechanism; (2) introduce a new technical tool for understanding scoring rules, which allows us to make truth-telling pay better than any other informative equilibrium; (3) leverage this tool to provide an optimal version of the previous result; that is, we optimize the gap between the expected payoff of truth-telling and other informative equilibria; and (4) show that with a slight modification to the peer-prediction framework, we can, in general, make the truth-telling equilibrium focal—that is, truth-telling pays more than any other equilibrium (including the uninformative equilibria).
Y. Kong—Supported by National Science Foundation Career Award 1452915 and CCF Award 1618187.
K. Ligett—Supported in part NSF grants 1254169 and 1518941, US-Israel Binational Science Foundation Grant 2012348, the Charles Lee Powell Foundation, a subcontract through the Brandeis project, a grant from the HUJI Cyber Security Research Center, and a startup grant from Hebrew Universitys School of Computer Science.
G. Schoenebeck—Supported by National Science Foundation Career Award 1452915 and Algorithms in the Field Award 1535912.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Explicit statement in full version.
References
Crémer, J., McLean, R.P.: Optimal selling strategies under uncertainty for a discriminating monopolist when demands are interdependent. Econometrica 53(2), 345–361 (1985)
Crémer, J., McLean, R.P.: Full extraction of the surplus in bayesian and dominant strategy auctions. Econometrica 56(6), 1247–1257 (1988)
Dasgupta, A., Ghosh, A.: Crowdsourced judgement elicitation with endogenous proficiency. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 319–330. International World Wide Web Conferences Steering Committee (2013)
d’Aspremont, C., Gérard-Varet, L.A.: Bayesian incentive compatible beliefs. J. Math. Econ. 10(1), 83–103 (1982)
d’Aspremont, C., Grard-Varet, L.A.: Incentives and incomplete information. J. Publ. Econ. 11(1), 25–45 (1979)
Faltings, B., Jurca, R., Pu, P., Tran, B.D.: Incentives to counter bias in human computation. In: Second AAAI Conference on Human Computation and Crowdsourcing (2014)
Gao, X.A., Mao, A., Chen, Y., Adams, R.P.: Trick or treat: putting peer prediction to the test. In: Proceedings of the Fifteenth ACM Conference on Economics and Computation, pp. 507–524. ACM (2014)
Ghosh, A., Ligett, K., Roth, A., Schoenebeck, G.: Buying private data without verification. In: Proceedings of the Fifteenth ACM Conference on Economics and Computation, pp. 931–948. ACM (2014)
Goel, S., Reeves, D.M., Pennock, D.M.: Collective revelation: a mechanism for self-verified, weighted, and truthful predictions. In: Proceedings of the 10th ACM Conference on Electronic Commerce (EC 2009) (2009)
Jurca, R., Faltings, B.: Incentives for answering hypothetical questions. In: Proceedings of the 1st Workshop on Social Computing and User Generated Content (SC 2011). ACM (2011)
Jurca, R., Faltings, B.: Minimum payments that reward honest reputation feedback. In: Proceedings of the 7th ACM Conference on Electronic Commerce (EC 2006) (2006)
Jurca, R., Faltings, B.: Collusion-resistant, incentive-compatible feedback payments. In: Proceedings of the 8th ACM Conference on Electronic Commerce, pp. 200–209. ACM (2007)
Jurca, R., Faltings, B.: Incentives for expressing opinions in online polls. In: Proceedings of the 9th ACM Conference on Electronic Commerce (EC 2008) (2008)
Jurca, R., Faltings, B.: Mechanisms for making crowds truthful. J. Artif. Int. Res. 34(1), 209 (2009)
Kamble, V., Shah, N., Marn, D., Parekh, A., Ramachandran, K.: Truth serums for massively crowdsourced evaluation tasks. arXiv preprint arXiv:1507.07045
Kong, Y., Schoenebeck, G.: A framework for designing information elicitation mechanisms that reward truth-telling. ArXiv e-prints, May 2016
Kong, Y., Schoenebeck, G.: Equilibrium selection in information elicitation without verification via information monotonicity. ArXiv e-prints, March 2016
Lambert, N., Shoham, Y.: Truthful surveys. In: Papadimitriou, C., Zhang, S. (eds.) WINE 2008. LNCS, vol. 5385, pp. 154–165. Springer, Heidelberg (2008). doi:10.1007/978-3-540-92185-1_23
Miller, N., Resnick, P., Zeckhauser, R.: Eliciting informative feedback: the peer-prediction method. Manag. Sci. 51(9), 1359–1373 (2005)
Prelec, D.: A Bayesian truth serum for subjective data. Science 306(5695), 462–466 (2004)
Radanovic, G., Faltings, B.: Incentive schemes for participatory sensing. In: Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems, pp. 1081–1089. International Foundation for Autonomous Agents and Multiagent Systems (2015)
Riley, B.: Minimum truth serums with optional predictions. In: Proceedings of the 4th Workshop on Social Computing and User Generated Content (SC14) (2014)
Shnayder, V., Agarwal, A., Frongillo, R., Parkes, D.C.: Informed truthfulness in multi-task peer prediction. ArXiv e-prints, March 2016
Witkowski, J., Parkes, D.C.: Peer prediction without a common prior. In: Proceedings of the 13th ACM Conference on Electronic Commerce, pp. 964–981. ACM (2012)
Witkowski, J., Parkes, D.C.: Learning the prior in minimal peer prediction. In: Proceedings of the 3rd Workshop on Social Computing and User Generated Content at the ACM Conference on Electronic Commerce, p. 14. Citeseer (2013)
Acknowledgments
We thank David Parkes and Paul Resnick for helpful suggestions.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer-Verlag GmbH Germany
About this paper
Cite this paper
Kong, Y., Ligett, K., Schoenebeck, G. (2016). Putting Peer Prediction Under the Micro(economic)scope and Making Truth-Telling Focal. In: Cai, Y., Vetta, A. (eds) Web and Internet Economics. WINE 2016. Lecture Notes in Computer Science(), vol 10123. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-54110-4_18
Download citation
DOI: https://doi.org/10.1007/978-3-662-54110-4_18
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-54109-8
Online ISBN: 978-3-662-54110-4
eBook Packages: Computer ScienceComputer Science (R0)