Abstract
Suppose you need to determine the correct answer to a complex question that depends on two logically independent premises. You can ask several agents to each evaluate either just one of those premises (which they can do with relatively high accuracy) or both premises (in which case their need to multitask will lower their individual accuracy). We first determine the optimal rule to aggregate the individual judgments reported by the agents and then analyse their strategic incentives, depending on whether they are motivated by (i) the group tracking the truth, by (ii) maximising their own reputation, or by (iii) maximising the agreement of the group’s findings with their own private judgments. We also study the problem of deciding how many agents to ask for two judgments and how many to ask for just a single judgment.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
For instance, doctors making judgments about their patients may simply care about the correctness of their collective judgment, participants of an experiment that are paid proportionally to their agreement with the group can be assumed to aim at being seen to agree with their peers, and people who like having their opinions confirmed might manipulate the group to agree with their own privately held judgment.
- 2.
The other direction does not hold. Importantly, a counterexample may go through under free but not fixed assignments because the agents have the option to manipulate by abstaining on some premise they have sincerely thought about.
- 3.
For any assignment, collective accuracy converges to 1 as the size of the group grows larger. Thus, our analysis is most interesting for groups that are not very large.
- 4.
The calculations were performed using a computer program in R.
References
Adler, R.F., Benbunan-Fich, R.: Juggling on a high wire: multitasking effects on performance. Int. J. Hum. Comput. Stud. 70(2), 156–168 (2012)
Ahn, D.S., Oliveros, S.: The condorcet jur(ies) theorem. J. Econ. Theor. 150, 841–851 (2014)
Ariely, D., Zakay, D.: A timely account of the role of duration in decision making. Acta Psychol. 108(2), 187–207 (2001)
Bovens, L., Rabinowicz, W.: Democratic answers to complex questions–an epistemic perspective. Synthese 150(1), 131–153 (2006)
Bozbay, I., Dietrich, F., Peters, H.: Judgment aggregation in search for the truth. Games Econ. Behav. 87, 571–590 (2014)
de Clippel, G., Eliaz, K.: Premise-based versus outcome-based information aggregation. Games Econ. Behav. 89, 34–42 (2015)
Cohensius, G., Porat, O.B., Meir, R., Amir, O.: Efficient crowdsourcing via proxy voting. In: Proceedings of the 7th International Workshop on Computational Social Choice (COMSOC-2018) (2018)
de Condorcet, M.: Essai sur l’application de l’analyse à la probabilité des décisions rendues à la pluralité des voix. Imprimerie Royale (1785)
Dawid, A.P., Skene, A.M.: Maximum likelihood estimation of observer error-rates using the EM algorithm. Appl. Stat. 28(1), 20–28 (1979)
Edland, A., Svenson, O.: Judgment and decision making under time pressure. In: Svenson, O., Maule, A.J. (eds.) Time Pressure and Stress in Human Judgment and Decision Making, pp. 27–40. Springer, Boston (1993)
Endriss, U.: Judgment aggregation. In: Brandt, F., Conitzer, V., Endriss, U., Lang, J., Procaccia, A.D. (eds.) Handbook of Computational Social Choice, pp. 399–426. Cambridge University Press, Cambridge (2016)
Grossi, D., Pigozzi, G.: Judgment Aggregation: A Primer, Synthesis Lectures on Artificial Intelligence and Machine Learning, vol. 8. Morgan & Claypool Publishers, Los Altos (2014)
Hartmann, S., Pigozzi, G., Sprenger, J.: Reliable methods of judgement aggregation. J. Logic Comput. 20(2), 603–617 (2010)
Hartmann, S., Sprenger, J.: Judgment aggregation and the problem of tracking the truth. Synthese 187(1), 209–221 (2012)
List, C., Pettit, P.: Aggregating sets of judgments: an impossibility result. Econ. Philos. 18(1), 89–110 (2002)
Payne, J.W., Bettman, J.R., Johnson, E.J.: Adaptive strategy selection in decision making. J. Exp. Psychol. Learn. Mem. Cogn. 14(3), 534 (1988)
Terzopoulou, Z., Endriss, U., de Haan, R.: Aggregating incomplete judgments: axiomatisations for scoring rules. In: Proceedings of the 7th International Workshop on Computational Social Choice (COMSOC-2018) (2018)
Wilhelm, O., Schulze, R.: The relation of speeded and unspeeded reasoning with mental speed. Intelligence 30(6), 537–554 (2002)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Terzopoulou, Z., Endriss, U. (2019). Optimal Truth-Tracking Rules for the Aggregation of Incomplete Judgments. In: Fotakis, D., Markakis, E. (eds) Algorithmic Game Theory. SAGT 2019. Lecture Notes in Computer Science(), vol 11801. Springer, Cham. https://doi.org/10.1007/978-3-030-30473-7_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-30473-7_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30472-0
Online ISBN: 978-3-030-30473-7
eBook Packages: Computer ScienceComputer Science (R0)