Abstract
This paper investigates the impact that task difficulty and crowd composition have on the success of the Wisdom of Artificial Crowds metaheuristic. The metaheuristic, which is inspired by the wisdom of crowds phenomenon, combines the intelligence from a group of optimization searches to form a new solution. Unfortunately, the aggregate formed by the metaheuristic is not always better than the best individual solution within the crowd, and little is known about the variables which maximize the metaheuristic’s success. Our study offers new insights into the influential factors of artificial crowds and the collective intelligence of multiple optimization searches performed on the same problem. The results show that favoring the opinions of experts (i.e., the better searches) improves the chances of the metaheuristic succeeding by more than 15% when compared to the traditional means of equal weighting. Furthermore, weighting expertise was found to require smaller crowd sizes for the metaheuristic to reach its peak chances of success. Finally, crowd size was discovered to be a critical factor, especially as problem complexity grows or average crowd expertise declines. However, crowd size matters only up to a point, after which the probability of success plateaus.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Ashby, L.H., Yampolskiy, R.V.: Genetic algorithm and wisdom of artificial crowds algorithm applied to light up. In: International Conference on Computer Games 2011, pp. 27–32 (2011)
Chen, J., Ren, Y., Riedl, J.: The effects of diversity on group productivity and member withdrawal in online volunteer groups. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 821–830 (2010)
Hughes, R., Yampolskiy, R.V.: Solving sudoku puzzles with wisdom of artificial crowds. Int. J. Intell. Games Simul. 7(1), 24–29 (2012)
Hundley, M., Yampolskiy, R.V.: Shortest total path length spanning tree via wisdom of artificial crowds algorithm. In: Proceedings of the 28th Modern Artificial Intelligence and Cognitive Science Conference (2017)
Kantardzic, M.: Data Mining: Concepts, Models, Methods, and Algorithms. Wiley, Hoboken (2011)
Khalifa, A.B., Yampolskiy, R.V.: Ga with wisdom of artificial crowds for solving mastermind satisfiability problem. Int. J. Intell. Games Simul. 6(2), 12–17 (2011)
Kittur, A., Kraut, R.E.: Harnessing the wisdom of crowds in wikipedia: quality through coordination. In: Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work, pp. 37–46 (2008)
Lowrance, C.J., Abdelwahab, O., Yampolskiy, R.V.: Evolution of a metaheuristic for aggregating wisdom from artificial crowds. In: Pereira, F., Machado, P., Costa, E., Cardoso, A. (eds.) EPIA 2015. LNCS (LNAI), vol. 9273, pp. 238–249. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23485-4_24
Mäntylä, M.V., Itkonen, J.: The effect of crowd size and time restriction in software testing. Inf. Softw. Technol. 55(6), 986–1003 (2013)
Moore, T., Clayton, R.: Evaluating the wisdom of crowds in assessing phishing websites. In: Tsudik, G. (ed.) FC 2008. LNCS, vol. 5143, pp. 16–30. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-85230-8_2
Port, A.C., Yampolskiy, R.V.: Using a GA and wisdom of artificial crowds to solve solitaire battleship puzzles. In: International Conference on Computer Games, pp. 25–29 (2012)
Robert, L., Romero, D.M.: Crowd size, diversity and performance. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 1379–1382 (2015)
Surowiecki, J.: The Wisdom of Crowds. Random House LLC, New York City (2005)
Trainor, P.J., Yampolskiy, R.V., DeFilippis, A.P.: Wisdom of artificial crowds feature selection in untargeted metabolomics: an application to the development of a blood-based diagnostic test for thrombotic myocardial infarction. J. Biomed. Inform. 81, 53–60 (2017)
Velic, M., Grzinic, T., Padavic, I.: Wisdom of crowds algorithm for stock market predictions. In: Proceedings of the International Conference on Information Technology Interfaces, pp. 137–144 (2013)
von der Gracht, H.A., Hommel, U., Prokesch, T., Wohlenberg, H.: Testing weighting approaches for forecasting in a group wisdom support system environment. J. Bus. Res. 69(10), 4081–4094 (2016)
Wagner, C., Suh, A.: The wisdom of crowds - impact of collective size and expertise transfer on collective performance. In: 47th Hawaii International Conference on System Sciences, pp. 594–603 (2014)
Welsh, M.: Expertise and the wisdom of crowds - whose judgments to trust and when. In: 34th Annual Meeting of the Cognitive Science Society, pp. 594–603 (2012)
Yampolskiy, R.V., Ashby, L., Hassan, L.: Wisdom of artificial crowds—a metaheuristic algorithm for optimization. J. Intell. Learn. Syst. Appl. 4(02), 98–107 (2012)
Yampolskiy, R.V., El-Barkouky, A.: Wisdom of artificial crowds algorithm for solving NP-hard problems. Int. J. Bio-inspired Comput. 3(6), 358–369 (2011)
Concorde TSP Solver. http://www.math.uwaterloo.ca/tsp/concorde/index.html. Accessed 10 July 2018
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Lowrance, C.J., Larkin, D.M., Yim, S.M. (2018). Effect of Crowd Composition on the Wisdom of Artificial Crowds Metaheuristic. In: Kim, D., Uma, R., Zelikovsky, A. (eds) Combinatorial Optimization and Applications. COCOA 2018. Lecture Notes in Computer Science(), vol 11346. Springer, Cham. https://doi.org/10.1007/978-3-030-04651-4_36
Download citation
DOI: https://doi.org/10.1007/978-3-030-04651-4_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-04650-7
Online ISBN: 978-3-030-04651-4
eBook Packages: Computer ScienceComputer Science (R0)