Abstract
Conventional incentive mechanisms were designed for business environments involving static business processes and a limited number of actors. They are not easily applicable to crowdsourcing and other social computing platforms, characterized by dynamic collaboration patterns and high numbers of actors, because the effects of incentives in these environments are often unforeseen and more costly than in a well-controlled environment of a traditional company.
In this paper we investigate how to design and calibrate incentive schemes for crowdsourcing processes by simulating joint effects of a combination of different participation and incentive mechanisms applied to a working crowd. More specifically, we present a simulation model of incentive schemes and evaluate it on a relevant real-world scenario. We show how the model is used to simulate different compositions of incentive mechanisms and model parameters, and how these choices influence the costs on the system provider side and the number of malicious workers.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Adar, E.: Why I Hate Mechanical Turk Research (and Workshops). Control (2011), http://www.cond.org/eadar-crowdsourcing-workshop.pdf
Bloom, M., Milkovich, G.: The relationship between risk, incentive pay, and organizational performance. The Academy of Management Journal 41(3), 283–297 (1998), http://www.jstor.org/pss/256908
Dorn, C., Edwards, G., Medvidovic, N.: Analyzing design tradeoffs in large-scale socio-technical systems through simulation of dynamic collaboration patterns. In: Meersman, R., et al. (eds.) OTM 2012, Part I. LNCS, vol. 7565, pp. 362–379. Springer, Heidelberg (2012)
Gal, Y., Grosz, B., Kraus, S., Pfeffer, A., Shieber, S.: Agent decision-making in open mixed networks. Artif. Intell. 174(18), 1460–1480 (2010), http://dx.doi.org/10.1016/j.artint.2010.09.002
Gilbert, N., Troitzsch, K.: Simulation for the social scientist. Open University Press, McGraw-Hill Education (2005)
Heckman, J., Smith, J.: What do bureaucrats do? The effects of performance standards and bureaucratic preferences on acceptance into the JTPA program. Advances in the Study of Entrepreneurship Innovation and Economic Growth 7, 191–217 (1996)
Kraus, S., Hoz-Weiss, P., Wilkenfeld, J., Andersen, D.R., Pate, A.: Resolving crises through automated bilateral negotiations. Artificial Intelligence 172(1), 1–18 (2008), http://www.sciencedirect.com/science/article/pii/S0004370207001051
Laffont, J.J., Martimort, D.: The Theory of Incentives. Princeton University Press, New Jersey (2002)
Little, G., Chilton, L.B., Goldman, M., Miller, R.: Exploring iterative and parallel human computation processes. In: Proceedings of the 28th of the International Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA 2010, p. 4309 (2010), http://portal.acm.org/citation.cfm?doid=1753846.1754145
Macal, C.M., North, M.J.: Agent-based modeling and simulation. In: Proceedings of the 2009 Winter Simulation Conference (WSC), pp. 86–98 (December 2009), http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5429318
Mason, W., Watts, D.J.: Financial incentives and the performance of crowds. In: Proceedings of the ACM SIGKDD Workshop on Human Computation (HCOMP 2009), vol. 11, pp. 77–85. ACM, Paris (May 2009), http://portal.acm.org/citation.cfm?doid=1809400.1809422
Milinski, M., Semmann, D., Krambeck, H.J.: Reputation helps solve the ‘tragedy of the commons’. Nature 415(6870), 424–426 (2002), http://dx.doi.org/10.1038/415424a
Prendergast, C.: The provision of incentives in firms. Journal of Economic Literature 37(1), 7–63 (1999), http://www.jstor.org/stable/2564725
Rockenbach, B., Milinski, M.: The efficient interaction of indirect reciprocity and costly punishment. Nature 444(7120), 718–723, http://dx.doi.org/10.1038/nature05229
Scekic, O., Truong, H.-L., Dustdar, S.: Modeling rewards and incentive mechanisms for social BPM. In: Barros, A., Gal, A., Kindler, E. (eds.) BPM 2012. LNCS, vol. 7481, pp. 150–155. Springer, Heidelberg (2012), http://dx.doi.org/10.1007/978-3-642-32885-5_11
Scekic, O., Truong, H.-L., Dustdar, S.: Incentives and rewarding in social computing. Commun. ACM 56(6), 72–82 (2013), http://doi.acm.org/10.1145/2461256.2461275
Scekic, O., Truong, H.-L., Dustdar, S.: Programming incentives in information systems. In: Salinesi, C., Norrie, M.C., Pastor, O. (eds.) CAiSE 2013. LNCS, vol. 7908, pp. 688–703. Springer, Heidelberg (2013), http://dx.doi.org/10.1007/978-3-642-38709-8_44
Semmann, D., Krambeck, H.J., Milinski, M.: Strategic investment in reputation. Behavioral Ecology and Sociobiology 56(3), 248–252 (2004), http://dx.doi.org/10.1007/s00265-004-0782-9
Tokarchuk, O., Cuel, R., Zamarian, M.: Analyzing crowd labor and designing incentives for humans in the loop. IEEE Internet Computing 16, 45–51 (2012)
Wedekind, C., Milinski, M.: Cooperation Through Image Scoring in Humans. Science 288(5467), 850–852 (2000), http://dx.doi.org/10.1126/science.288.5467.850
Wegner, R.: Multi-agent malicious behaviour detection. Phd thesis, University of Manitoba (2012), http://hdl.handle.net/1993/9673
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Scekic, O., Dorn, C., Dustdar, S. (2013). Simulation-Based Modeling and Evaluation of Incentive Schemes in Crowdsourcing Environments. In: Meersman, R., et al. On the Move to Meaningful Internet Systems: OTM 2013 Conferences. OTM 2013. Lecture Notes in Computer Science, vol 8185. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41030-7_11
Download citation
DOI: https://doi.org/10.1007/978-3-642-41030-7_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-41029-1
Online ISBN: 978-3-642-41030-7
eBook Packages: Computer ScienceComputer Science (R0)