Abstract
In this paper, we propose a multi-dimensional approach to support modeling and consensus management in collective crowdsourcing applications/problems. We define the notion of crowd-consensus, and, for each dimension of analysis, we set pre-defined dimensional levels capturing the different variabilities characterizing the crowd-consensus along that dimension in different applications/problems. The design of a crowdsourcing task for a given target problem requires to characterize the task with respect to each dimension following a pattern-based approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
For implementation, a threshold \(th_k\) is specified to set the maximum number of workers that can belong to a narrow task-force.
- 2.
Techniques for progressively updating the worker trustworthiness are out of the scope of this paper. A possible solution based on reinforcement learning is discussed in [4].
- 3.
References
Arolas, E.E., de Guevara, F.G.L.: Towards an integrated crowdsourcing definition. J. Inf. Sci. 38(2), 189–200 (2012)
Barowy, D.W., Curtsinger, C., Berger, E.D., McGregor, A.: AutoMan: a platform for integrating human-based and digital computation. In: Proceedings of the 27th Annual ACM SIGPLAN OOPSLA Conference, Tucson, AZ, USA (2012)
Bozzon, A., Brambilla, M., Ceri, S., Mauri, A.: Reactive crowdsourcing. In: Proceedings of the 22nd International WWW Conference, Rio de Janeiro, Brazil (2013)
Castano, S., Ferrara, A., Genta, L., Montanelli, S.: Combining crowd consensus and user trustworthiness for managing collective tasks. Future Gener. Comput. Syst. (to appear, 2015)
Chiu, C.M., Liang, T.P., Turban, E.: What can crowdsourcing do for decision support? Decis. Support Syst. 65, 40–49 (2014)
Demartini, G., Difallah, D.E., Cudré-Mauroux, P.: ZenCrowd: leveraging probabilistic reasoning and crowdsourcing techniques for large-scale entity linking. In: Proceedings of the 21st WWW Conference, Lyon, France (2012)
Doan, A., Ramakrishnan, R., Halevy, A.Y.: Crowdsourcing systems on the world-wide web. Commun. ACM 54(4), 86–96 (2011)
Finin, T., et al.: Annotating named entities in Twitter data with crowdsourcing. In: Proceedings of the NAACL HLT Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk. Los Angeles, CA, USA (2010)
Geiger, D., Rosemann, M., Fielt, E.: Crowdsourcing information systems a systems theory perspective. In: Proceedings of the 22nd Australasian Conference on Information Systems, Sydney, Australia (2011)
Geiger, D., Seedorf, S., Schulze, T., Nickerson, R.C., Schader, M.: Managing the crowd: towards a taxonomy of crowdsourcing processes. In: Proceedings of the Americas Conference on Information Systems (AMCIS), Detroit, MI, USA (2011)
Hoßfeld, T., et al.: Survey of web-based crowdsourcing frameworks for subjective quality assessment. In: Proceedings of the 16th IEEE International Workshop on Multimedia Signal Processing (MMSP), Jakarta, Indonesia (2014)
Luon, Y., Aperjis, C., Huberman, B.A.: Rankr: a mobile system for crowdsourcing opinions. In: Proceedings of the 3rd International MobiCASE Conference, Los Angeles, CA, USA (2012)
McCann, R., Shen, W., Doan, A.: Matching schemas in online communities: a Web 2.0 approach. In: Proceedings of the 24th International Conference on Data Engineering (ICDE 2008), Cancún, Mexico (2008)
Scekic, O., Truong, H.L., Dustdar, S.: Incentives and rewarding in social computing. Commun. ACM 56(6), 72–82 (2013)
Yuen, M.C., King, I., Leung, K.S.: A survey of crowdsourcing systems. In: Proceedings of the 3rd IEEE International PASSAT/SocialCom Conference, Boston, MA, USA (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Castano, S., Ferrara, A., Montanelli, S. (2015). A Multi-dimensional Approach to Crowd-Consensus Modeling and Evaluation. In: Johannesson, P., Lee, M., Liddle, S., Opdahl, A., Pastor López, Ó. (eds) Conceptual Modeling. ER 2015. Lecture Notes in Computer Science(), vol 9381. Springer, Cham. https://doi.org/10.1007/978-3-319-25264-3_31
Download citation
DOI: https://doi.org/10.1007/978-3-319-25264-3_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25263-6
Online ISBN: 978-3-319-25264-3
eBook Packages: Computer ScienceComputer Science (R0)