Abstract
The generalized use of the Internet and social network platforms has changed the way human beings establish relations, collaborate and share resources. In this context, crowdsourcing (or crowd computing) is becoming a common solution to provide answers to complex problems by automatically coordinating the potential of machines and human beings working together. Several challenges still separate crowdsourcing from its generalized acceptance by industry. For instance, the quality delivered by the workers in the crowd is crucial and depends on different aspects such as their skills, experience, commitment, etc. Trusting the individuals in a social network and their capacity to carry out the different tasks assigned to them becomes essential in speeding up the adoption of this new technology in industrial environments. Capacity to deliver on time, cost or confidentiality are just some other possible obstacles to be removed. In this paper, we discuss some of these issues, provide solutions to improve the quality in systems based on the use of crowdsourcing and present a real industrial problem where we use the crowd to leverage the work capacity of geographically distributed human beings.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Banerjee, S., Lavie, A.: METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pp. 65–72. Association for Computational Linguistics, Ann Arbor (2005), http://www.aclweb.org/anthology/W/W05/W05-0909
Bentivogli, L., Federico, M., Moretti, G., Paul, M.: Getting expert quality from the crowd for machine translation evaluation. In: MT Summit XIII, pp. 521–528 (2011)
Bernstein, M.S., Little, G., Miller, R.C., Hartmann, B., Ackerman, M.S., Karger, D.R., Crowell, D., Panovich, K.: Soylent: a word processor with a crowd inside. In: Procs. of the 23nd Annual ACM Symposium on User Interface Software and Technology, UIST 2010, New York, USA, pp. 313–322 (2010), http://dx.doi.org/10.1145/1866029.1866078
Denkowski, M., Lavie, A.: Exploring normalization techniques for human judgments of machine translation adequacy collected using amazon mechanical turk. In: Procs of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk, CSLDAMT 2010, pp. 57–61. Association for Computational Linguistics, Stroudsburg (2010), http://dl.acm.org/citation.cfm?id=1866696.1866705
Gao, Q., Vogel, S.: Consensus versus expertise: a case study of word alignment with mechanical turk. In: Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk, CSLDAMT 2010, pp. 30–34. Association for Computational Linguistics, Stroudsburg (2010), http://dl.acm.org/citation.cfm?id=1866696.1866700
Geiger, D., Seedorf, S., Schulze, T., Nickerson, R.C., Schader, M.: Managing the crowd: Towards a taxonomy of crowdsourcing processes. In: AMCIS (2011)
Harris, C.: You’re Hired! An Examination of Crowdsourcing Incentive Models in Human Resource Tasks. In: Lease, M., Carvalho, V., Yilmaz, E. (eds.) Proceedings of the Workshop on Crowdsourcing for Search and Data Mining (CSDM) at the Fourth ACM International Conference on Web Search and Data Mining (WSDM), Hong Kong, China, pp. 15–18 (February 2011)
Howe, J.: Wired 14.06: The Rise of Crowdsourcing, http://www.wired.com/wired/archive/14.06/crowds.html
Kittur, A., Smus, B., Khamkar, S., Kraut, R.E.: Crowdforge: crowdsourcing complex work. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST 2011, pp. 43–52. ACM, New York (2011), http://doi.acm.org/10.1145/2047196.2047202
Lease, M., Yilmaz, E.: Crowdsourcing for information retrieval. SIGIR Forum 45(2), 66–75 (2012), http://doi.acm.org/10.1145/2093346.2093356
Negri, M., Bentivogli, L., Mehdad, Y., Giampiccolo, D., Marchetti, A.: Divide and conquer: crowdsourcing the creation of cross-lingual textual entailment corpora. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP 2011, pp. 670–679. Association for Computational Linguistics, Stroudsburg (2011), http://dl.acm.org/citation.cfm?id=2145432.2145510
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL 2002, Stroudsburg, PA, USA, pp. 311–318 (2002), http://dx.doi.org/10.3115/1073083.1073135
Silberman, M.S., Irani, L., Ross, J.: Ethics and tactics of professional crowdwork. XRDS 17(2), 39–43 (2010), http://doi.acm.org/10.1145/1869086.1869100
Yan, T., Kumar, V., Ganesan, D.: CrowdSearch: exploiting crowds for accurate. In: Intl. Conf. on Mobile Systems, Applications, and Services, pp. 77–90. ACM, New York (2010), http://dx.doi.org/10.1145/1814433.1814443
Yuen, M.C., King, I., Leung, K.S.: A Survey of Crowdsourcing Systems. In: Proceedings of the IEEE Third International Conference on Social Computing (SocialCom), pp. 766–773. IEEE (October 2011), http://dx.doi.org/10.1109/PASSAT/SocialCom.2011.203
Zaidan, O.F., Callison-Burch, C.: Crowdsourcing Translation: Professional Quality from Non-Professionals. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, Oregon, USA, pp. 1220–1229 (June 2011), http://www.aclweb.org/anthology/P11-1122.pdf
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Muntés-Mulero, V., Paladini, P., Manzoor, J., Gritti, A., Larriba-Pey, JL., Mijnhardt, F. (2013). Crowdsourcing for Industrial Problems. In: Nin, J., Villatoro, D. (eds) Citizen in Sensor Networks. CitiSens 2012. Lecture Notes in Computer Science(), vol 7685. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-36074-9_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-36074-9_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-36073-2
Online ISBN: 978-3-642-36074-9
eBook Packages: Computer ScienceComputer Science (R0)