Stimulating Skill Evolution in Market-Based Crowdsourcing

  • Benjamin Satzger
  • Harald Psaier
  • Daniel Schall
  • Schahram Dustdar
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6896)


Crowdsourcing has emerged as an important paradigm in human problem-solving techniques on the Web. One application of crowdsourcing is to outsource certain tasks to the crowd that are difficult to implement in software. Another potential benefit of crowdsourcing is the on-demand allocation of a flexible workforce. Businesses may outsource tasks to the crowd based on temporary workload variations. A major challenge in crowdsourcing is to guarantee high-quality processing of tasks. We present a novel crowdsourcing marketplace that matches tasks to suitable workers based on auctions. The key to ensuring high quality lies in skilled members whose capabilities can be estimated correctly. We present a novel auction mechanism for skill evolution that helps to correctly estimate workers and to evolve skills that are needed. Evaluations show that this leads to improved crowdsourcing.


Human-centric BPM Crowdsourcing Online communities Task markets Auctions Skill evolution 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agichtein, E., Castillo, C., Donato, D., Gionis, A., Mishne, G.: Finding high-quality content in social media. In: WSDM 2008, pp. 183–194. ACM, New York (2008)Google Scholar
  2. 2.
    Agrawal, A., et al.: WS-BPEL Extension for People (BPEL4People) (2007)Google Scholar
  3. 3.
    Amazon Mechnical Turk, (last access March 2011)
  4. 4.
    Brabham, D.: Crowdsourcing as a model for problem solving: An introduction and cases. Convergence 14(1), 75 (2008)Google Scholar
  5. 5.
    Castellano, C., Fortunato, S., Loreto, V.: Statistical physics of social dynamics. Reviews of Modern Physics 81(2), 591–646 (2009)CrossRefGoogle Scholar
  6. 6.
    CrowdFlower, (last access March 2011)
  7. 7.
    DiPalantino, D., Vojnovic, M.: Crowdsourcing and all-pay auctions. In: EC 2009, pp. 119–128. ACM, New York (2009)Google Scholar
  8. 8.
    Doan, A., Ramakrishnan, R., Halevy, A.Y.: Crowdsourcing systems on the world-wide web. Commun. ACM 54(4), 86–96 (2011)CrossRefGoogle Scholar
  9. 9.
    eBay, (last access March 2011)
  10. 10.
    Ford, M., et al.: Web Services Human Task (WS-HumanTask), Version 1.0 (2007)Google Scholar
  11. 11.
    Ipeirotis, P.G.: Analyzing the Amazon Mechanical Turk Marketplace. SSRN eLibrary 17(2), 16–21 (2010)Google Scholar
  12. 12.
    Klemperer, P.: Auctions: Theory and Practice. Princeton University Press, Princeton (2004)zbMATHGoogle Scholar
  13. 13.
    Kumar, R., Lifshits, Y., Tomkins, A.: Evolution of two-sided markets. In: WSDM 2010, pp. 311–320. ACM, New York (2010)Google Scholar
  14. 14.
    Leymann, F.: Workflow-based coordination and cooperation in a service world. In: CoopIS 2006, pp. 2–16 (2006)Google Scholar
  15. 15.
    Schall, D., Dustdar, S.: Dynamic context-sensitive pagerank for expertise mining. In: Bolc, L., Makowski, M., Wierzbicki, A. (eds.) SocInfo 2010. LNCS, vol. 6430, pp. 160–175. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  16. 16.
    Schall, D., Truong, H.L., Dustdar, S.: Unifying human and software services in web-scale collaborations. IEEE Internet Computing 12(3), 62–68 (2008)CrossRefGoogle Scholar
  17. 17.
    Su, Q., Pavlov, D., Chow, J.H., Baker, W.C.: Internet-scale collection of human-reviewed data. In: WWW 2007, pp. 231–240. ACM, New York (2007)Google Scholar
  18. 18.
    Vukovic, M.: Crowdsourcing for Enterprises. In: Proceedings of the 2009 Congress on Services, pp. 686–692. IEEE Computer Society, Los Alamitos (2009)CrossRefGoogle Scholar
  19. 19.
    Yahoo: Answers scoring system, (last access March 2011)
  20. 20.
    Yahoo! Answers, (last access March 2011)

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Benjamin Satzger
    • 1
  • Harald Psaier
    • 1
  • Daniel Schall
    • 1
  • Schahram Dustdar
    • 1
  1. 1.Distributed Systems GroupVienna University of TechnologyViennaAustria

Personalised recommendations