Skip to main content
SpringerLink
Log in

Auction Based Mechanisms for Dynamic Task Assignments in Expert Crowdsourcing

  • Conference paper
  • First Online:
Book cover Agent-Mediated Electronic Commerce. Designing Trading Strategies and Mechanisms for Electronic Markets (AMEC/TADA 2015, AMEC/TADA 2016)

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 271))

Abstract

Crowdsourcing marketplaces link large populations of workers to an even larger number of tasks. Thus, it is necessary to have mechanisms for matching workers with interesting and suitable tasks. Earlier work has addressed the problem of finding optimal workers for a given set of tasks. However, workers also have preferences and will stay with a platform only if it gives them interesting tasks. We therefore analyze several matching mechanisms that take into account workers’ preferences as well. We propose that the workers pay premiums to get preferred matches and auction-based models where preferences are expressed through variations of the payment for a task. We analyze the properties of two matching different mechanisms: Split Dynamic VCG (SDV) and e-Auction. We compare both the mechanisms with Arrival Priority Serial Dictatorship (APSD) empirically for efficiency.

This work was carried out when the first author was a post-doctoral researcher at EPFL.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The workers has to be logged into the system for his availability and hence cannot report early arrival or late departure.

  2. 2.

    http://crowdsourcing.org.

  3. 3.

    It should be noted that, the settings of expert crowdsourcing are different than microtasking where the workers finish the task quickly and move on to a next task immediately.

  4. 4.

    \(\mu \) takes \(\mathbf {b_j}\)s, \(arr_j,dep_j\) as inputs and produces a bipartite matching. However to simplify notation, we just refer to \(\mu \) as a bipartite matching.

  5. 5.

    Note that this is upper bound on competitive ratio.

  6. 6.

    http://mturk.com.

  7. 7.

    Note that we are referring to expert crowdsourcing tasks and not the microtasks. Hence, such rewards are feasible.

References

  1. Babaioff, M., Immorlica, N., Kempe, D., Kleinberg, R.: Online auctions and generalized secretary problems. ACM SIGecom Exchanges 7(2), 7 (2008)

    Article  Google Scholar 

  2. Bhat, S., Nath, S., Zoeter, O., Gujar, S., Narahari, Y., Dance, C.: A mechanism to optimally balance cost and quality of labeling tasks outsourced to strategic agents. In: Thirtheenth International Conference on Autonomous Agents and Multiagent Systems, pp. 917–924 (2014)

    Google Scholar 

  3. Bigham, J.P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R.C., Miller, R., Tatarowicz, A., White, B., White, S., et al.: Vizwiz: nearly real-time answers to visual questions. In: Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology, pp. 333–342. ACM (2010)

    Google Scholar 

  4. Chen, X., Lin, Q., Zhou, D.: Optimistic knowledge gradient policy for optimal budget allocation in crowdsourcing. In: Proceedings of the 30th International Conference on Machine Learning (ICML-2013), pp. 64–72 (2013)

    Google Scholar 

  5. Difallah, D.E., Demartini, G., Cudré-Mauroux, P.: Pick-a-crowd: tell me what you like, and i’ll tell you what to do. In: Proceedings of the 22nd International Conference on World Wide Web, pp. 367–374 (2013)

    Google Scholar 

  6. Garg, D., Narahari, Y., Gujar, S.: Foundations of mechanism design: a tutorial - part 1: key concepts and classical results. In: Sadhana - Indian Academy Proceedings in Engineering Sciences, vol. 33(Part 2), pp. 83–130, April 2008

    Google Scholar 

  7. Garg, D., Narahari, Y., Gujar, S.: Foundations of mechanism design: a tutorial - part 2: advanced concepts and results. In: Sadhana - Indian Academy Proceedings in Engineering Sciences, vol. 33(Part 2), pp. 131–174, April 2008

    Google Scholar 

  8. Goel, G., Nikzad, A., Singla, A.: Allocating tasks to workers with matching constraints: truthful mechanisms for crowdsourcing markets. In: Proceedings of the Companion Publication of the 23rd International Conference on World Wide Web Companion, pp. 279–280 (2014)

    Google Scholar 

  9. Gujar, S., Faltings, B.: Dynamic task assignments: an online two sided matching approach. In: 3rd International Workshop on Matching Under Preferences, MATCHUP (2015)

    Google Scholar 

  10. Gujar, S., Faltings, B.: Online auctions for dynamic assignment: theory and empirical evaluation. In: ECAI 2016–22nd European Conference on Artificial Intelligence, 29 August-2 September 2016, The Hague, The Netherlands - Including Prestigious Applications of Artificial Intelligence (PAIS 2016), pp. 1035–1043 (2016)

    Google Scholar 

  11. Ho, C.J., Jabbari, S., Vaughan, J.W.: Adaptive task assignment for crowdsourced classification. In: Proceedings of the 30th International Conference on Machine Learning (ICML-2013), pp. 534–542 (2013)

    Google Scholar 

  12. Ho, C.J., Vaughan, J.W.: Online task assignment in crowdsourcing markets. In: AAAI (2012)

    Google Scholar 

  13. Howe, J.: The rise of crowdsourcing. Wired Mag. 14(6), 1–4 (2006)

    Google Scholar 

  14. Huang, E., Zhang, H., Parkes, D.C., Gajos, K.Z., Chen, Y.: Toward automatic task design: a progress report. In: Proceedings of the ACM SIGKDD Workshop on Human Computation, pp. 77–85. ACM (2010)

    Google Scholar 

  15. Ipeirotis, P.G., Provost, F., Wang, J.: Quality management on Amazon mechanical turk. In: Proceedings of the ACM SIGKDD Workshop on Human Computation, HCOMP 2010, pp. 64–67. ACM, New York (2010)

    Google Scholar 

  16. Jain, S., Gujar, S., Zoeter, O., Narahari, Y.: A quality assuring multi-armed bandit crowdsourcing mechanism with incentive compatible learning. In: Thirtheenth International Conference on Autonomous Agents and Multiagent Systems, pp. 1609–1610 (2014)

    Google Scholar 

  17. Karger, D.R., Oh, S., Shah, D.: Budget-optimal crowdsourcing using low-rank matrix approximations. In: 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 284–291. IEEE (2011)

    Google Scholar 

  18. Kittur, A., Chi, E.H., Suh, B.: Crowdsourcing user studies with mechanical turk. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2008, pp. 453–456. ACM, New York (2008)

    Google Scholar 

  19. Parkes, D.C.: Online mechanisms (2007)

    Google Scholar 

  20. Raykar, V.C., Yu, S., Zhao, L.H., Valadez, G.H., Florin, C., Bogoni, L., Moy, L.: Learning from crowds. J. Mach. Learn. Res. 11, 1297–1322 (2010)

    MathSciNet  Google Scholar 

  21. Sönmez, T., Ünver, M.U.: Matching, allocation, and exchange of discrete resources. Handbook Soc. Econ. 1, 781–852 (2011)

    Article  Google Scholar 

  22. Tran-Thanh, L., Stein, S., Rogers, A., Jennings, N.R.: Efficient crowdsourcing of unknown experts using multi-armed bandits. In: European Conference on Artificial Intelligence, pp. 768–773 (2012)

    Google Scholar 

  23. Zou, J.Y., Gujar, S., Parkes, D.C.: Tolerable manipulability in dynamic assignment without money. In: AAAI (2010)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. International Institute of Information Technology, Hyderabad, India

    Sujit Gujar & Boi Faltings

  2. Artifitial Intelligence Laboratory, EPFL, Lausanne, Switzerland

    Sujit Gujar & Boi Faltings

Authors
  1. Sujit Gujar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Boi Faltings
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sujit Gujar .

Editor information

Editors and Affiliations

  1. School of Informatics, University of Edinburgh , Cambridge, United Kingdom

    Sofia Ceppi

  2. Department of Computer Science, Ashkelon Academic College , Ashkelon, Israel

    Esther David

  3. Department of EE and CS, Vanderbilt University , Nashville, Tennessee, USA

    Chen Hajaj

  4. Smart Systems Group, Heriot-Watt University , Edinburgh, United Kingdom

    Valentin Robu

  5. Information Technology, The American College of Greece , Agia Paraskevi, Athens, Greece

    Ioannis A. Vetsikas

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Gujar, S., Faltings, B. (2017). Auction Based Mechanisms for Dynamic Task Assignments in Expert Crowdsourcing. In: Ceppi, S., David, E., Hajaj, C., Robu, V., Vetsikas, I. (eds) Agent-Mediated Electronic Commerce. Designing Trading Strategies and Mechanisms for Electronic Markets. AMEC/TADA AMEC/TADA 2015 2016. Lecture Notes in Business Information Processing, vol 271. Springer, Cham. https://doi.org/10.1007/978-3-319-54229-4_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-54229-4_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-54228-7

  • Online ISBN: 978-3-319-54229-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation