Worker Recommendation with High Acceptance Rates in Collaborative Crowdsourcing Systems

  • Mingchu Li
  • Xiaomei Sun
  • Xing JinEmail author
  • Linlin Tian
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 268)


Crowdsourcing has emerged as a popular Internet-based collaboration paradigm, in which tasks published by requesters can be economically and efficiently accomplished by crowd workers. To ensure the quality of service (QoS) provided by crowd workers, requesters are more likely to assign tasks to trustworthy workers, therefore, trust have played an important role in the design of worker recommendation mechanisms in crowdsourcing systems. Most existing studies focus on the trust that requesters place on workers, however, which would suffer the low-acceptance problem because crowd workers would refuse to participate in tasks published by low-trustworthy requesters with a great probability. In order to address the low-acceptance problem, in this paper, by using biased matrix factorization, we proposed a novel worker recommendation mechanism which can evaluate mutual trust relationship between requesters and workers. And also, to accurately measure the matching degree between tasks and workers, a comprehensive and practical task matching mechanism has been presented by incorporating time matching, skill matching, payment matching, and location matching. Finally, extensive simulations and real data experiments highlight the performance of our proposed worker recommendation mechanism.


Collaborative crowdsourcing Trust Worker recommendation Matrix factorization Task matching 



The paper is supported by National Nature Science foundation of China under grant Nos.: 61572095 and 61877007.


  1. 1.
    Abhinav, K., Dubey, A., Jain, S., Virdi, G., Kass, A., Mehta, M.: CrowdAdvisor: a framework for freelancer assessment in online marketplace. In: ICSE, pp. 93–102. IEEE (2017)Google Scholar
  2. 2.
    Alsayasneh, M., et al.: Personalized and diverse task composition in crowdsourcing. TKDE 30(1), 128–141 (2018)Google Scholar
  3. 3.
    Difallah, D.E., Demartini, G., Cudré-Mauroux, P.: Pick-a-crowd: tell me what you like, and I’ll tell you what to do. In: WWW, pp. 367–374. ACM (2013)Google Scholar
  4. 4.
    Farkas, K., Nagy, A.Z., Tomás, T., Szabó, R.: Participatory sensing based real-time public transport information service. In: PERCOM Workshops, pp. 141–144. IEEE (2014).
  5. 5.
    Gaikwad, S.N.S., et al.: Boomerang: rebounding the consequences of reputation feedback on crowdsourcing platforms. In: UIST, pp. 625–637. ACM (2016)Google Scholar
  6. 6.
    Goel, G., Nikzad, A., Singla, A.: Matching workers expertise with tasks: incentives in heterogeneous crowdsourcing markets. In: NIPS (2013)Google Scholar
  7. 7.
    ul Hassan, U., Curry, E.: Efficient task assignment for spatial crowdsourcing: a combinatorial fractional optimization approach with semi-bandit learning. Expert Syst. Appl. 58, 36–56 (2016)CrossRefGoogle Scholar
  8. 8.
    He, X., Zhang, H., Kan, M.Y., Chua, T.S.: Fast matrix factorization for online recommendation with implicit feedback. In: SIGIR, pp. 549–558. ACM (2016)Google Scholar
  9. 9.
    Jabeur, N., Karam, R., Melchiori, M., Renso, C.: A comprehensive reputation assessment framework for volunteered geographic information in crowdsensing applications. Ubiquit. Comput. 1–17 (2018)Google Scholar
  10. 10.
    Kamvar, S.D., Schlosser, M.T., Garcia-Molina, H.: The Eigentrust algorithm for reputation management in P2P networks. In: WWW, pp. 640–651. ACM (2003)Google Scholar
  11. 11.
    Koren, Y.: Factor in the neighbors: scalable and accurate collaborative filtering. TKDD 4(1), 1–24 (2010)CrossRefGoogle Scholar
  12. 12.
    Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)CrossRefGoogle Scholar
  13. 13.
    Kurve, A., Miller, D.J., Kesidis, G.: Multicategory crowdsourcing accounting for variable task difficulty, worker skill, and worker intention. TKDE 27(3), 794–809 (2015)Google Scholar
  14. 14.
    Liu, H., Zhang, X., Zhang, X.: Possible world based consistency learning model for clustering and classifying uncertain data. Neural Netw. 102, 48–66 (2018)CrossRefGoogle Scholar
  15. 15.
    Liu, H., Zhang, X., Zhang, X., Cui, Y.: Self-adapted mixture distance measure for clustering uncertain data. Knowl.-Based Syst. 126, 33–47 (2017)CrossRefGoogle Scholar
  16. 16.
    Lu, K., Wang, J., Li, M.: An Eigentrust dynamic evolutionary model in P2P file-sharing systems. Peer Peer Netw. Appl. 9(3), 599–612 (2016)CrossRefGoogle Scholar
  17. 17.
    Pu, L., Chen, X., Xu, J., Fu, X.: Crowdlet: optimal worker recruitment for self-organized mobile crowdsourcing. In: INFOCOM, pp. 1–9. IEEE (2016)Google Scholar
  18. 18.
    Qiao, L., Tang, F., Liu, J.: Feedback based high-quality task assignment in collaborative crowdsourcing. In: AINA, pp. 1139–1146. IEEE (2018)Google Scholar
  19. 19.
    Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, pp. 452–461. AUAI Press (2009)Google Scholar
  20. 20.
    Safran, M., Che, D.: Real-time recommendation algorithms for crowdsourcing systems. Appl. Comput. Inform. 13(1), 47–56 (2017)CrossRefGoogle Scholar
  21. 21.
    Schall, D., Satzger, B., Psaier, H.: Crowdsourcing tasks to social networks in BPEL4People. World Wide Web 17(1), 1–32 (2014)CrossRefGoogle Scholar
  22. 22.
    Schnitzer, S., Neitzel, S., Schmidt, S., Rensing, C.: Perceived task similarities for task recommendation in crowdsourcing systems. In: WWW, pp. 585–590 (2016)Google Scholar
  23. 23.
    Song, T., et al.: Trichromatic online matching in real-time spatial crowdsourcing. In: ICDE, pp. 1009–1020. IEEE (2017)Google Scholar
  24. 24.
    Tong, Y., She, J., Ding, B., Wang, L., Chen, L.: Online mobile micro-task allocation in spatial crowdsourcing. In: ICDE, pp. 49–60. IEEE (2016)Google Scholar
  25. 25.
    Wang, J., Wang, F., Wang, Y., Zhang, D., Lim, B.Y., Wang, L.: Allocating heterogeneous tasks in participatory sensing with diverse participant-side factorsGoogle Scholar
  26. 26.
    Wang, L., Yu, Z., Han, Q., Guo, B., Xiong, H.: Multi-objective optimization based allocation of heterogeneous spatial crowdsourcing tasks. TMC 17(17), 1637–1650 (2018)Google Scholar
  27. 27.
    Wang, Y., Tong, X., He, Z., Gao, Y., Wang, K.: A task recommendation model for mobile crowdsourcing systems based on dwell-time. In: BDCloud-SocialCom-SustainCom, pp. 170–177. IEEE (2016)Google Scholar
  28. 28.
    Wu, C., Luo, T., Wu, F., Chen, G.: EndorTrust: an endorsement-based reputation system for trustworthy and heterogeneous crowdsourcing. In: GLOBECOM, pp. 1–6. IEEE (2015)Google Scholar
  29. 29.
    Xiang, Q., Zhang, J., Nevat, I., Zhang, P.: A trust-based mixture of Gaussian processes model for reliable regression in participatory sensing. In: IJCAI, pp. 3866–3872 (2017)Google Scholar
  30. 30.
    Ye, B., Wang, Y.: CrowdRec: trust-aware worker recommendation in crowdsourcing environments. In: ICWS, pp. 1–8. IEEE (2016)Google Scholar
  31. 31.
    Ye, B., Wang, Y., Liu, L.: Crowd trust: a context-aware trust model for worker selection in crowdsourcing environments. In: ICWS, pp. 121–128. IEEE (2015)Google Scholar
  32. 32.
    Yu, H., Shen, Z., Miao, C., An, B.: A reputation-aware decision-making approach for improving the efficiency of crowdsourcing systems. In: AAMAS, pp. 1315–1316 (2013)Google Scholar
  33. 33.
    Yuan, F., Gao, X., Lindqvist, J.: How busy are you?: Predicting the interruptibility intensity of mobile users. In: CHI, pp. 5346–5360. ACM (2017)Google Scholar
  34. 34.
    Yuen, M.C., King, I., Leung, K.S.: TaskRec: a task recommendation framework in crowdsourcing systems. NPL 41(2), 223–238 (2015)Google Scholar
  35. 35.
    Zhang, X., Liu, H., Zhang, X.: Novel density-based and hierarchical density-based clustering algorithms for uncertain data. Neural Netw. 93, 240–255 (2017)CrossRefGoogle Scholar
  36. 36.
    Zhang, X., Liu, H., Zhang, X., Liu, X.: Novel density-based clustering algorithms for uncertain data. In: AAAI, pp. 2191–2197 (2014)Google Scholar
  37. 37.
    Zhang, X., Xue, G., Yu, R., Yang, D., Tang, J.: Truthful incentive mechanisms for crowdsourcing. In: INFOCOM, pp. 2830–2838. IEEE (2015)Google Scholar

Copyright information

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2019

Authors and Affiliations

  • Mingchu Li
    • 1
    • 2
  • Xiaomei Sun
    • 1
    • 2
  • Xing Jin
    • 1
    • 2
    Email author
  • Linlin Tian
    • 1
    • 2
  1. 1.School of Software TechnologyDalian University of TechnologyDalianChina
  2. 2.Key Laboratory for Ubiquitous Network and Service Software of Liaoning ProvinceDalianChina

Personalised recommendations