Information Systems Frontiers

, Volume 19, Issue 2, pp 301–319 | Cite as

Estimating participants for knowledge-intensive tasks in a network of crowdsourcing marketplaces

  • Yiwei Gong


Crowdsourcing has become an increasingly attractive practice for companies to abstain on-demand workforce and higher level of flexibility in open contexts. While knowledge-intensive crowdsourcing is expected to be prosperous, most current crowdsourcing calls are still about general and low-priced tasks. An obstacle of conducing knowledge-intensive crowdsourcing is the lack of diversity of expertise and the small scale of crowd in isolated crowdsourcing marketplaces. In this paper, a network of crowdsourcing marketplaces is envisioned for efficient knowledge-intensive crowdsourcing and engagement of massive and diverse participants across different marketplaces. Based on an algorithm for estimating participants for knowledge-intensive crowdsourcing tasks, an experiment with 100 simulations indicates that conducting crowdsourcing tasks in a network of crowdsourcing marketplaces results in higher customer satisfaction than doing that in isolated marketplaces. This finding advocates the development of a network of crowdsourcing marketplaces to open up the potential of knowledge-intensive crowdsourcing.


Knowledge-intensive crowdsourcing Flexibility Search friction Estimation algorithm 



This work is supported by the National Natural Science Foundation of China (Grand No. 71501145).


  1. Abascal-Mena, R., López-Ornelas, E., Zepeda-Hernández, J. S., Gómez-Torrero, B. E., León-Martagón, G., & Morales-Franco, H. (2014). Worker-community: using crowdsourcing to link informal workers with potential clients. Paper presented at the the 6th International Conference on Social Computing and Social Media, Heraklion, Crete, Greece,Google Scholar
  2. Afuah, A., & Tucci, C. L. (2012). Crowdsourcing as a solution to distant search. Academy of Management Review, 37(3), 355–375.CrossRefGoogle Scholar
  3. Alavi, M., & Leidner, D. E. (2001). Review: knowledge management and knowledge management systems: conceptual foundations and research issues. MIS Quarterly, 25(1), 107–136.CrossRefGoogle Scholar
  4. Brabham, D. C. (2008). Crowdsourcing as a model for problem solving. Convergence: The International Journal of Research into New Media Technologies, 14(1), 75–90.Google Scholar
  5. Chen, M.-F. (2004). Combining grey relation and TOPSIS concepts for selecting an expatriate host country. Mathematical and Computer Modelling, 40(13), 1473–1490.CrossRefGoogle Scholar
  6. Corvello, V., & Iazzolino, G. (2013). Factors affecting the practices of external problem solvers in broadcast search. Journal of Technology Management & Innovation, 8(2), 166–177.CrossRefGoogle Scholar
  7. Deng, J. (1989). Introduction to grey system. Journal of Grey System, 1(1), 1–24.Google Scholar
  8. Diamond, P. A. (1982). Aggregate demand management in search equilibrium. Journal of Political Economy, 90(5), 881–894.CrossRefGoogle Scholar
  9. Difallah, D. E., Catasta, M., Demartini, G., Ipeirotis, P. G., & Cudré-Mauroux, P. (2015). The dynamics of micro-task crowdsourcing: the case of Amazon MTurk. Paper presented at the the 24th International Conference on World Wide Web, Florence, ItalyGoogle Scholar
  10. Ford, R. C., Richard, B., & Ciuchta, M. P. (2015). Crowdsourcing: a new way of employing non-employees? Business Horizons, 58(4), 377–388.CrossRefGoogle Scholar
  11. Garrigos-Simon, F. J., Alcamí, R. L., & Ribera, T. B. (2012). Social networks and Web 3.0: their impact on the management and marketing of organizations. Management Decision, 50(10), 1880–1890.CrossRefGoogle Scholar
  12. Garrigos-Simon, F. J., & Narangajavana, Y. (2015). From crowdsourcing to the use of masscapital: the common perspective of the success of Apple, Facebook, Google, Lego, TripAdvisor, and Zara. In F. J. Garrigos-Simon, I. Gil-Pechuán, & S. Estelles-Miguel (Eds.), Advances in crowdsourcing (pp. 1–13). Switzerland: Springer.CrossRefGoogle Scholar
  13. Gefen, D., Gefen, G., & Carmel, E. (2015). How project description length and expected duration affect bidding and project success in crowdsourcing software development. Journal of Systems and Software.Google Scholar
  14. Geiger, D., & Schader, M. (2014). Personalized task recommendation in crowdsourcing information systems — current state of the art. Decision Support Systems, 65, 3–16.CrossRefGoogle Scholar
  15. Gong, Y. (2015). Enabling flexible IT services by crowdsourcing: a method for estimating crowdsourcing participants. In M. Janssen (Ed.), Open and big data management and innovation (Lecture Notes in Computer Science, Vol. 9373): Springer.Google Scholar
  16. Gong, Y., & Janssen, M. (2012). From policy implementation to business process management: principles for creating flexibility and agility. Government Information Quarterly, 29(1), 61–71.CrossRefGoogle Scholar
  17. Guittard, C., Schenk, E., & Burger-Helmchen, T. (2015). Crowdsourcing and the evolution of a business ecosystem. In F. J. Garrigos-Simon, I. Gil-Pechuán, & S. Estelles-Miguel (Eds.), Advances in crowdsourcing (pp. 49–60). Switzerland: Springer.CrossRefGoogle Scholar
  18. Heimerl, K., Gawalt, B., Chen, K., Parikh, T. S., & Hartmann, B. (2012). Communitysourcing: engaging local crowds to perform expert work via physical kiosks. Paper presented at the The ACM SIGCHI Conference on Human Factors in Computing Systems, Austin, Texas, USA.Google Scholar
  19. Howe, J. (2006), The rise of crowdsourcing. Wired Magazine.Google Scholar
  20. Hwang, C.-L., & Yoon, K. (1985). Multiple attribute decision making: methods and applications a state-of-the-art survey (Lecture Notes in Economics and Mathematical Systems): Springer-Verlag.Google Scholar
  21. IEEE (1990). Interoperability. IEEE standard computer dictionary: a compilation of IEEE standard computer glossaries. New York, USA: Institute of Electrical and Electronics Engineers.Google Scholar
  22. Jamali, R., & Tooranloo, H. S. (2009). Prioritizing academic library service quality indicators using fuzzy approach: case study: libraries of Ferdowsi university. Library Management, 30(4/5), 319–333.CrossRefGoogle Scholar
  23. Janssen, M., Estevez, E., & Janowski, T. (2014). Interoperability in big, open, and linked data--organizational maturity, capabilities, and data portfolios. Computer, 47(10), 44–49.CrossRefGoogle Scholar
  24. Jeppesen, L. B., & Lakhani, K. R. (2010). Marginality and problem solving effectiveness in broadcast search. Organization Science, 21(5), 1016–1033.CrossRefGoogle Scholar
  25. Kittur, A., Nickerson, J. V., Bernstein, M. S., Gerber, E. M., Shaw, A., Zimmerman, J., et al. (2013). The Future of Crowd Work. Paper presented at the The 16th ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW 2013), San Antonio, Texas, USA.Google Scholar
  26. Lakhani, K. R. (2013). Using the crowd as an innovation partner. Harvard Business Review, 91(4), 60–69.Google Scholar
  27. Lin, M.-C., Wang, C.-C., Chen, M.-S., & Chang, C. A. (2008). Using AHP and TOPSIS approaches in customer-driven product design process. Computers in Industry, 59(1), 17–31.CrossRefGoogle Scholar
  28. Lu, B., Hirschheim, R., & Schwarz, A. (2015). Examining the antecedent factors of online microsourcing. Information Systems Frontiers, 17(3), 601–617.CrossRefGoogle Scholar
  29. Mahr, D., Rindfleisch, A., & Slotegraaf, R. J. (2015). Enhancing crowdsourcing success: the role of creative and deliberate problem-solving styles. Customer Needs and Solutions, 2(3), 209–221.CrossRefGoogle Scholar
  30. Majchrzak, A., & Malhotra, A. (2013). Towards an information systems perspective and research agenda on crowdsourcing for innovation. Journal of Strategic Information Systems, 22(4), 257–268.CrossRefGoogle Scholar
  31. Mortensen, D. T. (2011). Markets with search friction and the DMP model. The American Economic Review, 101(4), 1073–1091.CrossRefGoogle Scholar
  32. Nevo, D., & Kotlarsky, J. (2014). Primary vendor capabilities in a mediated outsourcing model: can IT service providers leverage crowdsourcing? Decision Support Systems, 65, 17–27.CrossRefGoogle Scholar
  33. Ngai, E. W. T., & Chan, E. W. C. (2005). Evaluation of knowledge management tools using AHP. Expert Systems with Applications, 29(4), 889–899.CrossRefGoogle Scholar
  34. Oztaysi, B. (2014). A decision model for information technology selection using AHP integrated TOPSIS-Grey: The case of content management systems. Knowledge-Based Systems, 70, 44–54.CrossRefGoogle Scholar
  35. Patil, S. K., & Kant, R. (2014). A fuzzy AHP-TOPSIS framework for ranking the solutions of Knowledge Management adoption in Supply Chain to overcome its barriers. Expert Systems with Applications, 41(2), 679–693.CrossRefGoogle Scholar
  36. Pissarides, C. A. (2011). Equilibrium in the labor market with search frictions. The American Economic Review, 101(4), 1092–1105.CrossRefGoogle Scholar
  37. Roy, S. B., Lykourentzou, I., Thirumuruganathan, S., Amer-Yahia, S., & Das, G. (2015). Task assignment optimization in knowledge-intensive crowdsourcing. The VLDB Journal, 24(4), 467–491.CrossRefGoogle Scholar
  38. Saaty, T. L. (1980). The analytic hierarchy process: planning, priority setting, resource allocation (Decision Making Series): Mcgraw-Hill.Google Scholar
  39. Satzger, B., Psaier, H., Schall, D., & Dustdar, S. (2013). Auction-based crowdsourcing supporting skill management. Information Systems, 38(4), 547–560.CrossRefGoogle Scholar
  40. Schall, D., Satzger, B., & Psaier, H. (2014). Crowdsourcing tasks to social networks in BPEL4People. World Wide Web, 17(1), 1–32. doi: 10.1007/s11280-012-0180-6.CrossRefGoogle Scholar
  41. Sekhar, C., Patwardhan, M., & Vyas, V. (2015). A delphi-AHP-TOPSIS based framework for the prioritization of intellectual capital indicators: a SMEs perspective. Procedia - Social and Behavioral Sciences, 189, 275–284.CrossRefGoogle Scholar
  42. Slivkins, A., & Vaughan, J. W. (2013). Online decision making in crowdsourcing markets: theoretical challenges. ACM SIGecom Exchanges, 12(2), 4–23.CrossRefGoogle Scholar
  43. Sommer, R. A. (2003). Business process flexibility: a driver for outsourcing. Industrial Management & Data Systems, 103(3), 177–183.CrossRefGoogle Scholar
  44. Thuan, N. H., Antunes, P., & Johnstone, D. (2016). Factors influencing the decision to crowdsource: a systematic literature review. Information Systems Frontiers, 18(1), 47–68.CrossRefGoogle Scholar
  45. Vakharia, D., & Lease, M. (2015). Beyond AMT: an analysis of crowd work platforms. Paper presented at the iConference 2015, Newport Beach, CA, USA.Google Scholar
  46. Williamson, O. E. (2002). The theory of the firm as governance structure: from choice to contract. The Journal of Economic Perspectives, 16(3), 171–195.CrossRefGoogle Scholar
  47. Yang, J., Bozzon, A., & Houben, G.-J. (2015). Knowledge crowdsourcing acceleration. Paper presented at the the 15th International Conference on Web Engineering (ICWE 2015), Rotterdam, the NetherlandsGoogle Scholar
  48. Yuen, M.-C., King, I., & Leung, K.-S. (2015). TaskRec: a task recommendation framework in crowdsourcing systems. Neural Processing Letters, 41, 223–238.CrossRefGoogle Scholar
  49. Zenger, T., Felin, T., & Bigelow, L. S. (2011). Theories of the firm-market boundary. Academy of Management Annals, 5(1), 89–133.CrossRefGoogle Scholar
  50. Zhao, Y., & Zhu, Q. (2014). Evaluation on crowdsourcing research: current status and future direction. Information Systems Frontiers, 16(3), 417–434.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.School of Information ManagementWuhan UniversityWuhanPeople’s Republic of China

Personalised recommendations