Advertisement

LineChange: An Analytic Framework for Automated Moderation of Crowdsourcing Systems

  • Brent D. FegleyEmail author
  • Ryan Mullins
  • Ben Ford
  • Chad Weiss
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 850)

Abstract

If humans are more productive in collective problem-solving with a modicum of active help and guidance, then the potential of automated moderation of crowdsourcing systems has yet to be realized. Here, we present the conceptual design of an intelligent machine capable of (a) monitoring the temporal, structural, and emergent characteristics of participant behavior in a problem-solving process, and (b) modifying team structure and prompting participants for input at opportune or transitional moments in that process—by configuration, rule, or inference—to achieve collective goals and optimize output. The design is unique in treating teams as composable objects, in being scale-free, in relying on configuration and inference (not hard-coding), and in treating participant behaviors as sensory input.

Keywords

Crowdsourcing Insourcing Network generation Automated facilitation Decision support 

Notes

Acknowledgements

This research was performed in connection with contract N68335-18-C-0040 with the U.S. Office of Naval Research. We would like to thank Dr. Yiling Chen and Dr. Predrag Neskovic for their contributions to this work as thought partners.

References

  1. 1.
    Anson, R., Bostrom, R., Wynne, B.: An experiment assessing group support system and facilitator effects on meeting outcomes. Manag. Sci. 41(2), 189–208 (1995).  https://doi.org/10.1287/mnsc.41.2.189CrossRefGoogle Scholar
  2. 2.
    Austen-Smith, D., Feddersen, T.: Deliberation and voting rules. In: Austen-Smith, D., Duggan, J. (eds.) Social Choice and Strategic Decisions: Essays in Honor of Jeffrey S. Banks. Studies in Choice and Welfare, pp. 269–316. Springer, Heidelberg (2005).  https://doi.org/10.1007/3-540-27295-X_11CrossRefzbMATHGoogle Scholar
  3. 3.
    Bernstein, M.S., Little, G., Miller, R.C., Hartmann, B., Ackerman, M.S., Karger, D.R., Crowell, D., Panovich, K.: Soylent: a word processor with a crowd inside. Commun. ACM 58(8), 85–94 (2015).  https://doi.org/10.1145/2791285CrossRefGoogle Scholar
  4. 4.
    Briggs, R.O., Reinig, B.A.: Bounded ideation theory. J. Manag. Inf. Syst. 27(1), 123–144 (2010).  https://doi.org/10.2753/MIS0742-1222270106CrossRefGoogle Scholar
  5. 5.
    Dennis, A.R., Williams, M.L.: Electronic brainstorming: theory, research, and future directions. In: Paulus, P.B., Nijstad, B.A. (eds.) Group Creativity: Innovation Through Collaboration, pp. 160–180. Oxford University Press, Oxford (2003).  https://doi.org/10.1093/acprof:oso/9780195147308.003.0008CrossRefGoogle Scholar
  6. 6.
    Granovetter, M.: The strength of weak ties: a network theory revisited. Sociol. Theory 1, 201–233 (1983). http://www.jstor.org/stable/202051CrossRefGoogle Scholar
  7. 7.
    Guimer, R., Uzzi, B., Spiro, J., Amaral, L.A.N.: Team assembly mechanisms determine collaboration network structure and team performance. Science 308(5722), 697–702 (2005). http://science.sciencemag.org/content/308/5722/697CrossRefGoogle Scholar
  8. 8.
    Judd, S., Kearns, M., Vorobeychik, Y.: Behavioral dynamics and influence in networked coloring and consensus. Proc. Nat. Acad. Sci. 107(34), 14978–14982 (2010). http://www.pnas.org/content/107/34/14978.abstractCrossRefGoogle Scholar
  9. 9.
    Kim, T., Chang, A., Holland, L., Pentland, A.S.: Meeting mediator: enhancing group collaboration using sociometric feedback. In: Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work, CSCW 2008, pp. 457–466. ACM (2008).  https://doi.org/10.1145/1460563.1460636
  10. 10.
    Mao, A., Procaccia, A.D., Chen, Y.: Better human computation through principled voting. In: AAAI (2013)Google Scholar
  11. 11.
    Mellers, B., Stone, E., Murray, T., Minster, A., Rohrbaugh, N., Bishop, M., Chen, E., Baker, J., Hou, Y., Horowitz, M., Ungar, L., Tetlock, P.: Identifying and cultivating superforecasters as a method of improving probabilistic predictions. Perspect. Psychol. Sci. 10(3), 267–281 (2015).  https://doi.org/10.1177/1745691615577794CrossRefGoogle Scholar
  12. 12.
    Nemeth, C.J., Nemeth-Brown, B.: Better than individuals? The potential benefits of dissent and diversity for group creativity. In: Paulus, P.B., Nijstad, B.A. (eds.) Group Creativity: Innovation Through Collaboration, pp. 63–84. Oxford University Press, Oxford (2003).  https://doi.org/10.1093/acprof:oso/9780195147308.003.0004CrossRefGoogle Scholar
  13. 13.
    Ottaviani, M., Srensen, P.: Information aggregation in debate: who should speak first? J. Public Econ. 81(3), 393–421 (2001).  https://doi.org/10.1016/S0047-2727(00)00119-5CrossRefGoogle Scholar
  14. 14.
    Post, M., Callison-Burch, C., Osborne, M.: Constructing parallel corpora for six Indian languages via crowdsourcing. In: Proceedings of the Seventh Workshop on Statistical Machine Translation, pp. 401–409. Association for Computational Linguistics, Montréal, June 2012. http://www.aclweb.org/anthology/W12-3152
  15. 15.
    Spencer, D.: Card Sorting: Designing Usable Categories. Rosenfeld Media, New York (2009)Google Scholar
  16. 16.
    Wong, Z., Aiken, M.: Automated facilitation of electronic meetings. Inf. Manag. 41(2), 125–134 (2003).  https://doi.org/10.1016/S0378-7206(03)00042-9CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Brent D. Fegley
    • 1
    Email author
  • Ryan Mullins
    • 1
  • Ben Ford
    • 1
  • Chad Weiss
    • 1
  1. 1.Aptima, Inc.WoburnUSA

Personalised recommendations