Abstract
Crowdsourcing using independent tasks provides an effective means of leveraging human intelligence to solve discretized problems. However, this model cannot handle acquiring input on an ongoing task from workers. In order to expand the power of crowd algorithms, we present an overview of approaches to continuous real-time crowdsourcing that engages workers for longer periods of time, allowing workers to receive feedback from the system as the task evolves due to their and other’s input. We describe how models of continuous crowdsourcing can be used to enable task completion using both synchronous and asynchronous groups of workers. We then explore a new model of continuous real-time crowdsourcing called a “crowd agent” that allows groups of workers to interact with users and their environment as if they were a single individual. This model provides a means of abstracting away the collective in crowdsourcing by making it appear as a single intelligence.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bernstein MS, Little G, Miller RC, Hartmann B, Ackerman MS, Karger DR, Crowell D, Panovich K (2010) Soylent: a word processor with a crowd inside. In: Proceedings of the 23nd annual ACM symposium on user interface software and technology, UIST’10, ACM, New York, pp 313–322
Bernstein MS, Brandt JR, Miller RC, Karger DR (2011) Crowds in two seconds: enabling real-time crowd-powered interfaces. In: Proceedings of the 24th annual ACM symposium on user interface software and technology, UIST’11, ACM, New York, pp 33–42
Bernstein MS, Karger DR, Miller RC, Brandt J (2012) Analytic methods for optimizing real-time crowdsourcing. In: Proceedings of the collective intelligence, Boston, MA
Bigham JP, Jayant C, Ji H, Little G, Miller A, Miller RC, Miller R, Tatarowicz A, White B, White S, Yeh T (2010) Vizwiz: nearly real-time answers to visual questions. In: Proceedings of the 23nd annual ACM symposium on user interface software and technology, UIST’10, ACM, New York, pp 333–342
Brady E, Morris MR, Zhong Y, Bigham JP (2013) Visual challenges in the everyday lives of blind people. In: Proceedings of the ACM SIGCHI conference on human factors in computing systems (CHI 2013), Paris
Chilton L (2009) Seaweed: a web application for designing economic games. Master’s thesis, MIT
Cooper S, Khatib F, Treuille A, Barbero J, Lee J, Beenen M, Leaver-Fay A, Baker D, Popovic Z, Players F (2010) Predicting protein structures with a multiplayer online game. Nature 466(7307):756–760
Dai P, Mausam, Weld DS (2010) Decision-theoretic control of crowd-sourced workflows. In: Twenty-fourth AAAI conference on artificial intelligence (AAAI 2010), Atlanta
Kokkalis N, Köhn T, Pfeiffer C, Chornyi D, Bernstein MS, Klemmer SR (2013) EmailValet: managing email overload through private, accountable crowdsourcing. In: Proceedings of the 2013 conference on computer supported cooperative work (CSCW ’13), ACM, New York
Lasecki WS, Murray K, White S, Miller RC, Bigham JP (2011) Real-time crowd control of existing interfaces. In: Proceedings of the ACM symposium on User Interface Software and Technology, UIST’11, ACM, New York, pp 23–32
Lasecki WS, Miller CD, Sadilek A, Abumoussa A, Borrello D, Kushalnagar R, Bigham JP (2012a) Real-time captioning by groups of non-experts. In: Proceedings of the ACM symposium on User Interface Software and Technology (UIST 2012), Boston, MA, pp 23–34
Lasecki WS, White S, Murray K, Bigham JP (2012b) Crowd memory: learning in the collective. In: Proceedings of collective intelligence (CI’12), Boston
Lasecki WS, Miller CD, Bigham JP (2013a) Warping time for more effective real-time crowdsourcing. In: Proceedings of the international ACM conference on human factors in computing systems, CHI’13, page to appear, Paris, France
Lasecki WS, Song Y, Kautz H, Bigham J (2013b) Real-time crowd labeling for deployable activity recognition. In: Proceedings of the international ACM conference on computer supported cooperative work and social computing (CSCW 2013), San Antonio, TX, pp 1203–1212
Lasecki WS, Wesley R, Nichols J, Kulkarni A, Allen JF, Bigham J (2013c) Chorus: a crowd-powered conversational assistant. In: Proceedings of the 23nd annual ACM symposium on user interface software and technology (UIST’13), St. Andrews, UK. UIST 2013. PP 151-162
Little G, Chilton LB, Goldman M, Miller RC (2010) Turkit: human computation algorithms on mechanical Turk. In: Proceedings of the 23nd annual ACM symposium on user interface software and technology, UIST’10, ACM, New York, pp 57–66
Naim I, Lasecki WS, Bigham JP, Gildea D (2013) Text alignment for real-time crowd captioning. In: Proceedings of the North American Chapter of the association for computational linguistics (NAACL 2013), To appear, Atlanta, GA
Nielsen J (1993) Usability engineering. Morgan Kaufmann, San Francisco
von Ahn L, Dabbish L (2004) Labeling images with a computer game. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI’04, ACM, New York, pp 319–326
Zhang H, Law E, Miller RC, Gajos K, Parkes DC, Horvitz E (2012) Human computation tasks with global constraints. In: Proceedings of the SIGCHI conference on human factors in computing systems (CHI’12), pp 217–226, Austin, TX
Zhong Y, Thiha P, He G, Lasecki WS, Bigham JP (2012) In: Proceedings of the ACM conference on human factors in computing systems work-in-progress (CHI 2012), Austin
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer Science+Business Media New York
About this chapter
Cite this chapter
Lasecki, W.S., Bigham, J.P. (2013). Interactive Crowds: Real-Time Crowdsourcing and Crowd Agents. In: Michelucci, P. (eds) Handbook of Human Computation. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-8806-4_38
Download citation
DOI: https://doi.org/10.1007/978-1-4614-8806-4_38
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-8805-7
Online ISBN: 978-1-4614-8806-4
eBook Packages: Computer ScienceComputer Science (R0)