Abstract
Being considered as a valid solution to the lack of ground truth data problem, crowdsourcing has recently gained a lot of attention within the biomedical domain. However, available concepts in life science domain require expert knowledge and thereby restrict the access to only very specific communities. In this paper, we go beyond state-of-the-art and present a novel concept for seamlessly embedding biomedical science into a common game canvas. Besides introducing the visual saliency concept, we thereby essentially eliminate the requirement for prior knowledge. We have further implemented a game to evaluate our novel concept in three different user studies.
S. Albarqouni and S. Matl—contributed equally towards this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Albarqouni, S., Baur, C., Achilles, F., Belagiannis, V., Demirci, S., Navab, N.: Aggnet: deep learning from crowds for mitosis detection in breast cancer histology images. IEEE Trans. Med. Imaging 35(5), 1313–1321 (2016)
Aroyo, L., Welty, C.: The three sides of crowdtruth. Hum. Comput. 1(1), 31–44 (2014)
Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., Leaver-Fay, A., Baker, D., Popović, Z.: Foldit players: predicting protein structures with a multiplayer online game. Nature 446, 756–760 (2010)
Dumitrache, A., Aroyo, L., Welty, C., Sips, R.J., Levas, A.: Dr. detective: combining gamication techniques and crowdsourcing to create a gold standard in medical text. In: Proceedings of the 1st International Conference on Crowdsourcing the Semantic Web-Volume 1030, pp. 16–31. CEUR-WS. org (2013)
Fuchs, J., Isenberg, P., Bezerianos, A., Fischer, F., Bertini, E.: The influence of contour on similarity perception of star glyphs. IEEE Trans. Vis. Comput. Graph. 20(12), 2251–2260 (2014)
Good, B.M., Loguercio, S., Griffith, O.L., Nanis, M., Wu, C., Su, A.I.: The cure: design and evaluation of a crowdsourcing game for gene selection for breast cancer survival prediction. JMIR Serious Games 2(2), e7 (2014). doi:10.2196/games
Hamari, J., Koivisto, J., Sarsa, H.: Does gamification work?-a literature review of empirical studies on gamification. In: 2014 47th Hawaii International Conference on System Sciences (HICSS), pp. 3025–3034. IEEE (2014)
Littlestone, N., Warmuth, M.K.: The weighted majority algorithm. Inf. Comput. 108(2), 212–261 (1994)
Luengo-Oroz, M.A., Arranz, A., Frean, J.: Crowdsourcing malaria parasite quantification: an online game for analyzing images of infected thick blood smears. J. Med. Internet Res. 14(6), e167 (2012)
Macenko, M., Niethammer, M., Marron, J., Borland, D., Woosley, J.T., Guan, X., Schmitt, C., Thomas, N.E.: A method for normalizing histology slides for quantitative analysis. In: ISBI, vol. 9, pp. 1107–1110 (2009)
Maier-Hein, L., Ross, T., Glocker, B., Bodenstedt, S., Stock, C., Heim, E., Wirkert, S., Kenngott, H., Speidel, S., Maier-Hein, K., et al.: Crowd-algorithm collaboration for large-scale endoscopic image annotation with confidence
Maier-Hein, L., Kondermann, D., Roß, T., Mersmann, S., Heim, E., Bodenstedt, S., Kenngott, H.G., Sanchez, A., Wagner, M., Preukschas, A., Wekerle, A.L., Helfert, S., März, K., Mehrabi, A., Speidel, S., Stock, C.: Crowdtruth validation: a new paradigm for validating algorithms that rely on image correspondences. Int. J. Comput. Assist. Radiol. Surg. 10(8), 1201–1212 (2015)
Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online learning for matrix factorization and sparse coding. J. Mach. Learn. Res. 11, 19–60 (2010)
Veta, M., Van Diest, P.J., Willems, S.M., Wang, H., Madabhushi, A., Cruz-Roa, A., Gonzalez, F., Larsen, A.B., Vestergaard, J.S., Dahl, A.B., et al.: Assessment of algorithms for mitosis detection in breast cancer histopathology images. Med. Image Anal. 20(1), 237–248 (2015)
Wertheimer, M.: Untersuchungen zur lehre von der gestalt. ii. Psychologische Forschung 4(1), 301–350 (1923). http://dx.doi.org/10.1007/BF00410640
Zheng, Y., Liu, D., Georgescu, B., Nguyen, H., Comaniciu, D.: 3D deep learning for efficient and robust landmark detection in volumetric data. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 565–572. Springer, Heidelberg (2015). doi:10.1007/978-3-319-24553-9_69
Acknowledgment
We would like to thank all anonymous players who participate in our game. We are also grateful to Dr. Mitko Veta for giving us the permission to use the AMIDA13 dataset in our research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Albarqouni, S., Matl, S., Baust, M., Navab, N., Demirci, S. (2016). Playsourcing: A Novel Concept for Knowledge Creation in Biomedical Research. In: Carneiro, G., et al. Deep Learning and Data Labeling for Medical Applications. DLMIA LABELS 2016 2016. Lecture Notes in Computer Science(), vol 10008. Springer, Cham. https://doi.org/10.1007/978-3-319-46976-8_28
Download citation
DOI: https://doi.org/10.1007/978-3-319-46976-8_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46975-1
Online ISBN: 978-3-319-46976-8
eBook Packages: Computer ScienceComputer Science (R0)