Skip to main content
SpringerLink
Log in

Beyond Human-in-the-Loop: Empowering End-Users with Transparent Machine Learning

  • Chapter
  • First Online:
Human and Machine Learning

Part of the book series: Human–Computer Interaction Series ((HCIS))

Abstract

Advances in data analytics and human computation are transforming how researchers conduct science in domains like bioinformatics, computational social science, and digital humanities. However, data analytics requires significant programming knowledge or access to technical experts, while human computation requires in-depth knowledge of crowd management and is error-prone due to lack of scientific domain expertise. The goal of this research is to empower a broader range of scientists and end-users to conduct data analytics by adopting the End-User Development (EUD) models commonly found in today’s commercial software platforms like Microsoft Excel, Wikipedia and WordPress. These EUD platforms enable people to focus on producing content rather than struggling with a development environment and new programming syntax or relying on disciplinary non-experts for essential technical help. This research explores a similar paradigm for scientists and end-users that can be thought of as End-User Data Analytics (EUDA), or Transparent Machine Learning (TML).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abramovici, A., Althouse, W.E., Drever, R.W., Gürsel, Y., Kawamura, S., Raab, F.J., Shoemaker, D., Sievers, L., Spero, R.E., Thorne, K.S., et al.: Ligo: the laser interferometer gravitational-wave observatory. Science 256(5055), 325–333 (1992)

    Article  Google Scholar 

  2. Baglione, A.N., Girard, M.M., Price, M., Clawson, J., Shih, P.C.: Mobile technologies for grief support: prototyping an application to support the bereaved. In: Workshop on Interactive Systems in Health Care (2017)

    Google Scholar 

  3. Baglione, A.N., Girard, M.M., Price, M., Clawson, J., Shih, P.C.: Modern bereavement: a model for complicated grief in the digital age. In: ACM Conference on Human Factors in Computing Systems (2018)

    Google Scholar 

  4. Batista, G.E., Prati, R.C., Monard, M.C.: A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD Explor. Newsl. 6(1), 20–29 (2004)

    Article  Google Scholar 

  5. Bellotti, V.M., Cambridge, S., Hoy, K., Shih, P.C., Handalian, L.R., Han, K., Carroll, J.M.: Towards community-centered support for peer-to-peer service exchange: rethinking the timebanking metaphor. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2975–2984. ACM (2014)

    Google Scholar 

  6. Berthold, M.R., Cebron, N., Dill, F., Gabriel, T.R., Kötter, T., Meinl, T., Ohl, P., Thiel, K., Wiswedel, B.: Knime-the konstanz information miner: version 2.0 and beyond. ACM SIGKDD Explor. Newsl. 11(1), 26–31 (2009)

    Article  Google Scholar 

  7. Bonney, R., Cooper, C.B., Dickinson, J., Kelling, S., Phillips, T., Rosenberg, K.V., Shirk, J.: Citizen science: a developing tool for expanding science knowledge and scientific literacy. BioScience 59(11), 977–984 (2009)

    Article  Google Scholar 

  8. Bos, N., Zimmerman, A., Olson, J., Yew, J., Yerkie, J., Dahl, E., Olson, G.: From shared databases to communities of practice: a taxonomy of collaboratories. J. Comput.-Mediat. Commun. 12(2), 652–672 (2007)

    Article  Google Scholar 

  9. Brossard, D., Lewenstein, B., Bonney, R.: Scientific knowledge and attitude change: the impact of a citizen science project. Int. J. Sci. Educ. 27(9), 1099–1121 (2005)

    Article  Google Scholar 

  10. Carroll, J.M., Shih, P.C., Hoffman, B., Wang, J., Han, K.: Presence and hyperpresence: implications for community awareness. Interacting with Presence: HCI and the Sense of Presence in Computer-mediated Environments, pp. 70–82 (2014)

    Google Scholar 

  11. Carroll, J.M., Shih, P.C., Kropczynski, J., Cai, G., Rosson, M.B., Han, K.: The internet of places at community-scale: design scenarios for hyperlocal. Enriching Urban Spaces with Ambient Computing, the Internet of Things, and Smart City Design 1 (2016)

    Google Scholar 

  12. Carroll, J.M., Shih, P.C., Kropczynski, J.: Community informatics as innovation in sociotechnical infrastructures. J. Commun. Inf. 11(2) (2015)

    Google Scholar 

  13. Carroll, J.M., Wu, Y., Shih, P.C., Zheng, S.: Re-appropriating a question/answer system to support dialectical constructivist learning activity. Educ. Technol. Res. Dev. 64(1), 137–156 (2016)

    Article  Google Scholar 

  14. Carroll, J.M., Shih, P.C., Han, K., Kropczynski, J.: Coordinating community cooperation: integrating timebanks and nonprofit volunteering by design. Int. J. Des. 11(1), 51–63 (2017)

    Google Scholar 

  15. Chen, H., Chiang, R.H., Storey, V.C.: Business intelligence and analytics: from big data to big impact. MIS Q. pp. 1165–1188 (2012)

    Google Scholar 

  16. Cohn, J.P.: Citizen science: can volunteers do real research? AIBS Bull. 58(3), 192–197 (2008)

    Google Scholar 

  17. Computer Research Association: Taulbee survey. Comput. Res. News 28(5), 19 (2015)

    Google Scholar 

  18. Dautenhahn, K.: The art of designing socially intelligent agents: Science, fiction, and the human in the loop. Appl. Artif. Intell. 12(7–8), 573–617 (1998)

    Article  Google Scholar 

  19. Demšar, J., Curk, T., Erjavec, A., Gorup, Č., Hočevar, T., Milutinovič, M., Možina, M., Polajnar, M., Toplak, M., Starič, A., et al.: Orange: data mining toolbox in python. J. Mach. Learn. Res. 14(1), 2349–2353 (2013)

    MATH  Google Scholar 

  20. Dickinson, J.L., Zuckerberg, B., Bonter, D.N.: Citizen science as an ecological research tool: challenges and benefits. Annu. Rev. Ecol. Evol. Syst. 41, 149–172 (2010)

    Article  Google Scholar 

  21. Ding, X., Shih, P.C., Gu, N.: Socially embedded work: A study of wheelchair users performing online crowd work in china. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work and Social Computing, pp. 642–654. ACM (2017)

    Google Scholar 

  22. DomíNguez, A., Saenz-De-Navarrete, J., De-Marcos, L., FernáNdez-Sanz, L., PagéS, C., MartíNez-HerráIz, J.J.: Gamifying learning experiences: practical implications and outcomes. Comput. Educ. 63, 380–392 (2013)

    Article  Google Scholar 

  23. Dunbar, J.C., Connelly, C.L., Maestre, J.F., MacLeod, H., Siek, K., Shih, P.C.: Considerations for using the asynchronous remote communities (arc) method in health informatics research. In: Workshop on Interactive Systems in Health Care (2017)

    Google Scholar 

  24. Evans, C., Abrams, E., Reitsma, R., Roux, K., Salmonsen, L., Marra, P.P.: The neighborhood nestwatch program: participant outcomes of a citizen-science ecological research project. Conserv. Biol. 19(3), 589–594 (2005)

    Article  Google Scholar 

  25. Fails, J.A., Olsen Jr, D.R.: Interactive machine learning. In: Proceedings of the 8th international conference on Intelligent user interfaces, pp. 39–45. ACM (2003)

    Google Scholar 

  26. Finholt, T.A., Olson, G.M.: From laboratories to collaboratories: a new organizational form for scientific collaboration. Psychol. Sci. 8(1), 28–36 (1997)

    Article  Google Scholar 

  27. Fischer, G., Giaccardi, E., Ye, Y., Sutcliffe, A.G., Mehandjiev, N.: Meta-design: a manifesto for end-user development. Commun. ACM 47(9), 33–37 (2004)

    Article  Google Scholar 

  28. Gao, G., Min, A., Shih, P.C.: Gendered design bias: Gender differences of in-game character choice and playing style in league of legends. In: Australian Conference on Computer-Human Interaction. ACM Press (2017)

    Google Scholar 

  29. Ghose, A., Ipeirotis, P.G., Li, B.: Designing ranking systems for hotels on travel search engines by mining user-generated and crowdsourced content. Mark. Sci. 31(3), 493–520 (2012)

    Article  Google Scholar 

  30. Goncalves, J., Hosio, S., Ferreira, D., Kostakos, V.: Game of words: tagging places through crowdsourcing on public displays. In: Proceedings of the 2014 Conference on Designing Interactive Systems, pp. 705–714. ACM (2014)

    Google Scholar 

  31. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The weka data mining software: an update. ACM SIGKDD Explor. Newsl. 11(1), 10–18 (2009)

    Article  Google Scholar 

  32. Hamari, J., Koivisto, J., Sarsa, H.: Does gamification work?–a literature review of empirical studies on gamification. In: 2014 47th Hawaii International Conference on System Sciences (HICSS), pp. 3025–3034. IEEE (2014)

    Google Scholar 

  33. Han, K., Cook, K., Shih, P.C.: Exploring effective decision making through human-centered and computational intelligence methods. In: ACM Conference on Human Factors in Computing Systems: Workshop on Human-Centred Machine Learning (2016)

    Google Scholar 

  34. Han, K., Shih, P.C., Bellotti, V., Carroll, J.M.: Timebanking with a smartphone application. In: Collective Intelligence Conference (2014)

    Google Scholar 

  35. Han, K., Shih, P., Carroll, J.M.: Aggregating community information to explore social connections. In: When the City Meets the Citizen Workshop, ICWSM 2013: 7th International AAAI Conference On Weblogs And Social Media, pp. 8–11 (2013)

    Google Scholar 

  36. Han, K., Shih, P.C., Rosson, M.B., Carroll, J.M.: Enhancing community awareness of and participation in local heritage with a mobile application. In: Proceedings of the 17th ACM conference on Computer supported cooperative work and social computing, pp. 1144–1155. ACM (2014)

    Google Scholar 

  37. Han, K., Shih, P.C., Beth Rosson, M., Carroll, J.M.: Understanding local community attachment, engagement and social support networks mediated by mobile technology. Interact. Comput. 28(3), 220–237 (2014)

    Article  Google Scholar 

  38. Han, K., Shih, P.C., Carroll, J.M.: Local news chatter: augmenting community news by aggregating hyperlocal microblog content in a tag cloud. Int. J. Hum.-Comput. Interact. 30(12), 1003–1014 (2014)

    Article  Google Scholar 

  39. Han, K., Shih, P.C., Bellotti, V., Carroll, J.M.: It’s time there was an app for that too: a usability study of mobile timebanking. Int. J. Mob. Hum. Comput. Interact. (IJMHCI) 7(2), 1–22 (2015)

    Article  Google Scholar 

  40. Hanna, S.A., Kropczynski, J., Shih, P.C., Carroll, J.M.: Using a mobile application to encourage community interactions at a local event. In: ACM Richard Tapia Celebration of Diversity in Computing Conference (2015)

    Google Scholar 

  41. Hansson, K., Muller, M., Aitamurto, T., Irani, L., Mazarakis, A., Gupta, N., Ludwig, T.: Crowd dynamics: Exploring conflicts and contradictions in crowdsourcing. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, pp. 3604–3611. ACM (2016)

    Google Scholar 

  42. Hiler, L., Foulk, B., Nippert-Eng, C., Shih, P.C.: Detecting biological samples using olfactory sensors. In: Animal Behavior Conference (2017)

    Google Scholar 

  43. Hill, C., Corbett, C., St Rose, A.: Why so few? Women in science, technology, engineering, and mathematics. ERIC (2010)

    Google Scholar 

  44. House, W.: President obama to announce major expansion of educate to innovate campaign to improve science, technology, engineering and math (stem) education. Office of the Press Secretary (2010)

    Google Scholar 

  45. Howe, J.: The rise of crowdsourcing. Wired Mag. 14(6), 1–4 (2006)

    Google Scholar 

  46. Humphreys, P.: Extending ourselves: Computational science, empiricism, and scientific method. Oxford University Press (2004)

    Chapter  Google Scholar 

  47. Hunter, J.D.: Matplotlib: a 2d graphics environment. Comput. Sci. Eng. 9(3), 90–95 (2007)

    Article  Google Scholar 

  48. Hutchins, E.L., Hollan, J.D., Norman, D.A.: Direct manipulation interfaces. Hum.-Comput. Interact. 1(4), 311–338 (1985)

    Article  Google Scholar 

  49. Ide, N., Véronis, J.: Text encoding initiative: Background and contexts, vol. 29. Springer Science and Business Media (1995)

    Google Scholar 

  50. Irani, L.C., Silberman, M.: Turkopticon: Interrupting worker invisibility in amazon mechanical turk. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 611–620. ACM (2013)

    Google Scholar 

  51. Irwin, A.: Citizen Science: A Study of People, Expertise and Sustainable Development. Psychology Press (1995)

    Google Scholar 

  52. Irwin, A.: Constructing the scientific citizen: science and democracy in the biosciences. Public Underst. Sci. 10(1), 1–18 (2001)

    Article  MathSciNet  Google Scholar 

  53. Jang, J.Y., Han, K., Lee, D., Jia, H., Shih, P.C.: Teens engage more with fewer photos: temporal and comparative analysis on behaviors in instagram. In: Proceedings of the 27th ACM Conference on Hypertext and Social Media, pp. 71–81. ACM (2016)

    Google Scholar 

  54. Jang, J.Y., Han, K., Shih, P.C., Lee, D.: Generation like: comparative characteristics in instagram. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 4039–4042. ACM (2015)

    Google Scholar 

  55. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: Convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM international conference on Multimedia, pp. 675–678. ACM (2014)

    Google Scholar 

  56. Karpathy, A., Fei-Fei, L.: Deep visual-semantic alignments for generating image descriptions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3128–3137 (2015)

    Google Scholar 

  57. Kittur, A., Nickerson, J.V., Bernstein, M., Gerber, E., Shaw, A., Zimmerman, J., Lease, M., Horton, J.: The future of crowd work. In: Proceedings of the 2013 Conference on Computer Supported Cooperative Work, pp. 1301–1318. ACM (2013)

    Google Scholar 

  58. Koehne, B., Shih, P.C., Olson, J.S.: Remote and alone: coping with being the remote member on the team. In: Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work, pp. 1257–1266. ACM (2012)

    Google Scholar 

  59. Kouzes, R.T., Myers, J.D., Wulf, W.A.: Collaboratories: doing science on the internet. Computer 29(8), 40–46 (1996)

    Article  Google Scholar 

  60. Le, J., Edmonds, A., Hester, V., Biewald, L.: Ensuring quality in crowdsourced search relevance evaluation: the effects of training question distribution. In: SIGIR 2010 Workshop on Crowdsourcing for Search Evaluation, vol. 2126 (2010)

    Google Scholar 

  61. Lease, M.: On quality control and machine learning in crowdsourcing. Hum. Comput. 11(11) (2011)

    Google Scholar 

  62. Lieberman, H., Paternò, F., Klann, M., Wulf, V.: End-user development: an emerging paradigm. In: End User development, pp. 1–8. Springer (2006)

    Google Scholar 

  63. Liu, L.S., Shih, P.C., Hayes, G.R.: Barriers to the adoption and use of personal health record systems. In: Proceedings of the 2011 iConference, pp. 363–370. ACM (2011)

    Google Scholar 

  64. Maestre, J.F., MacLeod, H., Connelly, C.L., Dunbar, J.C., Beck, J., Siek, K., Shih, P.C.: Defining through expansion: conducting asynchronous remote communities (arc) research with stigmatized groups. In: ACM Conference on Human Factors in Computing Systems (2018)

    Google Scholar 

  65. Maestre, J.F., Shih, P.C.: Impact of initial trust on video-mediated social support. In: Australian Conference on Computer-Human Interaction. ACM Press (2017)

    Google Scholar 

  66. Malhotra, A., Van Alstyne, M.: The dark side of the sharing economy and how to lighten it. Commun. ACM 57(11), 24–27 (2014)

    Article  Google Scholar 

  67. McAfee, A., Brynjolfsson, E., Davenport, T.H., Patil, D., Barton, D.: Big data: the management revolution. Harv. Bus. Rev. 90(10), 60–68 (2012)

    Google Scholar 

  68. McCoy, C., Shih, P.C.: Teachers as producers of data analytics: a case study of a teacher-focused educational data science program. J. Learn. Anal. 3(3), 193–214 (2016)

    Article  Google Scholar 

  69. McKinney, W., et al.: Data structures for statistical computing in python. In: Proceedings of the 9th Python in Science Conference, vol. 445, pp. 51–56. Austin, TX (2010)

    Google Scholar 

  70. Min, A., Lee, D., Shih, P.C.: Potentials of smart breathalyzer: an interventions for excessive drinking among college students. In: Proceedings of the iConference (2018)

    Chapter  Google Scholar 

  71. Min, A., Shih, P.C.: Exploring new design factors for electronic interventions to prevent college students from excessive drinking by using personal breathalyzers. In: Workshop on Interactive Systems in Health Care (2017)

    Google Scholar 

  72. Nelson, J.K., Shih, P.C.: Companionviz: mediated platform for gauging canine health and enhancing human-pet interactions. Int. J. Hum.-Comput. Stud. 98, 169–178 (2017)

    Article  Google Scholar 

  73. Ongwere, T., Cantor, G., Martin, S.R., Shih, P.C., Clawson, J., Connelly, K.: Too many conditions, too little time: designing technological intervention for patients with type-2 diabetes and discordant chronic comorbidities. In: Workshop on Interactive Systems in Health Care (2017)

    Google Scholar 

  74. Oxford English Dictionary: OED Online (2015)

    Google Scholar 

  75. Parry-Hill, J., Shih, P.C., Mankoff, J., Ashbrook, D.: Understanding volunteer at fabricators: opportunities and challenges in diy-at for others in e-nable. In: ACM Conference on Human Factors in Computing Systems, pp. 6184–6194. ACM (2017)

    Google Scholar 

  76. Pena, J., Shih, P.C., Rosson, M.B.: Instructors as end-user developers: technology usage opportunities in the inverted classroom. In: Handbook of Research on Applied Learning Theory and Design in Modern Education, pp. 560–571. IGI Global (2016)

    Google Scholar 

  77. Peña, J., Shih, P.C., Rosson, M.B.: Scenario-based design of technology to support teaching in inverted classes. IConference 2016 Proceedings (2016)

    Google Scholar 

  78. Rani, P., Liu, C., Sarkar, N., Vanman, E.: An empirical study of machine learning techniques for affect recognition in human-robot interaction. Pattern Anal. Appl. 9(1), 58–69 (2006)

    Article  Google Scholar 

  79. Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N., Eastmond, E., Brennan, K., Millner, A., Rosenbaum, E., Silver, J., Silverman, B., et al.: Scratch: programming for all. Commun. ACM 52(11), 60–67 (2009)

    Article  Google Scholar 

  80. Riesch, H., Potter, C.: Citizen science as seen by scientists: methodological, epistemological and ethical dimensions. Public Underst. Sci. 23(1), 107–120 (2014)

    Article  Google Scholar 

  81. Salehi, N., Irani, L.C., Bernstein, M.S., Alkhatib, A., Ogbe, E., Milland, K., et al.: We are dynamo: overcoming stalling and friction in collective action for crowd workers. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 1621–1630. ACM (2015)

    Google Scholar 

  82. Schirner, G., Erdogmus, D., Chowdhury, K., Padir, T.: The future of human-in-the-loop cyber-physical systems. Computer 46(1), 36–45 (2013)

    Article  Google Scholar 

  83. Schumann, J., Shih, P.C., Redmiles, D.F., Horton, G.: Supporting initial trust in distributed idea generation and idea evaluation. In: Proceedings of the 17th ACM International Conference on Supporting Group Work, pp. 199–208. ACM (2012)

    Google Scholar 

  84. Scriven, M., Paul, R.: Critical thinking as defined by the national council for excellence in critical thinking. In: 8th Annual International Conference on Critical Thinking and Education Reform, Rohnert Park, CA, pp. 25–30 (1987)

    Google Scholar 

  85. Seaborn, K., Fels, D.I.: Gamification in theory and action: a survey. Int. J. Hum.-Comput. Stud. 74, 14–31 (2015)

    Article  Google Scholar 

  86. Shih, P.C., Bellotti, V., Han, K., Carroll, J.M.: Unequal time for unequal value: implications of differing motivations for participation in timebanking. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 1075–1084. ACM (2015)

    Google Scholar 

  87. Shih, P.C., Christena, N.E.: From quantified self to quantified other: engaging the public on promoting animal well-being. In: ACM Conference on Human Factors in Computing Systems: Workshop on HCI Goes to the Zoo (2016)

    Google Scholar 

  88. Shih, P.C., Han, K., Carroll, J.M.: Community incident chatter: informing local incidents by aggregating local news and social media content. In: ISCRAM (2014)

    Google Scholar 

  89. Shih, P.C., Han, K., Carroll, J.M.: Community poll: externalizing public sentiments in social media in a local community context. In: Second AAAI Conference on Human Computation and Crowdsourcing (2014)

    Google Scholar 

  90. Shih, P.C., Han, K., Carroll, J.M.: Engaging community members with digitally curated social media content at an arts festival. In: Digital Heritage, 2015, vol. 1, pp. 321–324. IEEE (2015)

    Google Scholar 

  91. Shih, P.C., Han, K., Poole, E.S., Rosson, M.B., Carroll, J.M.: Use and adoption challenges of wearable activity trackers. IConference 2015 Proceedings (2015)

    Google Scholar 

  92. Shih, P.C., Nguyen, D.H., Hirano, S.H., Redmiles, D.F., Hayes, G.R.: Groupmind: supporting idea generation through a collaborative mind-mapping tool. In: Proceedings of the ACM 2009 International Conference on Supporting Group Work, pp. 139–148. ACM (2009)

    Google Scholar 

  93. Shih, P.C., Olson, G.M.: Using visualization to support idea generation in context. In: ACM Creativity and Cognition Conference Workshop: Creativity and Cognition in Engineering Design. ACM (2009)

    Google Scholar 

  94. Shih, P.C., Venolia, G., Olson, G.M.: Brainstorming under constraints: why software developers brainstorm in groups. In: Proceedings of the 25th BCS Conference on Human-computer Interaction, pp. 74–83. British Computer Society (2011)

    Google Scholar 

  95. Shih, P.C., Han, K., Carroll, J.M.: Using social multimedia content to inform emergency planning of recurring and cyclical events in local communities. J. Homel. Secur. Emerg. Manag. 12(3), 627–652 (2015)

    Google Scholar 

  96. Silvertown, J.: A new dawn for citizen science. Trends Ecol. Evol. 24(9), 467–471 (2009)

    Article  Google Scholar 

  97. Su, N.M., Shih, P.C.: Virtual spectating: hearing beyond the video arcade. In: Proceedings of the 25th BCS Conference on Human-Computer Interaction, pp. 269–278. British Computer Society (2011)

    Google Scholar 

  98. Terveen, L., Hill, W.: Beyond recommender systems: helping people help each other. HCI New Millenn. 1(2001), 487–509 (2001)

    Google Scholar 

  99. Tinati, R., Van Kleek, M., Simperl, E., Luczak-Rösch, M., Simpson, R., Shadbolt, N.: Designing for citizen data analysis: A cross-sectional case study of a multi-domain citizen science platform. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 4069–4078. ACM (2015)

    Google Scholar 

  100. Tomlinson, B., Ross, J., Andre, P., Baumer, E., Patterson, D., Corneli, J., Mahaux, M., Nobarany, S., Lazzari, M., Penzenstadler, B., et al.: Massively distributed authorship of academic papers. In: CHI’12 Extended Abstracts on Human Factors in Computing Systems, pp. 11–20. ACM (2012)

    Google Scholar 

  101. Vaghela, S.J.D., Shih, P.C.: Walksafe: College campus safety app. In: International Conference on Information Systems for Crisis Response and Management (2018)

    Google Scholar 

  102. Vinyals, O., Toshev, A., Bengio, S., Erhan, D.: Show and tell: a neural image caption generator. In: Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on, pp. 3156–3164. IEEE (2015)

    Google Scholar 

  103. Walt, S.v.d., Colbert, S.C., Varoquaux, G.: The numpy array: a structure for efficient numerical computation. Comput. Sci. Eng. 13(2), 22–30 (2011)

    Article  Google Scholar 

  104. Wang, J., Shih, P.C., Carroll, J.M.: Life after weight loss: design implications for community-based long-term weight management. Comput. Support. Coop. Work (CSCW) 24(4), 353–384 (2015)

    Article  Google Scholar 

  105. Wang, J., Shih, P.C., Carroll, J.M.: Revisiting linuss law: benefits and challenges of open source software peer review. Int. J. Hum.-Comput. Stud. 77, 52–65 (2015)

    Article  Google Scholar 

  106. Wang, J., Shih, P.C., Wu, Y., Carroll, J.M.: Comparative case studies of open source software peer review practices. Inf. Softw. Technol. 67, 1–12 (2015)

    Article  Google Scholar 

  107. Ware, M., Frank, E., Holmes, G., Hall, M., Witten, I.H.: Interactive machine learning: letting users build classifiers. Int. J. Hum.-Comput. Stud. 55(3), 281–292 (2001)

    Article  Google Scholar 

  108. Warschauer, M., Matuchniak, T.: New technology and digital worlds: analyzing evidence of equity in access, use, and outcomes. Rev. Res. Educ. 34(1), 179–225 (2010)

    Article  Google Scholar 

  109. Wiggins, A., Crowston, K.: From conservation to crowdsourcing: a typology of citizen science. In: 2011 44th Hawaii International Conference on System Sciences (HICSS), pp. 1–10. IEEE (2011)

    Google Scholar 

  110. Williams, K., Li, L., Khabsa, M., Wu, J., Shih, P.C., Giles, C.L.: A web service for scholarly big data information extraction. In: 2014 IEEE International Conference on Web Services (ICWS), pp. 105–112. IEEE (2014)

    Google Scholar 

  111. Wing, J.: Computational thinking benefits society. 40th Anniversary Blog of Social Issues in Computing 2014 (2014)

    Google Scholar 

  112. Wing, J.M.: Computational thinking. Commun. ACM 49(3), 33–35 (2006)

    Article  Google Scholar 

  113. Witten, I.H., Frank, E., Hall, M.A., Pal, C.J.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann (2016)

    Google Scholar 

  114. Wu, Y., Kropczynski, J., Shih, P.C., Carroll, J.M.: Exploring the ecosystem of software developers on github and other platforms. In: Proceedings of the Companion Publication of the 17th ACM Conference on Computer Supported Cooperative Work and Social Computing, pp. 265–268. ACM (2014)

    Google Scholar 

  115. Wu, Y., Shih, P.C., Carroll, J.M.: Design for supporting dialectical constructivist learning activities. In: EDULEARN14 Proceedings, pp. 4156–4164. IATED (2014)

    Google Scholar 

  116. Yang, S., Chen, P.Y., Shih, P.C., Bardzell, J., Bardzell, S.: Cross-strait frenemies: Chinese netizens vpn in to facebook taiwan. Proceedings of the ACM on Human-Computer Interaction 1(CSCW), Article–115 (2017)

    Google Scholar 

  117. Zheng, S., Rosson, M.B., Shih, P.C., Carroll, J.M.: Designing moocs as interactive places for collaborative learning. In: Proceedings of the Second (2015) ACM Conference on Learning@ Scale, pp. 343–346. ACM (2015)

    Google Scholar 

  118. Zheng, S., Rosson, M.B., Shih, P.C., Carroll, J.M.: Understanding student motivation, behaviors and perceptions in moocs. In: Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work and Social Computing, pp. 1882–1895. ACM (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Informatics, School of Informatics Computing and Engineering, Indiana University Bloomington, Bloomington, IN, 47408, USA

    Patrick C. Shih

Authors
  1. Patrick C. Shih
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Patrick C. Shih .

Editor information

Editors and Affiliations

  1. DATA61, CSIRO, Eveleigh, New South Wales, Australia

    Jianlong Zhou

  2. DATA61, CSIRO, Eveleigh, New South Wales, Australia

    Fang Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Shih, P.C. (2018). Beyond Human-in-the-Loop: Empowering End-Users with Transparent Machine Learning. In: Zhou, J., Chen, F. (eds) Human and Machine Learning. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-319-90403-0_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-90403-0_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-90402-3

  • Online ISBN: 978-3-319-90403-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation