Interface Metaphors for Interactive Machine Learning

  • Robert J. JasperEmail author
  • Leslie M. Blaha
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10284)


To promote more interactive and dynamic machine learning, we revisit the notion of user-interface metaphors. User-interface metaphors provide intuitive constructs for supporting user needs through interface design elements. A user-interface metaphor provides a visual or action pattern that leverages a user’s knowledge of another domain. Metaphors suggest both the visual representations that should be used in a display as well as the interactions that should be afforded to the user. We argue that user-interface metaphors can also offer a method of extracting interaction-based user feedback for use in machine learning. Metaphors offer indirect, context-based information that can be used in addition to explicit user inputs, such as user-provided labels. Implicit information from user interactions with metaphors can augment explicit user input for active learning paradigms. Or it might be leveraged in systems where explicit user inputs are more challenging to obtain. Each interaction with the metaphor provides an opportunity to gather data and learn. We argue this approach is especially important in streaming applications, where we desire machine learning systems that can adapt to dynamic, changing data.


Interactive machine learning User interface Interface metaphor 



The research described in this document was sponsored the U.S. Department of Energy (DOE) through the Analysis in Motion Initiative at Pacific Northwest National Laboratory. The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Government.


  1. 1.
    Amershi, S., Cakmak, M., Knox, W.B., Kulesza, T.: Power to the people: the role of humans in interactive machine learning. AI Mag. 35(4), 105–120 (2014)Google Scholar
  2. 2.
    Apple Computer Inc: Macintosh Human Interface Guidelines. Addison-Wesley, Boston (1992)Google Scholar
  3. 3.
    Barr, P.: User-interface metaphors in theory and practice. Master’s thesis, University of Wellington, Victoria (2003)Google Scholar
  4. 4.
    Barr, P., Biddle, R., Noble, J.: A taxonomy of user-interface metaphors. In: Proceedings of the SIGCHI-NZ Symposium on Computer-Human Interaction, pp. 25–30. ACM (2002)Google Scholar
  5. 5.
    Barr, P., Biddle, R., Noble, J.: A semiotic model of user-interface metaphor. In: Liu, K. (ed.) Virtual, Distributed and Flexible Organisations, pp. 189–215. Springer, Dordrecht (2004)Google Scholar
  6. 6.
    Barr, P., Khaled, R., Noble, J., Biddle, R.: A taxonomic analysis of user-interface metaphors in the microsoft office project gallery. In: Proceedings of the Sixth Australasian Conference on User interface, vol. 40, pp. 109–117. Australian Computer Society, Inc. (2005)Google Scholar
  7. 7.
    Brockerhoff, R.: User interface metaphors. In: MacHack Conference proceeding (2000)Google Scholar
  8. 8.
    Brown, E.T.: Learning from users’ interactions with visual analytics systems. Ph.D. thesis, Tufts University (2015)Google Scholar
  9. 9.
    Brown, E.T., Ottley, A., Zhao, H., Lin, Q., Souvenir, R., Endert, A., Chang, R.: Finding waldo: learning about users from their interactions. IEEE Trans. Vis. Comput. Graph. 20(12), 1663–1672 (2014)CrossRefGoogle Scholar
  10. 10.
    Erickson, T.D.: Working with interface metaphors. In: Baecker, R.M. (ed.) Readings in Human-Computer Interaction: Toward the Year 2000, vol. 11, pp. 147–151 (1995)Google Scholar
  11. 11.
    Fails, J.A., Olsen, D.R.: Interactive machine learning. In: Proceedings of the 8th International Conference on Intelligent User Interfaces, pp. 39–45. ACM (2003)Google Scholar
  12. 12.
    Gentner, D.: Structure-mapping: a theoretical framework for analogy. Cogn. Sci. 7(2), 155–170 (1983)CrossRefGoogle Scholar
  13. 13.
    Gentner, D., Holyoak, K.J.: Reasoning and learning by analogy: introduction. Am. Psychol. 52(1), 32–34 (1997)CrossRefGoogle Scholar
  14. 14.
    Gentner, D., Holyoak, K.J., Kokinov, B.N.: The Analogical Mind: Perspectives from Cognitive Science. MIT Press, Cambridge (2001)Google Scholar
  15. 15.
    Horvitz, E.: Principles of mixed-initiative user interfaces. In: CHI (1999)Google Scholar
  16. 16.
    Lakoff, G., Johnson, M.: Metaphors We Live By. University of Chicago Press, Chicago (1980)Google Scholar
  17. 17.
    Nielsen, J., Molich, R.: Heuristic evaluation of user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (1990)Google Scholar
  18. 18.
    Norman, D.A.: Things that Make us Smart: Defending Human Attributes in the Age of the Machine. Basic Books, New York (1993)Google Scholar
  19. 19.
    Patterson, R.E., Blaha, L.M., Grinstein, G.G., Liggett, K.K., Kaveney, D.E., Sheldon, K.C., Havig, P.R., Moore, J.A.: A human cognition framework for information visualization. Comput. Graph. 42, 42–58 (2014)CrossRefGoogle Scholar
  20. 20.
    Saket, B., Kim, H., Brown, E.T., Endert, A.: Visualization by demonstration: an interaction paradigm for visual data exploration. IEEE Trans. Visual Comput. Graph. 23(1), 331–340 (2017)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Pacific Northwest National LaboratoryRichlandUSA

Personalised recommendations