Characterizing Deixis over Surfaces to Improve Remote Embodiments

  • Aaron Genest
  • Carl Gutwin
Conference paper


Deictic gestures are ubiquitous when people work over tables and whiteboards, but when collaboration occurs across distributed surfaces, the embodiments used to represent other members of the group often fail to convey the details of these gestures. Although both gestures and embodiments have been well studied, there is still little information available to groupware designers about what components and characteristics of deictic gesture are most important for conveying meaning through remote embodiments. To provide this information, we conducted three observational studies in which we recorded and analysed more than 450 deictic gestures. We considered four issues that are important for the design of embodiments on surfaces: what parts of the body are used to produce a deictic gesture, what atomic movements make up deixis, where gestures occur in the space above the surface, and what other characteristics deictic gestures exhibit in addition to pointing. Our observations provide a new design understanding of deictic gestures. We use our results to identify the limitations of current embodiment techniques in supporting deixis, and to propose new hybrid designs that can better represent the range of behavior seen in real-world settings.


Hesitation Atom Computer Support Cooperative Work Indicative Atom Collaborative Virtual Environment Temporal Trace 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Apperly, M., McLeod, L., Masoodian, M., Paine, L., Phillips, M., Rogers, B., and Thomson, K (2003): ‘Use of video shadow for small group interaction awareness on a large interactive display surface’, Proceedings of the 4th Australasian user interface conference, vol. 18, 81–90.Google Scholar
  2. Bekker, M. M., Olson, J. S., & Olson, G. M. (1995): ‘Analysis of gestures in face-to-face design teams provides guidance for how to use groupware in design’, Proceedings of the 1st conference on Designing Interactive Systems, 157–166.Google Scholar
  3. Benford, S., Bowers, J., Fahlén, L.E., Greenhalgh, C., and Snowdon, D. (1995): ‘User embodiment in collaborative virtual environments’, Proceedings of the SIGCHI conference on Human Factors in Computing Systems, 242–249.Google Scholar
  4. Clark, H.H. (2003): ‘Pointing and placing’, in S. Kita (eds): Pointing: Where language, culture, and cognition meet, Erlbaum, Hillsdale, NJ, 2003, 243–268.Google Scholar
  5. Fraser, M., McCarthy, M.R., Shaukat, M, and Smith, P. (2007): ‘Seconds Matter: Improving Distributed Coordination by Tracking and Visualizing Display Trajectories’, Proceedings of the SIGCHI conference on Human Factors in Computing Systems, 1303–1312.Google Scholar
  6. Greenberg, S., Gutwin, C., and Roseman, M. (1996): ‘Semantic telepointers for groupware’, Proceedings of the 6 th Australian conference on Computer-Human Interaction, 54–61.Google Scholar
  7. Gutwin, C., & Greenberg, S. (1998): ‘Effects of awareness support on groupware usability’, Proceedings of the SIGCHI conference on Human Factors in Computing Systems, 511–518.Google Scholar
  8. Gutwin, C., & Greenberg, S. (2000): ‘The mechanics of collaboration: Developing low cost usability evaluation methods for shared workspaces’, Proceedings of the 9 th IEEE workshop on Enabling Technologies: Infrastructure for Collaborative Enterprises, 98–103.Google Scholar
  9. Gutwin, C., & Penner, R. (2002): ‘Improving interpretation of remote gestures with telepointer traces’, Proceedings of the ACM conference on Computer Supported Cooperative Work, 49–57.Google Scholar
  10. Hayne, S., Pendergast, M., and Greenberg, S. (1994): ‘Implementing Gesturing with Cursors in Group Support Systems’, Journal of Management Information Systems, 10, 3, 1994, 43–62.Google Scholar
  11. Hindmarsh, J., Fraser, M., Heath, C., Benford, S., and Greenhalgh, C. (2000): ‘Object-focused interaction in collaborative virtual environments’, ACM Transactions on Computer-Human Interaction, vol. 7, no. 4, 477–509.CrossRefGoogle Scholar
  12. Hindmarsh, J., & Heath, C. (2000): ‘Embodied reference: A study of deixis in workplace interaction’, Journal of Pragmatics, vol. 32, no. 12, 1855–1878.CrossRefGoogle Scholar
  13. Izadi, S., Agarwal, A., Criminisi, A., Winn, J., Blake, A., Fitzgibbon, A. (2007): ‘C-Slate: A Multi-Touch and Object Recognition System for Remote Collaboration using Horizontal Surfaces’, 2nd Workshop on Horizontal Interactive Human-Computer Systems, 3–10.Google Scholar
  14. Kettebekov, S., & Sharma, R. (2000): ‘Understanding gestures in multimodal human computer interaction’, International Journal on Artificial Intelligence Tools, vol. 9, no. 2, 205–223.CrossRefGoogle Scholar
  15. Kirk, D., Crabtree, A., & Rodden, T. (2005): ‘Ways of the hands’, Proceedings of the European conference on Computer Supported Cooperative Work, 1–21.Google Scholar
  16. Kirk, D., Rodden, T., & Fraser, D. S. (2007): ‘Turn it this way: grounding collaborative action with remote gestures’, Proceedings of the SIGCHI conference on Human Factors in Computing Systems, 1039–1048.Google Scholar
  17. Krauss, R.M. and Fussell, S. R. (1990): ‘Mutual Knowledge and Communicative Effectiveness’, in J. Galegher, R. E. Kraut, and C. Egido (eds.) Intellectual Teamwork: Social and Technological Foundations of Cooperative Work, Erlbaum, Hillsdale, NJ, 111–145.Google Scholar
  18. Li, J., Wessels, A., Alem, L., & Stitzlein, C. (2007): ‘Exploring interface with representation of gesture for remote collaboration’, Proceedings of the 19th Australasian conference on Computer-Human Interaction: Entertaining User Interfaces, 179–182.Google Scholar
  19. McNeill, D. (1992): Hand and mind, University of Chicago Press, Chicago.Google Scholar
  20. Ou, J., Chen, X., Fussell, S. R., & Yang, J. (2003): ‘DOVE: Drawing over video environment’, Proceedings of the 11 th ACM international conference on Multimedia, 100–101.Google Scholar
  21. Ou, J., Fussell, S. R., Chen, X., Setlock, L. D., & Yang, J. (2003): ‘Gestural communication over video stream: supporting multimodal interaction for remote collaborative physical tasks’, Proc. of the 5 th conference on Multimodal Interfaces, 242–249.Google Scholar
  22. Pinelle, D., Nacenta, M., Gutwin, C., & Stach, T. (2008): ‘The effects of co-present embodiments on awareness and collaboration in tabletop groupware’, Proceedings of Graphics Interface, 1–8.Google Scholar
  23. Stach, T., Gutwin, C., Pinelle, D., & Irani, P. (2007): ‘Improving recognition and characterization in groupware with rich embodiments’, Proceedings of the SIGCHI conference on Human Factors in Computing Systems, 11–20.Google Scholar
  24. Tang, A., Neustaedter, C., & Greenberg, S. (2007): ‘Videoarms: embodiments for mixed presence groupware’, in N. Bryan-Kinns, A. Blanford, P. Curzon and L. Niggy (eds.): People and Computers XX, Springer, London, 2007, 85–102.CrossRefGoogle Scholar
  25. Tang, A., Pahud, M., Inkpen, K., Benko, H., Tang, J. C., & Buxton, B. (2010): ‘Three’s company: understanding communication channels in three-way distributed collaboration’, Proceedings of ACM conference on Computer Supported Cooperative Work, 271–280.Google Scholar
  26. Tang, J. C., & Minneman, S. (1991a): ‘VideoWhiteboard: video shadows to support remote collaboration’, Proceedings of the SIGCHI conference on Human Factors in Computer Systems, 315–322.Google Scholar
  27. Tang, J., and Minneman, S. (1991b): Videodraw: a video interface for collaborative drawing. Transansactions on Information Systems 9, 2 (April 1991), 170–184.CrossRefGoogle Scholar
  28. Wigdor, D., Williams, S., Cronin, M., Levy, R., White, K. (2009): ‘Ripples: utilizing per-contact visualizations to improve user interaction with touch displays,’ Proceedings of the ACM symposium on User Interface Software and Technology, 3–12.Google Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  1. 1.University of SaskatchewanSaskatoonCanada

Personalised recommendations