Tracking Deictic Gestures over Large Interactive Surfaces



In a collaborative environment, non-verbal communication elements carry important contents. These contents are partially or completely lost in remote collaboration. This paper presents a system to address this issue by tracking pointing gestures, the main non-verbal communication element prevalent in such meetings. The setup employs a touchscreen tabletop computer system for representing the visual content of the meeting, together with three motion trackers for tracking the pointing gestures.


Computer supported collaborative work Non-verbal communication Remote collaboration Tabletop computing Gesture detection 



This work was done within the project CR21I2L_138601 of the Swiss National Science Foundation.


  1. Aitenbichler, Erwin, Jussi Kangasharju, and Max Mühlhäuser (2007). MundoCore: A Light-weight Infrastructure for Pervasive Computing. Journal of Pervasive and Mobile Computing, vol. 3, no. 4, pp. 332–361.CrossRefGoogle Scholar
  2. Berndtsson, Johan, and Maria Normark (1999). The coordinative functions of flight strips: Air Traffic Control work revisited. GROUP’99: International Conference on Supporting Group Work, Phoenix, Arizona, 1417 November 1999, New York: ACM Press, pp. 101–110.Google Scholar
  3. Bly, Sara (1988). A Use of Drawing Surfaces in Different Collaborative Settings. CSCW’88 Proceedings of the 1988 ACM conference on Computer-supported cooperative work, New York: ACM Press, pp. 250–258.Google Scholar
  4. Bly, Sara A., and Scott L. Minnemann (1990). Commune: A Shared Drawing Surface. COCS’90 Proceedings of the ACM SIGOIS and IEEE CS TC-OA conference on Office information systems, New York: ACM Press, pp. 184–192.Google Scholar
  5. Gaver, William W., Abigail Sellen, Christian Heath, and Paul Luff (1993). One is Not Enough: Multiple Views in a Media Space. CHI’93 Proceedings of the INTERACT’93 and CHI’93 Conference on Human Factors in Computing Systems, New York: ACM Press, pp. 335–341.Google Scholar
  6. Gross, Tom (2013). Supporting effortless coordination: 25 years of awareness research. Computer Supported Cooperative Work (CSCW): The Journal of Collaborative Computing and Work Practices, vol. 22, nos. 4–6, 1. August 2013, pp. 425–474.Google Scholar
  7. Gutwin, Carl, and Saul Greenberg (2002). A descriptive framework of workspace awareness for real-time groupware. Computer Supported Cooperative Work. The Journal of Collaborative Computing, vol. 11, nos. 3–4, pp. 411–446.CrossRefGoogle Scholar
  8. Ishii, Hiroshi, and Minoru Kobayashi (1992). ClearBoard: A Seamless Medium for Shared Drawing and Conversation with Eye Contact. CHI’92 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York: ACM Press, pp. 525–532.Google Scholar
  9. Kirk, David., Tom Rodden, and Danea Stanton Fraser (2007). Turn It This Way: Grounding Collaborative Action with Remote Gestures. CHI’07 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York: ACM Press, pp. 1039–1048.Google Scholar
  10. Kirk, David, and Danea Stanton Fraser (2006). Comparing Remote Gesture Technologies for Supporting Collaborative Physical Tasks. CHI’06 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York: ACM Press, pp.1191–1200.Google Scholar
  11. Krueger, Myron (1983). Artificial Reality, Addison-Wesley ProfessionalGoogle Scholar
  12. Kunz, Andreas, Thomas Nescher, and Martin Küchler (2010). CollaBoard: A Novel Interactive Whiteboard for Remote Collaboration with People on Content. CW 2010 Proceeding of the International Conference on Cyberworlds, 20. – 22. October 2010, IEEE, pp. 430–437.Google Scholar
  13. Kunz, Andreas, Ali Alavi, and Philipp Sinn (2014). Integrating Pointing Gesture Detection for Enhancing Brainstorming Meetings Using Kinect and PixelSense. 8th International Conference on Digital Enterprise Technology, Stuttgart, Germany, 25. – 28. March 2014, pp. 1–8.Google Scholar
  14. Kuzuoka, Hideaki, Jun Yamashita, Keiichi Yamazaki, and Akiko Yamazaki (1999). Agora: A Remote Collaboration System that Enables Mutual Monitoring. CHI EA’99 CHI’99 Extended Abstracts on Human Factors in Computing Systems, New York: ACM Press, 15. – 20. May 1999, pp. 190–191.Google Scholar
  15. LEAP Motion. Accessed 10. March 2014.
  16. Louwerse, Max, and Adrian Bangerter (2005). Focusing Attention with Deictic Gestures and Linguistic Expressions. Proceedings of the 27th Annual Meeting of the Cognitive Science Society, 23 pp. 1331–1336.Google Scholar
  17. Nescher, Thomas, and Andreas Kunz (2011). An Interactive Whiteboard for Immersive Telecollaboration. The Visual Computer: International Journal of Computer Graphics - Special Issue on CYBERWORLDS 2010, vol. 27, no. 4, April 2011, New York: Springer, pp. 311–320.Google Scholar
  18. Schmidt, Kjeld (2002). The problem with “awareness”: Introductory remarks on “Awareness in CSCW”. Computer Supported Cooperative Work (CSCW): The Journal of Collaborative Computing, vol. 11, nos. 3–4, pp. 285–298.CrossRefGoogle Scholar
  19. Stotts, David, Jason McC. Smith, and D. Jen (2003). The Vis-a-Vid Transparent Video Facetop. Proceedings of the UIST 2003, pp. 57–58.Google Scholar
  20. Sutton, Robert, and Andrew Hargadon (1996). Brainstorming Groups in Context: Effectiveness in a Product Design Film. Administrative Science Quarterly, vol. 41, no. 4, pp. 685–718.CrossRefGoogle Scholar
  21. Tang, John C. (1991). Findings from Observational Studies of Collaborative Work. International Journal Man–machine Studies, vol. 34, no. 2, pp. 143–160.CrossRefGoogle Scholar
  22. Tang, John C., and Scott L. Minneman (1991). VideoDraw: A Video Interface for Collaborative Drawing. ACM Transactions on Information Systems (TOIS) - Special issue on computer—human interaction, vol. 9, no. 2, New York: ACM Press, April 1991, pp. 170–184.Google Scholar
  23. Tang, John C., and Scott L. Minneman (1991). VideoWhiteboard: Video Shadows to Support Remote Collaboration. CHI’91 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York: ACM Press, pp. 315–322.Google Scholar
  24. Tang, Anthony, Carman Neustaedter, and Saul Greenberg (2004). VideoArms: Supporting Remote Embodiment in Groupware. Video Proceedings of the ACM Conference on Computer Supported Cooperative Work - ACM CSCW’04, New York: ACM Press.Google Scholar
  25. Tang, Anthony, Carman Neustaedter, and Saul Green (2006). VideoArms: Embodiments for Mixed Presence Groupware. Proceedings of HCI 2006, London: Springer, pp. 85–102.Google Scholar
  26. Wellner, Pierre (1993). Interaction with Paper on the Digital Desk. Communications of the ACM - Special issue on computer augmented environments: back to the real world, vol. 36, no. 7, New York: ACM Press, July 1993, pp. 87–96.Google Scholar
  27. Wellner, Pierre, and Stephen Freeman (1993). The Double Digital Desk: Shared Editing of Paper Documents. XEROX Euro PARC Technical Report EPC-93-108.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  1. 1.Innovation Center Virtual RealityETH ZurichZurichSwitzerland

Personalised recommendations