Skip to main content
SpringerLink
Log in

Diagram Editing on Interactive Displays Using Multi-touch and Pen Gestures

  • Conference paper
Diagrammatic Representation and Inference (Diagrams 2010)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 6170))

Included in the following conference series:

Abstract

Creating and editing graphs and node-link diagrams by means of digital tools are crucial activities in domains such as software or business process modeling. However, these tools have several drawbacks with regard to interaction techniques and usability. In order to address these issues, we investigate the promising combination of pen and multi-touch input on interactive displays. In this work, we contribute a gesture set to make the interaction with diagrams more efficient and effective by means of pen and hand gestures. Thereby, two prevalent mental models are supported: structural editing and sketching. The gesture set is based on the results of a previous pilot study asking users for suggestions to accomplish diagram editing tasks on tabletops. In this paper, we provide a careful analysis of the resulting user-elicited gestures. We propose solutions to resolve ambiguities within this gesture collection and discuss design decisions for a comprehensible diagram editor. We also present the multi-touch and pen gesture set as a fully implemented prototype for diagram editing on interactive surfaces.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Anoto Group AB, http://www.anoto.com/

  2. Brandl, P., Forlines, C., Wigdor, D., Haller, M., Shen, C.: Combining and measuring the benefits of bimanual pen and direct-touch interaction on horizontal interfaces. In: Proc. of AVI 2008, pp. 154–161. ACM, New York (2008)

    Google Scholar 

  3. Brandl, P., Haller, M., Hurnaus, M., Lugmayr, V., Oberngruber, J., Oster, C., Schafleitner, C., Billinghurst, M.: An Adaptable Rear-Projection Screen Using Digital Pens And Hand Gestures. In: Proc. of ICAT 2007, pp. 49–54. IEEE Computer Society, Los Alamitos (2007)

    Google Scholar 

  4. Cao, X., Wilson, A.D., Balakrishnan, R., Hinckley, K., Hudson, S.E.: ShapeTouch: Leveraging Contact Shape on Interactive Surfaces. In: Proc. of TABLETOP 2008, pp. 139–146 (2008)

    Google Scholar 

  5. Chen, Q., Grundy, J., Hosking, J.: An e-whiteboard application to support early design-stage sketching of UML diagrams. In: Proc. of HCC 2003, pp. 219–226. IEEE, Los Alamitos (2003)

    Google Scholar 

  6. Cherubini, M., Venolia, G., DeLine, R., Ko, A.J.: Let’s go to the whiteboard: how and why software developers use drawings. In: Proc. of CHI 2007, pp. 557–566. ACM, New York (2007)

    Google Scholar 

  7. Community Core Vision, NUI Group, http://ccv.nuigroup.com/

  8. Damm, C.H., Hansen, K.M., Thomsen, M.: Tool support for cooperative object-oriented design: gesture based modeling on an electronic whiteboard. In: Proc. of CHI 2000, pp. 518–525. ACM, New York (2000)

    Google Scholar 

  9. Dekel, U.: Supporting distributed software design meetings: what can we learn from co-located meetings? In: Proc. of HSSE 2005, pp. 1–7. ACM, New York (2005)

    Google Scholar 

  10. Dietz, P., Leigh, D.: DiamondTouch: a multi-user touch technology. In: Proc. of UIST 2001, pp. 219–226. ACM, New York (2001)

    Google Scholar 

  11. Dwyer, T., Lee, B., Fisher, D., Quinn, K.I., Isenberg, P., Robertson, G., North, C.: A Comparison of User-Generated and Automatic Graph Layouts. IEEE Trans. on Visualization and Computer Graphics 15(6), 961–968 (2009)

    Article  Google Scholar 

  12. Epps, J., Lichman, S., Wu, M.: A study of hand shape use in tabletop gesture interaction. In: CHI 2006 Ext. Abstracts, pp. 748–753. ACM, New York (2006)

    Google Scholar 

  13. Freeman, D., Benko, H., Morris, M.R., Wigdor, D.: ShadowGuides: Visualizations for In-Situ Learning of Multi-Touch and Whole-Hand Gestures. In: Proc. ACM ITS 2009, pp. 183–190. ACM, New York (2009)

    Google Scholar 

  14. Frisch, M., Heydekorn, J., Dachselt, R.: Investigating Multi-Touch and Pen Gestures for Diagram Editing on Interactive Surfaces. In: Proc. of ACM IST 2009, pp. 167–174. ACM, New York (2009)

    Google Scholar 

  15. Grundy, J., Hosking, J.: Supporting Generic Sketching-Based Input of Diagrams in a Domain-Specific Visual Language Meta-Tool. In: Pro. of ICSE 2007, pp. 282–291 (2007)

    Google Scholar 

  16. Guiard, Y.: Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model. The Journal of Motor Behavior 19(4), 486–517 (1987)

    Article  Google Scholar 

  17. Hammond, T., Davis, R.: Tahuti: a geometrical sketch recognition system for UML class diagrams. In: ACM SIGGRAPH 2006 Courses. ACM, New York (2006)

    Google Scholar 

  18. Han, J.Y.: Low-cost multi-touch sensing through frustrated total internal reflection. In: Proc. of UIST 2005, pp. 115–118. ACM, New York (2005)

    Google Scholar 

  19. Leitner, J., Powell, J., Brandl, P., Seifried, T., Haller, M., Dorray, B., To, P.: Flux: a tilting multi-touch and pen based surface. In: CHI 2009 Ext. Abstracts, pp. 3211–3216. ACM, New York (2009)

    Google Scholar 

  20. Malik, S., Ranjan, A., Balakrishnan, R.: Interacting with large displays from a distance with vision-tracked multi-finger gestural input. In: Proc. UIST 2005, pp. 43–52. ACM, New York (2005)

    Google Scholar 

  21. Micire, M., Desai, M., Courtemanche, A., Tsui, K., Yanco, H.: Analysis of Natural Gestures for Controlling Robot Teams on Multi-touch Tabletop Surfaces. In: Proc. ACM IST 2009. ACM, New York (2009)

    Google Scholar 

  22. Microsoft Surface, http://www.microsoft.com/surface/

  23. Moran, T.P., Chiu, P., van Melle, W.: Pen-based interaction techniques for organizing material on an electronic whiteboard. In: Proc. UIST 1997, pp. 45–54. ACM, New York (1997)

    Google Scholar 

  24. Nielsen, M., Störring, M., Moeslund, T.B., Granum, E.: A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 409–420. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  25. Rekimoto, J.: SmartSkin: an infrastructure for freehand manipulation on interactive surfaces. In: Proc. CHI 2002, pp. 113–120. ACM, New York (2002)

    Google Scholar 

  26. Ringel, M., Ryall, K., Shen, C., Forlines, C., Vernier, F.: Release, relocate, reorient, resize: fluid techniques for document sharing on multi-user interactive tables. In: Ext. Abstracts CHI 2004, pp. 1441–1444. ACM, New York (2004)

    Google Scholar 

  27. TUIO Protocol, http://www.tuio.org/

  28. Wobbrock, J.O., Morris, M.R., Wilson, A.D.: User-defined gestures for surface computing. In: Proc. of CHI 2009, pp. 1083–1092. ACM, New York (2009)

    Google Scholar 

  29. Wu, M., Balakrishnan, R.: Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays. In: Proc. of UIST 2003, pp. 193–202. ACM, New York (2003)

    Google Scholar 

  30. Wu, M., Shen, C., Ryall, K., Forlines, C., Balakrishnan, R.: Gesture Registration, Relaxation, and Reuse for Multi-Point Direct-Touch Surfaces. In: Proc. TABLETOP 2006, pp. 185–192. IEEE, Los Alamitos (2006)

    Google Scholar 

  31. Yee, K.: Two-handed interaction on a tablet display. In: Ext. Abs. CHI 2004, pp. 1493–1496. ACM, New York (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. User Interface & Software Engineering Group, Otto-von-Guericke-University Magdeburg, 39106, Magdeburg, Germany

    Mathias Frisch, Jens Heydekorn & Raimund Dachselt

Authors
  1. Mathias Frisch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jens Heydekorn
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Raimund Dachselt
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Interactive Computing, Design & Intelligence Laboratory, Georgia Institute of Technology,, 85 Fifth Street, NW 30308, Atlanta, GA, USA

    Ashok K. Goel

  2. Computer Laboratory, University of Cambridge, JJ Thomson Avenue, CB3 0FD, Cambridge, UK

    Mateja Jamnik

  3. Department of Computer Science & Software Engineering, Auburn University, 345 W. Magnolia Street, 36849-5347, Auburn, AL, USA

    N. Hari Narayanan

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Frisch, M., Heydekorn, J., Dachselt, R. (2010). Diagram Editing on Interactive Displays Using Multi-touch and Pen Gestures. In: Goel, A.K., Jamnik, M., Narayanan, N.H. (eds) Diagrammatic Representation and Inference. Diagrams 2010. Lecture Notes in Computer Science(), vol 6170. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-14600-8_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-14600-8_18

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-14599-5

  • Online ISBN: 978-3-642-14600-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation