An Observational Study of Simultaneous and Sequential Interactions in Co-located Collaboration

  • Shuo NiuEmail author
  • D. Scott McCrickard
  • Steve Harrison
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10515)


Large-scale multi-touch displays provide highly interactive spaces for small group activities. These devices feature the ability to detect concurrent touch inputs, which enable multiple co-located collaborators to manipulate virtual spaces in myriad ways. This paper explores two types of interaction, simultaneous and sequential, with regard to how people engage in shared virtual space during a collaborative ideation task. Our findings suggest that the two types of interactions present different patterns in both temporal and spatial dimensions. Sequential interaction is the major interaction technique, while the simultaneous interaction is actively used for information exploration and manipulating objects in personal space. Observation of semantic actions suggests that some behaviors are preferably performed in turns, while others are used more in simultaneous manner. The relationship between the two interaction types with regard to different collaboration factors is explored. We share lessons learned from the study and suggest design implications for multi-user touch interfaces.


Simultaneous Sequential Turn-taking Multi-user touch display Collaboration Coupling Interaction stage Territoriality Semantic action 


  1. 1.
    Bortolaso, C., Oskamp, M., Phillips, W.G., Gutwin, C., Graham, T.C.N.: The effect of view techniques on collaboration and awareness in tabletop map-based tasks. In: BT - Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces, ITS 2014, Dresden, Germany, 16–19 November 2014 (2014)Google Scholar
  2. 2.
    Brandl, P., Haller, M., Hurnaus, M., Lugmayr, V., Oberngruber, J., Oster, C., Schafleitner, C., Billinghurst, M.: An adaptable rear-projection screen using digital pens and hand gestures. In: 17th International Conference on Artificial Reality and Telexistence (ICAT 2007), pp. 49–54. IEEE (2007)Google Scholar
  3. 3.
    Chewar, C.M., McCrickard, D.S., Sutcliffe, A.G.: Unpacking critical parameters for interface design. In: Proceedings of the 2004 Conference on Designing Interactive Systems Processes, Practices, Methods, and Techniques - DIS 2004, p. 279. ACM, Cambridge (2004)Google Scholar
  4. 4.
    Doucette, A., Gutwin, C., Mandryk, R.L., Nacenta, M.A., Sharma, S.: Sometimes when we touch: how arm embodiments change reaching and collaboration on digital tables. In: Proceedings of the 2013 Conference on Computer Supported Cooperative Work (CSCW 2013), pp. 193–202. ACM, San Antonio (2013)Google Scholar
  5. 5.
    Endert, A., Fiaux, P., North, C.: Semantic interaction for visual text analytics. In: Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems - CHI 2012, pp. 473–482. ACM, Austin (2012)Google Scholar
  6. 6.
    Evans, A.C., Wobbrock, J.O., Davis, K.: Modeling collaboration patterns on an interactive tabletop in a classroom setting. In: Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, pp. 860–871. ACM, San Francisco (2016)Google Scholar
  7. 7.
    Fan, M., Antle, A.N., Neustaedter, C., Wise, A.F.: Exploring how a co-dependent tangible tool design supports collaboration in a tabletop activity. In: Proceedings of the 18th International Conference on Supporting Group Work (GROUP 2014), pp. 81–90. ACM, Sanibel (2014)Google Scholar
  8. 8.
    Frisch, M., Heydekorn, J., Dachselt, R.: Investigating multi-touch and pen gestures for diagram editing on interactive surfaces. In: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS 2009), pp. 149–156. ACM, Banff (2009)Google Scholar
  9. 9.
    Geyer, F., Pfeil, U., Hochtl, A., Budzinski, J., Reiterer, H.: Designing reality-based interfaces for creative group work (2011)Google Scholar
  10. 10.
    Gutwin, C., Greenberg, S.: A descriptive framework of workspace awareness for real-time groupware. Comput. Support. Coop. Work. 11(3–4), 411–446 (2002). (CSCW 2002)Google Scholar
  11. 11.
    Gutwin, C., Greenberg, S.: Design for individuals, design for groups: tradeoffs between power and workspace awareness. In: Proceedings of the Conference on Computer-Supported Cooperative Work, pp. 207–216. ACM, Seattle (1998)Google Scholar
  12. 12.
    Halskov, K., Dalsgård, P.: Inspiration card workshops. In: Proceedings of the 6th Conference on Designing Interactive Systems, pp. 2–11. ACM, University Park, Pennsylvania (2006)Google Scholar
  13. 13.
    Harris, A., Rick, J., Bonnett, V., Yuill, N., Fleck, R., Marshall, P., Rogers, Y.: Around the table: are multiple-touch surfaces better than single-touch for children’s collaborative interactions? In: Proceedings of the 9th International Conference on Computer Supported Collaborative Learning, ISLS, Rhodes, pp. 335–344 (2009)Google Scholar
  14. 14.
    Hartmann, B., Morris, M.R., Benko, H., Wilson, A.D.: Pictionaire: supporting collaborative design work by integrating physical and digital artifacts (2010)Google Scholar
  15. 15.
    Hinrichs, U., Carpendale, S.: Gestures in the wild: studying multi-touch gesture sequences on interactive tabletop exhibits. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2011), pp. 3023–3032. ACM, Vancouver (2011)Google Scholar
  16. 16.
    Hornecker, E., Marshall, P., Dalton, N.S., Rogers, Y.: Collaboration and interference: awareness with mice or touch input. In: Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work, pp. 167–176. ACM, New York (2008)Google Scholar
  17. 17.
    Hunter, S., Maes, P.: WordPlay: a table-top interface for collaborative brainstorming and decision making. In: Proceedings of IEEE Tabletops and Interactive Surfaces, pp. 2–5. IEEE (2008)Google Scholar
  18. 18.
    Isenberg, P., Fisher, D., Paul, S.A., Morris, M.R., Inkpen, K., Czerwinski, M., Paul, S.A., Morris, M.R., Inkpen, K., Czerwinski, M.: An exploratory study of co-located collaborative visual analytics around a tabletop display. In: 2010 IEEE Symposium on Visual Analytics Science and Technology (VAST), pp. 179–186. IEEE, Salt Lake City (2010)Google Scholar
  19. 19.
    Isenberg, P., Tang, A., Carpendale, S.: An exploratory study of visual information analysis. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1217–1226. ACM, Florence (2008)Google Scholar
  20. 20.
    Jakobsen, M.R., Hornbæk, K.: Up close and personal: collaborative work on a high-resolution multitouch wall display. ACM Trans. Comput. Interact. 21(2), 1–34 (2014)CrossRefGoogle Scholar
  21. 21.
    Klemmer, S.R., Newman, M.W., Farrell, R., Bilezikjian, M., Landay, J.A.: The designers’ outpost: a tangible interface for collaborative web site design. In: Proceedings of 14th Annual ACM Symposium User Interface Software Technology (UIST 2001), vol. 3, no. 2, pp. 1–10 (2001)Google Scholar
  22. 22.
    Marshall, P., Morris, R., Rogers, Y., Kreitmayer, S., Davies, M.: Rethinking “multi-user”: an in-the-wild study of how groups approach a walk-up-and-use tabletop interface. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2011), pp. 3033–3042. ACM, Vancouver (2011)Google Scholar
  23. 23.
    McCrickard, D.S., Abel, T.D., Scarpa, A., Wang, Y., Niu, S.: Collaborative design for young children with autism: design tools and a user study. In: 2015 International Conference on Collaboration Technologies and Systems (CTS), pp. 175–182. IEEE (2015)Google Scholar
  24. 24.
    Morris, M.R., Huang, A., Paepcke, A., Winograd, T.: Cooperative gestures: multi-user gestural interactions for co-located groupware. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2006), pp. 1201–1210. ACM, Montréal (2006)Google Scholar
  25. 25.
    Morris, M.R., Lombardo, J., Wigdor, D.: WeSearch: supporting collaborative search and sensemaking on a tabletop display. In: Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work (CSCW 2010), pp. 401–410. ACM, Savannah (2010)Google Scholar
  26. 26.
    North, C., Dwyer, T., Lee, B., Fisher, D., Isenberg, P., Robertson, G., Inkpen, K.: Understanding multi-touch manipulation for surface computing. In: Human-Computer Interaction (INTERACT 2009), pp. 236–249. Springer, Heidelberg (2009)Google Scholar
  27. 27.
    Peltonen, P., Kurvinen, E., Salovaara, A., Jacucci, G., Ilmonen, T., Evans, J., Oulasvirta, A., Saarikko, P.: It’s Mine, Don’t Touch!: interactions at a large multi-touch display in a city centre. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2008), pp. 1285–1294. ACM, Florence (2008)Google Scholar
  28. 28.
    Piper, A.M., O’Brien, E., Morris, M.R., Winograd, T.: SIDES: a cooperative tabletop computer game for social skills development. In: Proceedings of the 2006 ACM Conference on Computer Supported Cooperative Work (CSCW 2006), pp. 1–10. ACM, Banff (2006)Google Scholar
  29. 29.
    Pirolli, P., Card, S.: The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In: Proceedings of International Conference on Intelligence Analysis, pp. 2–4 (2005)Google Scholar
  30. 30.
    Robinson, A.C.: Collaborative synthesis of visual analytic results. In: IEEE Symposium on Visual Analytics Science and Technology (VAST 2008), pp. 67–74. IEEE, Columbus (2008)Google Scholar
  31. 31.
    Rogers, Y., Hazlewood, W., Blevis, E., Lim, Y.-K.: Finger talk: collaborative decision-making using talk and fingertip interaction around a tabletop display. In: CHI 2004 Extended Abstracts on Human Factors in Computing Systems (CHI EA 2004), pp. 1271–1274. ACM, Vienna (2004)Google Scholar
  32. 32.
    Russell, D.M., Drews, C., Sue, A.: Social aspects of using large public interactive displays for collaboration. In: Borriello, G., Holmquist, L. (eds.) Proceedings of the 4th International Conference on Ubiquitous Computing (UbiComp 2002), vol. 2498, pp. 229–236. Springer, Heidelberg (2002). doi: 10.1007/3-540-45809-3_18
  33. 33.
    Schon, D.A.: The Reflective Practitioner - How Professionals Think in Action, vol. 5126, pp. 1–8 (1983). ReviewGoogle Scholar
  34. 34.
    Scott, S.D., Carpendale, M.S.T., Inkpen, K.M., Sheelagh, M., Carpendale, T., Inkpen, K.M.: Territoriality in collaborative tabletop workspaces. In: Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work (CSCW 2004), pp. 294–303. ACM, Chicago (2004)Google Scholar
  35. 35.
    Scott, S.D., Grant, K.D., Mandryk, R.L.: System guidelines for co-located, collaborative work on a tabletop display. In: Kuutti, K. et al. (eds.) Proceedings of the Eighth European Conference on Computer Supported Cooperative Work (ECSCW 2003), pp. 159–178. Springer, Heidelberg (2003). doi: 10.1007/978-94-010-0068-0_9
  36. 36.
    Niu, S., McCrickard, D.S., Nguyen, S.M.: Learning with interactive tabletop displays. In: 2016 IEEE Frontiers in Education Conference (FIE), pp. 1–9. IEEE (2016)Google Scholar
  37. 37.
    Spence, S.H., Rapee, R., McDonald, C., Ingram, M.: The structure of anxiety symptoms among preschoolers. Behav. Res. Ther. 39(11), 1293–1316 (2001)CrossRefGoogle Scholar
  38. 38.
    Tang, A., Tory, M., Po, B., Neumann, P., Carpendale, S.: Collaborative coupling over tabletop displays. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2006), pp. 1181–1190. ACM, Montréal (2006)Google Scholar
  39. 39.
    Tholander, J., Normark, M., Rossitto, C.: Understanding agency in interaction design materials. In: Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems - CHI 2012, p. 2499. ACM, Austin (2012)Google Scholar
  40. 40.
    Tominski, C., Gladisch, S., Kister, U., Dachselt, R., Schumann, H.: A survey on interactive lenses in visualization. In: Eurographics Conference Visualization (2014)Google Scholar
  41. 41.
    Tse, E., Greenberg, S., Shen, C., Forlines, C., Kodama, R.: Exploring true multi-user multimodal interaction over a digital table. In: Proceedings of the 7th ACM Conference on Designing Interactive Systems, pp. 109–118. ACM, Cape Town (2008)Google Scholar
  42. 42.
    Vogt, K., Bradel, L., Andrews, C., North, C., Endert, A., Hutchings, D.: Co-located collaborative sensemaking on a large high-resolution display with multiple input devices. In: Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M. (eds.) INTERACT 2011. LNCS, vol. 6947, pp. 589–604. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-23771-3_44 CrossRefGoogle Scholar
  43. 43.
    Wahid, S., Branham, S.M., Cairco, L., McCrickard, D.S., Harrison, S.: Picking up artifacts: storyboarding as a gateway to reuse. In: Gross, T., Gulliksen, J., Kotzé, P., Oestreicher, L., Palanque, P., Prates, R.O., Winckler, M. (eds.) INTERACT 2009. LNCS, vol. 5727, pp. 528–541. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-03658-3_57 CrossRefGoogle Scholar
  44. 44.
    Wu, M., Balakrishnan, R.: Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays. In: Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology (UIST 2003), pp. 193–202. ACM, Vancouver (2003)Google Scholar
  45. 45.
    Yuill, N., Rogers, Y.: Mechanisms for collaboration: a design and evaluation framework for multi-user interfaces. ACM Trans. Comput. Interact. 19(1), 1:1–1:25 (2012)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2017

Authors and Affiliations

  • Shuo Niu
    • 1
    Email author
  • D. Scott McCrickard
    • 1
  • Steve Harrison
    • 1
  1. 1.Computer Science DepartmentVirginia TechBlacksburgUSA

Personalised recommendations