ToCoPlay: Graphical Multi-touch Interaction for Composing and Playing Music

  • Sean Lynch
  • Miguel A. Nacenta
  • Sheelagh Carpendale
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6948)


With the advent of electronic music and computers, the human-sound interface is liberated from the specific physical constraints of traditional instruments, which means that we can design musical interfaces that provide arbitrary mappings between human actions and sound generation. This freedom has resulted in a wealth of new tools for electronic music generation that expand the limits of expression, as exemplified by projects such as Reactable and Bricktable. In this paper we present ToCoPlay, an interface that further explores the design space of collaborative, multi-touch music creation systems. ToCoPlay is unique in several respects: it allows creators to dynamically transition between the roles of composer and performer, it takes advantage of a flexible spatial mapping between a musical piece and the graphical interface elements that represent it, and it applies current and traditional interface interaction techniques for the creation of music.


Multi-touch collaboration composition music musical instrument 

Supplementary material

Electronic Supplementary material (38,591 KB)


  1. 1.
    Akustich, (last accessed January 2011)
  2. 2.
    Bischof, M., Conradi, B., Lachenmaier, P., Linde, K., Meier, M., Pötzl, André, E.: Xenakis: combining tangible interaction with probability-based musical composition. In: Proceedings of TEI 2008, pp. 121–124. ACM, New York (2008)CrossRefGoogle Scholar
  3. 3.
    Bonabeau, E., Dorigo, M., Theraulaz, G.: Swarm Intelligence: From Natural to Artificial Systems. Oxford University Press, Oxford (1999)zbMATHGoogle Scholar
  4. 4.
    Blaine, T., Perkis, T.: The Jam-O-Drum interactive music system: a study in interaction design. In: Proceedings of DIS 2000, pp. 165–173. ACM, New York (2000)Google Scholar
  5. 5.
    Condio, (last accessed January 2011)
  6. 6.
  7. 7.
    Davidson, P.L., Han, J.L.: Synthesis and control on large scale multi-touch sensing displays. In: Proceedings of NIME 2006, IRCAM, pp. 216–219 (2006)Google Scholar
  8. 8.
    Fyfe, L., Lynch, S., Hull, C., Carpendale, S.: SurfaceMusic: Mapping Virtual Touch-based Instruments to Physical Models. In: Proceedings NIME 2010, pp. 360–363 (2010)Google Scholar
  9. 9.
    Garage Band, (last accessed January 2011)
  10. 10.
    Gardner, M.: On cellular automata, self-replication, the Garden of Eden and the game life. Scientific American 224(4), 112–117 (1971)CrossRefGoogle Scholar
  11. 11.
    Hochenbaum, J., Vallis, O.: Bricktable: A Musical Tangible Multi-Touch Interface. In: Proceedings of Berlin Open Conference 2009, Berlin, Germany (2009)Google Scholar
  12. 12.
    Hochenbaum, J., Vallis, O., Diakopolous, D., Murphy, J., Kapur, A.: Designing Expressive Musical Interfaces for Tabletop Surfaces. In: Proceedings of NIME 2010, pp. 315–318 (2010)Google Scholar
  13. 13.
    Hauert, S., Reichmuth, D.: Instant City, (last accessed January 2011)
  14. 14.
    Jordá, S.: Sonigraphical instruments: from FMOL to the reacTable. In: Proceedings of NIME 2003, pp. 70–76 (2003)Google Scholar
  15. 15.
    Jordà, S., Geiger, G., Alonso, M., Kaltenbrunner, M.: The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces. In: Proceedings of TEI 2007, pp. 139–146. ACM, New York (2007)CrossRefGoogle Scholar
  16. 16.
    Max/MSP, (last accessedJanuary 2011)
  17. 17.
    Microsoft Corporation. Microsoft Surface, (last accessedJanuary 2011)
  18. 18.
    Krueger, M.W., Gionfriddo, T., Hinrichsen, K.: VIDEOPLACE - An Artificial Reality. In: Proceedings ofCHI 1985, pp. 35–40. ACM, New York (1985)Google Scholar
  19. 19.
    Newton-Dunn, H., Nakano, H., Gibson, J.: Block jam: a tangible interface for interactive music. In: Proceedings of NIME 2003, pp. 170–177 (2003)Google Scholar
  20. 20.
    Noteput, (last accessed January 2009)
  21. 21.
    Patten, J., Recht, B., Ishii, H.: Audiopad: a tag-based interface for musical performance. In: Proceedings of NIME 2002, pp. 1–6 (2002)Google Scholar
  22. 22.
    Scott, S.D., Carpendale, M.S., Inkpen, K.M.: Territoriality in collaborative tabletop workspaces. In: Proceedings of CSCW 2004, pp. 294–303. ACM, New York (2004)CrossRefGoogle Scholar
  23. 23.
    Sound Storm, (last accessed January 2011)
  24. 24.
    Stavness, I., Gluck, J., Vilhan, L., Fels, S.: The MUSICtable: a map-based ubiquitous system for social interaction with a digital music collection. In: Kishino, F., Kitamura, Y., Kato, H., Nagata, N. (eds.) ICEC 2005. LNCS, vol. 3711, pp. 291–302. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  25. 25.
    Stereotronic Multi-synth Orchestra, (last accessed January 2011)
  26. 26.
    Taylor, S., Izadi, S., Kirk, D., Harper, R., Garcia-Mendoza, A.: Turning the tables: an interactive surface for vjing. In: Proceedings CHI 2009, pp. 1251–1254. ACM, New York (2009)Google Scholar
  27. 27.
    Villar, N., Lindsay, A.T., Gellersen, H.: Pin & Play & Perform: a rearrangeable interface for musical composition and performance. In: Proceedings of NIME 2005, pp. 188-191 (2005)Google Scholar

Copyright information

© IFIP International Federation for Information Processing 2011

Authors and Affiliations

  • Sean Lynch
    • 1
  • Miguel A. Nacenta
    • 1
  • Sheelagh Carpendale
    • 1
  1. 1.University of CalgaryCalgaryCanada

Personalised recommendations