Advertisement

Understanding Music Interaction, and Why It Matters

  • Simon HollandEmail author
  • Tom Mudd
  • Katie Wilkie-McKenna
  • Andrew McPherson
  • Marcelo M. Wanderley
Chapter
Part of the Springer Series on Cultural Computing book series (SSCC)

Abstract

This is the introductory chapter of a book dedicated to new research in, and emerging new understandings of, music and human-computer interaction—known for short as music interaction. Music interaction research plays a key role in innovative approaches to diverse musical activities, including performance, composition, education, analysis, production and collaborative music making. Music interaction is pivotal in new research directions in a range of activities, including audience participation, interaction between music and dancers, tools for algorithmic music, music video games, audio games, turntablism and live coding. More generally, music provides a powerful source of challenges and new ideas for human-computer interaction (HCI). This introductory chapter reviews the relationship between music and human-computer interaction and outlines research themes and issues that emerge from the collected work of researchers and practitioners in this book.

Notes

Acknowledgements

The editors would like to thank workshop co-organisers not represented by chapters: Sile O’Modhrain, Michael Gurevich and Andrew Johnston. We would also like to thank workshop participants not otherwise represented in this book who made valued contributions to discussions at the workshop: Ge Wang, Gian-Marco Schmid, Jordi Janer, Jeff Gregorio, Sam Ferguson, Frédéric Bevilacqua, Edgar Berdahl and Mathieu Barthet. Finally, we would like to thank Helen Desmond at Springer.

References

  1. Angelis V, Holland S, Clayton M, Upton PJ (2013) Testing a computational model of rhythm perception using polyrhythmic stimuli. J New Music Res 42(1)CrossRefGoogle Scholar
  2. Arom S (1991) African polyphony and polyrhythm: musical structure and methodology. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  3. Baecker R (1969) Picture driven animation. In: Proceedings of the AFIPS spring joint computer conference, vol 34, pp 273–288Google Scholar
  4. Balzano GJ (1980) The group-theoretic description of 12-fold and microtonal pitch systems. Comput Music J 4(4). Winter 1980CrossRefGoogle Scholar
  5. Beaudouin-Lafon M (2000) Instrumental Interaction: an interaction model for designing post-WIMP user interfaces. In: Proceedings ACM CHI ‘00, pp 446–453Google Scholar
  6. Blanchard C, Burgess S, Harvill Y, Lanier J, Lasko A, Oberman M, Teitel M (1990) Reality built for two: a virtual reality tool. ACM SIGGRAPH Comput Graph 24(2):35–36. ACMCrossRefGoogle Scholar
  7. Bongers B (2000) Physical interfaces in the electronic arts. Trends in gestural control of music, pp 41–70Google Scholar
  8. Bouwer A, Holland S, Dalgleish M (2013) The haptic bracelets: learning multi-limb rhythm skills from haptic stimuli while reading. In: Holland S, Wilkie K, Mulholland P, Seago A (eds) Music and human-computer interaction. Cultural Computing. Springer, LondonCrossRefGoogle Scholar
  9. Brown DE (1991) Human universals. McGraw-Hill, New YorkGoogle Scholar
  10. Brown C, Paine G (2019) A case study in collaborative learning via DMIs for participatory music: interactive Tango Milonga. In: Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM (eds) New directions in music and human-computer interaction. Springer, London. ISBN 978-3-319-92069-6Google Scholar
  11. Brown S, Martinez MJ, Parsons LM (2004) Passive music listening spontaneously engages limbic and paralimbic systems. NeuroReport 15:2033–2037CrossRefGoogle Scholar
  12. Buxton W (2008) My vision isn’t my vision: making a career out of getting back to where I started. In: Erickson T, McDonald D (eds) HCI remixed: reflections on works that have influenced the HCI community. MIT Press, Cambridge, MA, pp 7–12Google Scholar
  13. Camci A, Cakmak C, Forbes AG (2019) Applying game mechanics to networked music HCI applications. In: Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM (eds) New directions in music and human-computer interaction. Springer, London. ISBN 978-3-319-92069-6Google Scholar
  14. Church L, Nash C, Blackwell A (2010) Liveness in notation use: from music to programming. Psychology of Programming Interest Group PPIG 2010Google Scholar
  15. Cross I (2016) The nature of music and its evolution. In: Hallam S, Cross I, Thaut M (eds) Oxford handbook of music psychology, 2nd edn. Oxford, Oxford University Press, pp 3–17Google Scholar
  16. Darrow A (2006) The role of music in deaf culture: deaf students’ perception of emotion in music. J Music Ther 43(1):2–15. 1 March 2006,  https://doi.org/10.1093/jmt/43.1.2CrossRefGoogle Scholar
  17. DʼErrico F, Henshilwood C, Lawson G, Vanhaeren M, Tillier A-M, Soressi M et al (2003) Archaeological evidence for the emergence of language, symbolism, and music—an alternative multidisciplinary perspective. J World Prehistory 17(1):1–70Google Scholar
  18. Dourish P (2004) Where the action is: the foundations of embodied interaction. MIT PressGoogle Scholar
  19. Duchesne-Guillemin M (1963) Découverte D’une Gamme Babylonienne. Revue De Musicologie 49(126):3–17CrossRefGoogle Scholar
  20. Engelbart DC, English WK (1968) A research center for augmenting human intellect. In: Proceedings of the December 9–11, 1968, fall joint computer conference, part I (AFIPS ‘68 (Fall, part I)). ACM, New York, NY, USA, pp 395–410Google Scholar
  21. Fallman D (2003) Design-oriented human-computer Interaction. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’03. ACM, New York, NY, USA, pp 225–232.  https://doi.org/10.1145/642611.642652
  22. Fenichel E (2002) The musical lives of babies and families. Zero Three 23(1). National Center for Infants, Toddlers and Families, Washington DCGoogle Scholar
  23. Fitch WT (2006) The biology and evolution of music: a comparative perspective. Cognition 100:173–215CrossRefGoogle Scholar
  24. Garcia J, Tsandilas T, Agon C, Mackay W (2012) Interactive paper substrates to support musical creation. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, pp 1825–1828Google Scholar
  25. Hamilton R (2019) Mediated musical interactions in virtual environments. In: Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM (eds) New directions in music and human-computer interaction. Springer, London. ISBN 978-3-319-92069-6Google Scholar
  26. Hargreaves D, North A (1997) The social psychology of music. Oxford University Press, OxfordGoogle Scholar
  27. Harrison S, Tatar D, Sengers P (2007) The three paradigms of HCI. In: Alt. Chi. session at the SIGCHI conference on human factors in computing systems, San Jose, California, USA, pp 1–18Google Scholar
  28. Hein E, Srinivasan S (2019) The Groove Pizza. In: Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM (eds) New directions in music and human-computer interaction. Springer, London. ISBN 978-3-319-92069-6Google Scholar
  29. Hödl O, Kayali F, Fitzpatrick G, Holland S (2019) TMAP design cards for technology-mediated audience participation in live music. In: Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM (eds) New directions in music and human-computer interaction. Springer, London. ISBN 978-3-319-92069-6Google Scholar
  30. Holland S (1989) Artificial intelligence, education and music. PhD thesis, The Open University, Milton Keynes, UKGoogle Scholar
  31. Holland S (2000) Artificial intelligence in music education: a critical review, pp 239–274. In: Miranda ER (ed) Artificial intelligence in music education: a critical review. Readings in music and artificial intelligence. Routledge, pp 249–284Google Scholar
  32. Holland S, Fiebrink R (2019) Machine learning, music and creativity: an interview with Rebecca Fiebrink. In: Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM (eds) New directions in music and human-computer interaction. Springer, London. ISBN 978-3-319-92069-6Google Scholar
  33. Holland S, Wilkie K, Bouwer A, Dalgleish M, Mulholland P (2011) Whole body interaction in abstract domains. In: England D (ed) Whole body interaction. Human–computer interaction series, Springer Verlag, London. ISBN 978-0-85729-432-6CrossRefGoogle Scholar
  34. Holland S, Wilkie K, Mulholland P, Seago A (2013) Music interaction: understanding music and human-computer interaction. In: Holland S, Wilkie K, Mulholland P, Seago A (eds) Music and human-computer interaction, pp 1–28CrossRefGoogle Scholar
  35. Holland S, Wright RL, Wing A, Crevoisier T, Hödl O, Canelli M (2014) A Gait Rehabilitatin pilot study using tactile cueing following Hemiparetic Stroke. In: Proceedings of the 8th International Conference on Pervasive Computing Technologies for Healthcare Pervasive Health 2014, pp 402–405Google Scholar
  36. Holland S, McPherson AP, Mackay WE, Wanderley MM, Gurevich MD, Mudd TW, O’Modhrain S, Wilkie KL, Malloch J, Garcia J, Johnston A (2016) Music and HCI. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA’16). ACM, New York, NY, USA, 3339–3346Google Scholar
  37. Holland S, Bouwer A, Hödl O (2018) Haptics for the development of fundamental rhythm skills, including multi-limb coordination. In: Papetti S, Saitis C (eds) Musical haptics. Springer series on touch and haptic systems. Springer International Publishing, pp 215–237Google Scholar
  38. Honing H, Ladinig O, Winkler I, Háden G (2009) Is beat induction innate or learned? Probing emergent meter perception in adults and newborns using event-related brain potentials (ERP). Ann N Y Acad Sci 1169:93–96CrossRefGoogle Scholar
  39. Hook J, Schofield G, Taylor R, Bartindale T, McCarthy J, Wright P (2012) Exploring HCI’s relationship with liveness. CHI ‘12 extended abstracts on human factors in computing systems (CHI EA ‘12). ACM, New York, NY, USA, pp 2771–2774Google Scholar
  40. Hunt A, Wanderley MM, Kirk R (2000) Towards a model for instrumental mapping in expert musical interaction. In: ICMCGoogle Scholar
  41. Hutchinson H, Mackay W, Westerlund B, Bederson BB, Druin A, Plaisant C, Beaudouin-Lafon M et al (2003) Technology probes: inspiring design for and with families. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, pp 17–24Google Scholar
  42. Justus T, Jamshed B (2002) Music perception and cognition. In: Stevens’ handbook of experimental psychology. Wiley, pp 453–492Google Scholar
  43. Koetsier T (2001) On the prehistory of programmable machines: musical automata, looms, calculators. Mech Mach Theory 36:589–603CrossRefGoogle Scholar
  44. Lanier J (1989) Personal communication with Simon Holland and other attendees at 1989. In: Nato advanced research workshop on multimedia interface design in education, Lucca, ItalyGoogle Scholar
  45. Longuet-Higgins HC (1976) Perception of melodies. Nature 263:646–653CrossRefGoogle Scholar
  46. Mackay WE (2000) Responding to cognitive overload: co-adaptation between users and technology. Intellectica 30(1):177–193Google Scholar
  47. Magnusson T (2010) Designing constraints: composing and performing with digital musical systems. Comput Music J 34(4):62–73CrossRefGoogle Scholar
  48. Malloch J, Garcia J, Wanderley MM, Mackay WE, Beaudouin-Lafon M, Huot S (2019) A design WorkBench for interactive music systems. In: Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM (eds) New directions in music and human-computer interaction. Springer, London. ISBN 978-3-319-92069-6Google Scholar
  49. Martin CP, Gardner H (2019) Free-improvised rehearsal-as-research for musical HCI. In: Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM (eds) New directions in music and human-computer interaction. Springer, London. ISBN 978-3-319-92069-6Google Scholar
  50. McPherson A, Benford S (2019) Music, design and ethnography: an interview with Steve Benford. In: Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM (eds) New directions in music and human-computer interaction. Springer, London. ISBN 978-3-319-92069-6Google Scholar
  51. McPherson A, Verplank B (2019) The poetry of strange connections: an interview with Bill Verplank. In: Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM (eds) New directions in music and human-computer interaction. Springer, London. ISBN 978-3-319-92069-6Google Scholar
  52. McPherson A, Morreale F, Harrison J (2019) Musical instruments for novices: comparing NIME, HCI and Crowdfunding approaches. In: Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM (eds) New directions in music and human-computer interaction. Springer, London. ISBN 978-3-319-92069-6Google Scholar
  53. Mehr SA et al (2016) For 5-month olds melodies are social. Psychol Sci 27:486–501CrossRefGoogle Scholar
  54. Milne A (2019) XronoMorph: investigating paths through rhythmic space. In: Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM (eds) New directions in music and human-computer interaction. Springer, London. ISBN 978-3-319-92069-6Google Scholar
  55. Milne AJ, Holland S (2016) Empirically testing Tonnetz, voice-leading, and spectral models of perceived triadic distance. J Math Music 10(1):59–85MathSciNetCrossRefGoogle Scholar
  56. Milne AJ, Carlé M, Sethares WA, Noll T, Holland S (2011) Scratching the scale labyrinth. International conference on mathematics and computation in music. Springer, Berlin, Heidelberg, pp 180–195CrossRefGoogle Scholar
  57. Mithen S (2006) The ‘Singing Neanderthals’: the origins of music, language, mind and body. Camb Archaeol J 16:97–112CrossRefGoogle Scholar
  58. Mudd T (2019) Material-oriented musical interactions. In: Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM (eds) New directions in music and human-computer interaction. Springer, London. ISBN 978-3-319-92069-6Google Scholar
  59. Mudd T, Andersen K (2019) Making as research: an interview with Kristina Andersen. In: Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM (eds) New directions in music and human-computer interaction. Springer, London. ISBN 978-3-319-92069-6Google Scholar
  60. NRC (1970) From handel to haydn to the headless musician, science dimension 2(3), June 1970. http://ieee.ca/millennium/electronic_music/em_headless.html
  61. NRC (1971). The music machine. 11 minute 16 mm film produced by the National Research Council of Canada. http://www.billbuxton.com
  62. Pennycook BW (1985) Computer-music interfaces: a survey. ACM Comput Surv 17(2):267–289CrossRefGoogle Scholar
  63. Poupyrev I, Lyons MJ, Fels S, Blaine TB (2001) New interfaces for musical expression. In: Workshop proposal for SIGCHI 2001, Seattle, WA. http://www.nime.org/2001/docs/proposal.pdf
  64. Prechtl A, Milne AJ, Holland S, Laney R, Sharp DB (2009) A midi sequencer that widens access to the compositional possibilities of novel tunings. Comput Music J 36(1):42–54CrossRefGoogle Scholar
  65. Rasmussen J (1986) Information processing and human-machine interaction: an approach to cognitive engineering. Elsevier Science Inc, New York, USAGoogle Scholar
  66. Salimpoor VN et al (2013) Interactions between the nucleus accumbens and auditory cortices predict music reward value. Science 340:216–219CrossRefGoogle Scholar
  67. Standley JM (2011) Efficacy of music therapy for premature infants in the neonatal intensive care unit: a meta-analysis. Arch Dis Childhood Fetal Neonatal Ed 96:Fa52CrossRefGoogle Scholar
  68. Sutherland IE (1964) Sketch pad a man-machine graphical communication system. In: Proceedings of the SHARE design automation workshop. ACM, pp 6–329Google Scholar
  69. Tanaka A (2000) Musical performance practice on sensor-based instruments. Trends Gestural Control Music 13(389–405):284Google Scholar
  70. Tanaka A (2019) Embodied musical interaction: body physiology, cross modality, and sonic experience. In: Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM (eds) New directions in music and human-computer interaction. Springer, London. ISBN 978-3-319-92069-6Google Scholar
  71. Tanimoto SL (1990) VIVA: a visual language for image processing. J Vis Lang Comput 1(2):127–139CrossRefGoogle Scholar
  72. Toussaint GT (2013) The geometry of musical rhythm: what makes a “Good” rhythm good?. CRC Press, Boca RatonzbMATHGoogle Scholar
  73. Tuuri K, Jaana P, Pirhonen A (2017) Who controls who? Embodied control within human–technology choreographies. Interact Comput 29(4):494–511Google Scholar
  74. Ungvary T, Vertegaal R (2000) Cognition and physicality in musical cyberinstruments. In: Trends in gestural control of music, pp 371–386Google Scholar
  75. Vuilleumier P, Trost W (2015) Music and emotions: from enchantment to entrainment. Neurosci Mus V Cogn Stimul Rehabilit 1337:212–222.  https://doi.org/10.1111/nyas.12676CrossRefGoogle Scholar
  76. Wallis I, Ingalls T, Campana E, Vuong C (2013) Amateur musicians, long-term engagement, and HCI. In: Music and human-computer interaction. Springer, London, pp 49–66CrossRefGoogle Scholar
  77. Wanderley MM, Battier M (2000) Trends in gestural control of music. IRCAM, Centre PompidouGoogle Scholar
  78. Wanderley MM, Mackay W (2019) HCI, music and art: an interview with Wendy Mackay. In: Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM (eds) New directions in music and human-computer interaction. Springer, London. ISBN 978-3-319-92069-6Google Scholar
  79. Wilkie K, Holland S, Mulholland P (2009) Evaluating musical software using conceptual metaphors. In: Blackwell A (ed) Proceedings of the 23rd British HCI group annual conference on people and computers. pp 232–237. ISSN 1477-9358Google Scholar
  80. Wilkie K, Holland S, Mulholland P (2010) What can the language of musicians tell us about music interaction design? Comput Music J Winter 34(4):34–48. Massachusetts Institute of TechnologyCrossRefGoogle Scholar
  81. Xenakis I (1992) Formalized music: thought and mathematics in composition, second, revised English edition, with additional material translated by Sharon Kanach. Harmonologia Series No. 6. Pendragon Press, Stuyvesant, NY. ISBN 1-57647-079-2Google Scholar
  82. Yuksel BF, Oleson KB, Chang R, Jacob RJK (2019) Detecting and adapting to users’ cognitive and affective state to develop intelligent musical interfaces. In: Holland S, Mudd T, Wilkie-McKenna K, McPherson A, Wanderley MM (eds) New directions in music and human-computer interaction. Springer, London. ISBN 978-3-319-92069-6Google Scholar
  83. Zhang Y, Cai J, An L, Hui F, Ren T, Ma H et al (2017) Does music therapy enhance behavioral and cognitive function in elderly dementia patients? A systematic review and meta-analysis. Ageing Res Rev 35:1–11CrossRefGoogle Scholar
  84. Zimmerman TG (1982) An optical flex sensor US patent 4542291 VPL Res Inc Redwood City California USGoogle Scholar
  85. Zimmerman TG, Lanier J, Blanchard C, Bryson S, Harvill Y (1986) A hand gesture interface device. In: Carroll JM, Tanner PP (eds) Proceedings of the SIGCH conference on human factors in computing systems (CHI ’87). ACM, New York, pp 189–192Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Simon Holland
    • 1
    Email author
  • Tom Mudd
    • 2
  • Katie Wilkie-McKenna
    • 3
  • Andrew McPherson
    • 4
  • Marcelo M. Wanderley
    • 5
    • 6
  1. 1.Music Computing Lab, Centre for Research in ComputingThe Open UniversityMilton KeynesUK
  2. 2.Reid School of MusicUniversity of EdinburghEdinburghUK
  3. 3.Music Computing Lab, Centre for Research in ComputingThe Open UniversityMilton KeynesUK
  4. 4.School of Electronic Engineering and Computer Science, Centre for Digital MusicQueen Mary University of LondonLondonUK
  5. 5.Centre for Interdisciplinary Research in Music Media and TechnologyMcGill UniversityMontrealCanada
  6. 6.Inria Lille – Nord EuropeVilleneuve d’AscqFrance

Personalised recommendations