2006: Mobile Music Technology: Report on an Emerging Community

Abstract

The new field of mobile music emerges at the intersection of ubiquitous computing, portable audio technology and NIME. We have held a series of international workshop on this topic with leading projects and speakers, in order to establish a community and stimulate the development of the field. In this report, we define mobile music, and map out the field by reporting on the workshop series and accounting for the state-of-the-art.

Appendices

Author Commentary: Mobile Music Technology: From Innovation to Ubiquitous Use

Frauke Behrendt

A time before smartphones becomes more difficult to imagine by the day, a time before digital, networked, sensor-studded personal mobile devices became ubiquitous. In such a time, more specifically in 2004, drawing on music’s rich history of mobility and responding to emerging developments and innovations in mobile technology, I was part of an interdisciplinary group of researchers and designers that came together to experiment with and analyse mobility and music in the context of increasingly ubiquitous networked devices, and we thus contributed to establishing the field of mobile music.

We organised five ‘International Mobile Music (Technology) Workshops’ between 2004 and 2008, in Gothenburg, Vancouver, Brighton, Amsterdam and Vienna. These are mentioned in the paper and documented in more detail in the book accompanying the final event (Kirisits et al. 2008). By 2008, mobile music had become mainstream and an integral part of several fields of research and practice, including NIME and app design culture.

Our 2006 paper on the emerging community around mobile music technology has been used and developed in a range of research areas, by researchers and designers from around the world, both within and beyond the NIME community, as becomes evident from reviewing the papers and patents citing the paper. The main contribution of the paper has been for those designing mobile music products or services, such as mobile phone apps, software or hardware, for example (Wang 2009). Almost equal interest has come from those designing or evaluating interactive and/or collaborative performances with mobile phones for performers and for audience participation (e.g. mobile phone orchestras or social music making platforms). Overviews, classifications and taxonomies of the field of mobile music from various perspectives have also drawn on the work, for example by considering the social, cultural and historic dimensions of mobile music (Gopinath and Stanyek 2014). The field of sonic interaction design (Rocchesso et al. 2008), the field of locative music and sound (e.g. GPS sound walks), the educational use of mobile music (e.g. mobile phone music learning for children), sound studies (e.g. ubiquitous listening) and media studies (e.g. global mobile media), are other areas where the paper has made a contribution.

My own research contributions on mobile music technology include developing a taxonomy of mobile music with four categories: musical instruments, sonified mobility, sound platforms and placed sound. These categories were explored in more detail through a number of detailed analysis of specific artworks and apps, drawing on empirical material gathered through interviews, observations, ethnographies and case studies. This research material was analysed in light of theories and concepts from media studies, mobility studies, NIME and sonic interaction design (Behrendt 2015). More recently, I have drawn on the field of mobile music as research partner on the NetPark project¹ that turned a public park into an ongoing and growing collection of mobile and locative artworks, many of which focus on sound and music. This presents a platform for ongoing research on both the design process and the audience perspective/user experience of the NetPark and the works hosted and curated in it. There is also a close relation between this most recent engagement with mobile music technology and my other research around mobile media, smart cities, the Internet of Things and sustainable mobility, in that all my work considers mobility and musical/sonic perspective, in part inspired by this NIME paper. Over time, my engagement with the mobile music technology community has shifted from a more technical perspective and an active engagement in the NIME community towards a more theoretical and empirical analysis of the social and cultural aspects of mobile music in the field of media studies. In the years since our early community and the series of workshops on mobile music technology, mobile music has become so ubiquitous that the topic is now well-established in a range of research and practice communities.

Expert Commentary: Mobile Music Making Paradigm: Towards a New Culture of Use

Koray Tahiroğlu

Although the primary focus in the first era of mobile music research was on the ways in which mobile devices raised unique opportunities in locative media, it is clear from a historical perspective that the actual goal was to establish mobile music making as a research field and to create scientific and artistic legitimacy around it. This was achieved by bringing together the NIME community’s early adopters to explore mobile technologies as new platforms for music making. The idea of organising workshops was successful in getting the attention of the musicians, designers, researchers and industry people who shared interests common with the fundamental NIME approach to music, interaction and technology. These workshops, where the projects, ideas and concepts of practitioners were introduced and shared, were the first steps taken towards establishing a platform for mobile music. It is important to remember that early workshops were organised before the first generation of smartphones. Regardless of the mobile technology specifications, the creative and interactive focus differentiated mobile music from existing forms of practices and presented possibilities that were clearly distinct from traditional interactive music systems.

The early workshops presented ideas for some aspects of future developments in the field, such as possible social-music experiences and interaction models that question the roles of artists and listeners in the creative process. However, the future of music practise using mobile platforms was explored only with general statements and its relation to the relative economic, cultural and mediated paradigms was barely considered. Many of the projects presented for new ways of creating music focused significantly on the design constraints of working with state-of-art technology. The music industry was undergoing a transitional period at the time, as it can be argued it still is, so the workshops could have explored the technological context of mobile music making from a wider perspective; consciously evaluating the economic, cultural and social factors in the way musicians have always had to. Perhaps proposing tentative hypotheses on the evolution of mobile music for a period of time when mobile devices have developed beyond portable-playback devices to smart systems, could have offered more insight into future directions.

Nevertheless, after the first era of mobile music research, the collective community continued in its efforts to reflect on and share the developments in the field of mobile music. For instance, the Designing Musical Interactions for Mobile Systems workshop was organised in order to discuss the specific interaction design challenges for deploying engaging and creative musical activities on mobile devices in the smartphone era (Tahiroğlu et al. 2012). Simultaneously, the explosion of commercial music apps has directed the industry and the research to the widespread potential of music on mobile devices. During the workshop these different categories of use were discussed in detail with a set of interface design models for music instruments, controllers, portable studios, game and ambient interactions, social/network components for creating rich musical interactions that push the capabilities of present day mobile phone technologies (Tahiroğlu et al. 2012).

Mobile music making holds a special place in social-interactive aspects of research in NIME community (Bryan-Kinns and Healey 2004; Yang and Essl 2015). Mobile phone technology supports mobile and casual music-playing, facilitating interactive performances (Wang 2009). Current smartphones are powerful, network connected and equipped with Audio I/O, touchscreens, cameras and other embedded sensor input mechanisms. The increasing processor power of mobile devices makes real time signal processing and sound synthesis possible, enabling advanced music composition and performance tasks to be carried out on a mobile device. Most importantly, mobile devices advance opportunities for interaction in a collaborative context and have created a culture in music practices that was unlikely foreseen. In order to envision new strategies for mobile music making that could allow mobile technology to expand the meaning of “mobile,” it is worth considering the ways the community has defined work methods, practices and criteria for musical expressivity (Tanaka et al. 2012).

It is important to be aware of the unrealised potential of mobile music making. More design work needs to be done in order to explore the full potential of mobile technologies, the different ways in which user interfaces can be manipulated and the gestural capabilities of the devices. Furthermore, it is critical to consider the paradigms of computer music, “real world” instruments and the listening experience within an appropriately broad view of musical interactions. This can be achieved by giving equal weight to the performer, audience, technologies and cultural forces when making them mobile.