Skip to main content
SpringerLink
Log in

Ears on My Back - Experiencing the Soundscape Without Hearing

  • Conference paper
  • First Online:
Sense, Feel, Design (INTERACT 2021)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13198))

Included in the following conference series:

Abstract

This paper analyses the artistic dimensions of fine art, focusing on film and performances, and the aspects of experience by the audience of this type of art. We analyze the contribution of sound, and consider a way to, either translate relevant aspects of sound to other modalities, or use other modalities to artistically enhance the experience. We provide a conceptual framework of dynamic and interactive types of art and we discuss the development of the relation between visual art and sound and the concept of experience of art. We discuss the problems of experience if sound is not available and consider the translation of sound to other modalities. We propose an experimental design, “Ears on my Back” and show the initial development of a garment that aims at experiencing relevant sound aspects in alternative modalities. Finally, we show the challenges and opportunities of multidisciplinary collaboration in this domain.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Giannetti, L.: Understanding Movies, 14th edn. Pearson, USA (2017)

    Google Scholar 

  2. Boardman, A.A.: An Illustrated History of Filmmaking. Walker Books Australia, AU (2018)

    Google Scholar 

  3. Liu, J., Zhang, L.: Melody in Movies. Nanjing University Press, China (2017)

    Google Scholar 

  4. Tong, l., Han, k.: Creating Interactive Audio Applications with Pure Data. Post & Telecom Press, China (2018)

    Google Scholar 

  5. Academy of Motion Picture Arts: Motion Picture Sound Engineering: A Series of Lectures Presented to the Classes Enrolled in the Courses in Sound Engineering Given by the Research Council of the Academy of Motion Picture Arts and Sciences, Hollywood, California (Classic Reprint). Forgotten Books, London (2018)

    Google Scholar 

  6. Douglass, J.S., Harnden, G.P.: The Art of Technique: An Aesthetic Approach to Film and Video Production. Pearson Education, Peking (2004)

    Google Scholar 

  7. Smith, T. J.: Audiovisual correspondences in Sergei Eisenstein’s Alexander Nevsky: a case study in viewer attention. In: Taberham, P., Nannicelli, T. (eds.) Cognitive Media Theory. AFI Film Readers, Taylor & Francis, Abingdon, UK (2014)

    Google Scholar 

  8. Millet, B., Chattah, J., Ahn, S.: Soundtrack design: the impact of music on visual attention and affective responses. Applied Ergonomics, (2021)

    Google Scholar 

  9. Vyas, D., van der Veer, G.: Experience as meaning: creating, communicating and maintaining in real-spaces. In: McCarthy, J., Wright, P. (eds) Tenth IFIP TC13 International Conference on Human-Computer Interaction - Rome, Italy (2005)

    Google Scholar 

  10. Gfeller, K., et al.: Musical backgrounds, listening habits, and aesthetic enjoyment of adult cochlear implant recipients. J. Am. Acad. Audiol. 11(7), 390–406 (2000)

    Article  Google Scholar 

  11. Warren, V.E., McPherson, C., Giorli, G., Goetz, K.T., Radford, C.A.: Marine Soundscape variation reveals insights into baleen whales and their environment. https://royalsocietypublishing.org/doi/https://royalsocietypublishing.org/doi/10.1098/rsos.201503. Accessed 31 May 2021

  12. Schafer, R.M. : The Soundscape: Our Sonic Environment and the Tuning of the World. Destiny Books, Rochester (1993)

    Google Scholar 

  13. Hong, J.Y., et al.: Quality assessment of acoustic environment reproduction methods for cinematic virtual reality in soundscape applications. Build. Environ. 149, 1–4 (2019)

    Article  Google Scholar 

  14. Meyer, J.: Acoustics and the Performance of Music. Springer, Cham (2009). https://doi.org/10.1007/978-0-387-09517-2

    Book  Google Scholar 

  15. Candy, L., Amitani, S., Bilda, Z.: Practice-led strategies for interactive art research. CoDesign. In: International Journal of Co-Creation in Design and the Arts. Taylor & Francis, UK, 24 April 2007, pp 209–223 (2007)

    Google Scholar 

Download references

Acknowledgment

Thanks to our collaborators: Lu Xun Academy of fine arts (LAFA): Multimedia and animation Collage Dalian campus, and Fashion and Fabric institute Shenyang campus, China. Shenyang Conservatory of music: Department of composition, and Department of vocal music. Shenyang, China. Northeastern University: Department of Computer science and software engineering, Shenyang, China. And our technical consultants. Dalian University: School of Drama and Performance, Dalian, China. Lun Xu and Changzhe Cai, electronics engineers. Geoffrey Donaldson Instituut, Noord-Scharwoude, Netherlands. And thanks to all faculty and student volunteers for their contribution to the project Ears on my back.

Author information

Authors and Affiliations

  1. Luxun Academy of Fine Arts (LAFA), Liaoning, China

    Danzhu Li & Gerrit van der Veer

  2. University of Twente, Enschede, Netherlands

    Danzhu Li & Gerrit van der Veer

  3. Vrije Universiteit Amsterdam, Amsterdam, Netherlands

    Gerrit van der Veer

Authors
  1. Danzhu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gerrit van der Veer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Danzhu Li or Gerrit van der Veer .

Editor information

Editors and Affiliations

  1. Polytechnic University of Bari, Bari, Italy

    Carmelo Ardito

  2. University of Bari Aldo Moro, Bari, Italy

    Rosa Lanzilotti

  3. Computer Science Department, University of Pisa, Pisa, Italy

    Alessio Malizia

  4. Reykjavik University, Reykjavik, Iceland

    Marta Larusdottir

  5. University of Cagliari, Caglieri, Italy

    Lucio Davide Spano

  6. Department of Information, University of Minho, Braga, Portugal

    José Campos

  7. Department of Communication, University of Copenhagen, Copenhagen, Denmark

    Morten Hertzum

  8. Fachbereich Informatik, Trier University of Applied Sciences, Trier, Germany

    Tilo Mentler

  9. ITI/Larsys, University of West London, London, UK

    José Abdelnour Nocera

  10. Knowledge Media Institute, Open University, Milton Keynes, UK

    Lara Piccolo

  11. University of Paderborn, Paderborn, Germany

    Stefan Sauer

  12. Faculty of Sciences, Department of Computer Science, Vrije Universiteit, Amsterdam, The Netherlands

    Gerrit van der Veer

Rights and permissions

Reprints and permissions

Copyright information

© 2022 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, D., van der Veer, G. (2022). Ears on My Back - Experiencing the Soundscape Without Hearing. In: Ardito, C., et al. Sense, Feel, Design. INTERACT 2021. Lecture Notes in Computer Science, vol 13198. Springer, Cham. https://doi.org/10.1007/978-3-030-98388-8_41

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-98388-8_41

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-98387-1

  • Online ISBN: 978-3-030-98388-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation