Experiencing Sound Through Interactive Jewellery and Fashion Accessories

  • Patrizia MartiEmail author
  • Iolanda Iacono
  • Michele Tittarelli
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 824)


In this paper we report the results of a series of experiments conducted as part of a participatory Research through Design (RtD) project [1] which aims to develop interactive jewels for deaf women. The project moved from the idea of bringing mindful attention to aesthetics in designing for disability, exploring cross-modal sensory associations. Our jewels detect ambient sound and represent them with light patterns, vibration patterns, and shape change. Preferences can be set through an app that allows to record sounds of interest (e.g. doorbell, a household appliance) and to notify them through visual or tactile feedback of the jewel.

In order to design meaningful cross-modal sensory associations between sounds in input and behaviours of the jewels in output, we engaged hearing and deaf people in an experiment on the association of light patterns to sound types. Preliminary results show the difficulty in associating light patterns to sound types without listening to the sound during the task. On the contrary, when the sound was preliminary heard, there was a higher possibility to associate it with the correct light pattern. Even if the results of the experiment are preliminary due to the small sample size of participants and the limited number of sounds used in the study, nevertheless they gave us insights about the design of the jewelry system and for follow up studies.


Interactive jewellery Disability Deaf women Participatory Research-through-Design Cross-modal associations 


  1. 1.
    Wilde D, Marti P (2018) Exploring aesthetic enhancement of wearable technologies for deaf women. In: 13th ACM SIGCHI conference proceedings designing interactive system. ACM, Hong KongGoogle Scholar
  2. 2.
    Quietude. Accessed 4 June 2018
  3. 3.
    WEAR (Wearable technologists Engage with Artists for Responsible innovation). Accessed 4 June 2018
  4. 4.
    Zimmerman J, Forlizzi J (2014) Research through design in HCI. In: Olson JS, Kellogg WA (eds) Ways of knowing in HCI, pp 167–189. SpringerGoogle Scholar
  5. 5.
    Zimmerman J, Forlizzi J, Evenson S (2007) Research through design as a method for interaction design research in HCI. In: SIGCHI conference on human factors in computing systems proceedings, CHI 2007, pp 493–502. ACM, San Jose, California, USAGoogle Scholar
  6. 6.
    Löwgren J (2016) On the significance of making in interaction design research. Interactions 23:27–33CrossRefGoogle Scholar
  7. 7.
    Gaver, W.: What should we expect from research through design? In: SIGCHI conference on human factors in computing systems proceedings, CHI 2012, pp 937–946. ACM, Austin, Texas, USAGoogle Scholar
  8. 8.
    Gaver W, Blythe M, Boucher A, Jarvis N, Bowers J, Wright P (2010) The prayer companion: openness and specificity, materiality and spirituality. In: SIGCHI conference on human factors in computing systems proceedings, CHI 2010, pp 2055–2064. ACM, Atlanta, Georgia, USAGoogle Scholar
  9. 9.
    Odom W (2015) Understanding long-term interactions with slow technology: an investigation of experiences with FutureMe. In: SIGCHI conference on human factors in computing systems proceedings, CHI 2015, pp 575–584. ACM, Seoul, Republic of KoreaGoogle Scholar
  10. 10.
  11. 11.
    Gorman BM, VisAural: a wearable sound localisation device for people with impaired hearing. In: 16th international ACM SIGACCESS conference proceedings on computers & accessibility, pp 337–338. ACM, Rochester, New York, USAGoogle Scholar
  12. 12.
  13. 13.
    Cute Circuit’s Sound-Shirt. Accessed 4 June 2018
  14. 14.
    VibroHear. Accessed 4 June 2018
  15. 15.
  16. 16.
    Harrison C, Horstman J, Hsieh G, Hudson S (2012) Unlocking the expressivity of point lights. In: SIGCHI conference on human factors in computing systems proceedings, Austin, Texas, USAGoogle Scholar
  17. 17.
    Kamlesh Sridhar P, Petry B, Pakianathan PVS, Kartolo AS, Nanayakkara S (2016) Towards one-pixel-displays for sound information visualization. In: 28th australian conference on computer-human interaction, pp 91–95, Launceston, AustraliaGoogle Scholar
  18. 18.
    Shinohara K, Tenenberg J (2009) A blind person’s interactions with technology. Commun ACM 52(8):58–66CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Patrizia Marti
    • 1
    Email author
  • Iolanda Iacono
    • 2
  • Michele Tittarelli
    • 2
  1. 1.University of SienaSienaItaly
  2. 2.Glitch Factory S.r.l.ArezzoItaly

Personalised recommendations