Speech Communication Through the Skin: Design of Learning Protocols and Initial Findings
Evidence for successful communication through the sense of touch is provided by the natural methods of tactual communication that have been used for many years by persons with profound auditory and visual impairments. However, there are still currently no wearable devices that can support speech communication effectively without extensive learning. The present study reports the design and testing of learning protocols with a system that translates English phonemes to haptic stimulation patterns (haptic symbols). In one pilot study and two experiments, six participants went through the learning and testing of phonemes and words that involved different vocabulary sizes and learning protocols. We found that with a distinctive set of haptic symbols, it was possible for the participants to learn phonemes and words in small chunks of time. Further, our results provided evidence of the memory consolidation theory in that recognition performance improved after a period of inactivity on the part of a participant. Our findings pave the way for future work on improving the haptic symbols and on protocols that support the learning of a tactile speech communication system in hours as opposed to the much longer periods of time that are required to learn a new language.
KeywordsHaptic speech communication Memory consolidation Phoneme
This work was partially supported by a research grant from Facebook Inc. The authors thank all participants for their time and dedication, and Keith Klumb for project management.
- 4.Reed, C.M., Durlach, N.I., Braida, L.D.: Research on tactile communication of speech: a review. In: American Speech-Language-Hearing Association (ASHA) Monographs Number 20, May 1982, 23 p. (1982)Google Scholar
- 5.Reed, C.M., Durlach, N.I., Delhorne, L.A., Rabinowitz, W.M., Grant, K.W.: Research on tactual communication of speech: ideas, issues, and findings. Volta Rev. 91(5), 65–78 (1989). Monograph entitled “Research on the use of sensory aids for hearing-impaired people,” N.S. McGarr, EditorGoogle Scholar
- 8.Israr, A., Reed, C.M., Tan, H.Z.: Discrimination of vowels with a multi-finger tactual display. In: Proceedings of the Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 17–24 (2008)Google Scholar
- 10.Verrillo, R.T., Gescheider, G.A.: Perception via the sense of touch (Chap. 1). In: Summers, I.R. (ed.) Tactile Aids for the Hearing Impaired, pp. 1–36. Whurr Publishers, London (1992)Google Scholar
- 14.Ecroyd, D.H.: Voice and Articulation: A Hanbook, pp. 63–87. Scott Foresman & Co Publisher, Glenview (1966)Google Scholar
- 15.Reed, C.M., et al.: A phonemic-based tactual display for speech communication. IEEE Trans. Haptics (2018, in preparation)Google Scholar