Media Conversion of Paralinguistic and Nonlinguistic Speech Information into Animated Texts for Foreign Language Learning

Conference paper


The use of visualization in speech utterances is very efficient for the study of foreign language learning or welfare application in the hearing impairment field. In this paper, we propose a media conversion interface of speech sounds into rich animated texts. As a confirmation method, it will help nonnative speakers who are learning foreign languages to check their pronunciations. In the system, user’s speech attribute information that includes paralinguistic and nonlinguistic information is converted into text attribute information. In this research, the easier representation format of texts is designed. As for speech information, we examine accent types and position of pauses in mora units contained in Japanese speech sentences. The extracted data are visualized into animated texts; then, the texts are evaluated by 13 Japanese language learners from Malaysia.


Text Information Speech Data Text Element Speech Information Pitch Accent 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Kato S, Short G, Minematsu N, Hirose K (2011) Perceptual study on the effects of language transfer on the naturalness of Japanese prosody for isolated words. IEICE SP, Speech 110(452):19–24Google Scholar
  2. 2.
    Masafumi N, Yasuo H, Shingo K, Akira I (2009) Analysis of utterance impressions in speech and text for indexing to transcriptions. IPSJ 50(2):460–468Google Scholar
  3. 3.
    Kimoto S, Taira K, Horiguchi Y, Kuroiwa S (2011) Transcription method to express utterance impressions. Acoustical society of Japan (research committee meeting), pp 439–442Google Scholar
  4. 4.
    Hirose K (2006) Prosody and spoken language information processing (in Japanese). Maruzen, p 2Google Scholar
  5. 5.
    Scheffé H (1952) An analysis of variance for paired comparison. J Am Stat Assoc 47:381–400MATHGoogle Scholar
  6. 6.
    Samsudin NSB, Mano K (2013) Text visualization of paralinguistic and nonlinguistic information of voices. In: Proceedings of the 7th SEATUC Symposium, OS-2.3Google Scholar
  7. 7.
    NHK Publishing (1998) NHK new Japanese accent dictionaryGoogle Scholar

Copyright information

© Springer Japan 2015

Authors and Affiliations

  1. 1.Systems Engineering and Science, Shibaura Institute of Technology, Graduate School of Engineering and ScienceSaitamaJapan

Personalised recommendations