MAGEFACE: Performative Conversion of Facial Characteristics into Speech Synthesis Parameters

  • Nicolas d’Alessandro
  • Maria Astrinaki
  • Thierry Dutoit
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 124)


In this paper, we illustrate the use of the MAGE performative speech synthesizer through its application to the conversion of realtime-measured facial features with FaceOSC into speech synthesis features such as vocal tract shape or intonation. MAGE is a new software library for using HMM-based speech synthesis in reactive programming environments. MAGE uses a rewritten version of the HTS engine enabling the computation of speech audio samples on a two-label window instead of the whole sentence. Only this feature enables the realtime mapping of facial attributes to synthesis parameters.


speech synthesis software library performative media streaming architecture HTS MAGE realtime audio software face tracking mapping 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    d’Alessandro, N., Dutoit, T.: HandSketch Bi-Manual Controller: Investigation on Expressive Control Issues of an Augmented Tablet. In: Proc. International Conference on New Interfaces for Musical Expression, pp. 78–81 (2007)Google Scholar
  2. 2.
    MAGE and Face Tracking,
  3. 3.
    MAGE and HandSketch,
  4. 4.
  5. 5.
    Nordstorm, K., et al.: Developing Vowels Mappings for an Interactive Voice Synthesis System Controlled by Hand Motions. Journal of the Acoustical Society of America 127, 2021 (2010)CrossRefGoogle Scholar
  6. 6.
    Zen, H., Tokuda, K., Black, A.: Statistical Parametric Speech Synthesis. Speech Communications 51(11), 1039–1064 (2009)CrossRefGoogle Scholar
  7. 7.

Copyright information

© ICST Institute for Computer Science, Social Informatics and Telecommunications Engineering 2013

Authors and Affiliations

  • Nicolas d’Alessandro
    • 1
  • Maria Astrinaki
    • 1
  • Thierry Dutoit
    • 1
  1. 1.Institute for New Media Art Technology, Signal Processing LaboratoryUniversity of MonsMonsBelgium

Personalised recommendations