MAGEFACE: Performative Conversion of Facial Characteristics into Speech Synthesis Parameters
- 487 Downloads
In this paper, we illustrate the use of the MAGE performative speech synthesizer through its application to the conversion of realtime-measured facial features with FaceOSC into speech synthesis features such as vocal tract shape or intonation. MAGE is a new software library for using HMM-based speech synthesis in reactive programming environments. MAGE uses a rewritten version of the HTS engine enabling the computation of speech audio samples on a two-label window instead of the whole sentence. Only this feature enables the realtime mapping of facial attributes to synthesis parameters.
Keywordsspeech synthesis software library performative media streaming architecture HTS MAGE realtime audio software face tracking mapping
Unable to display preview. Download preview PDF.
- 1.d’Alessandro, N., Dutoit, T.: HandSketch Bi-Manual Controller: Investigation on Expressive Control Issues of an Augmented Tablet. In: Proc. International Conference on New Interfaces for Musical Expression, pp. 78–81 (2007)Google Scholar
- 2.MAGE and Face Tracking, https://vimeo.com/39567236
- 3.MAGE and HandSketch, https://vimeo.com/39558917
- 4.MAGE website, http://mage.numediart.org
- 7.FaceOSC, https://vimeo.com/26098366