Time Delay Neural Networks for Articulatory Estimation from Speech: Suitable Subjective Evaluation Protocols

Abstract

The approach described in this paper is based on the use of time-delay neural networks for solving the task of articulatory estimation from acoustic speech and on image vector quantization as far as the visual synthesis is concerned. Once the system has been trained on a reference speaker, the association of visual cues is performed in real-time to each 20 ms of incoming speech. Preliminary results are reported with reference to the on-going experimentation both with normal hearing people and with deaf persons to estimate some of the many perceptual thresholds involved in the complex task of speechreading from synthetic images. This experimental phase is carried on in cooperation with FIADDA, the italian association of the families of hearing impaired children, and is based on a flexible simulation environment.