On Practical Aspects of Multi-condition Training Based on Augmentation for Reverberation-/Noise-Robust Speech Recognition

Abstract

Multi-condition training achieved through data augmentation belongs to the most successful techniques for noise/reverberation-robust automatic speech recognition (ASR). Its basic principle, i.e., generation of artificially distorted speech signals, is well documented in the literature. However, the specific choice of hyper-parameters for the generation process and its influence on the results of the subsequent ASR is usually not discussed in detail. Often, it is simply assumed that the augmentation should include as many acoustic conditions as possible. When designed in this broad manner, the computational/storage demands of the augmentation process grow rapidly.

In this paper, we rather aim for careful selection of a limited number of acoustic conditions that are highly relevant with respect to the target environment. In this manner, we keep the computational requirements feasible, while retaining the improved accuracy of the augmented models. We experimentally analyze two augmentation scenarios and draw conclusions regarding suitable setup choices. The first case concerns augmentation for reverberation-robust ASR. We propose to exploit Clarity $C_{50}$ as a feature for selection of Room Impulse Responses (RIRs) crucial for the augmentation. We show that mismatches in other RIR-related parameters, such as Reverberation Time $T_{60}$ or the room dimension, have small influence on ASR accuracy, as long as the training-test conditions are matched from the $C_{50}$ perspective. Subsequently, we investigate the augmentation for noise-reverberation-robust ASR. We discuss selection of Signal-to-Noise Ratio (SNR), the type of noise and reverberation level of speech. We observe the influence of mismatches in these parameters on the ASR accuracy.