Robust Recognition of Conversational Telephone Speech via Multi-condition Training and Data Augmentation

  • Jiří MálekEmail author
  • Jindřich Ždánský
  • Petr Červa
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11107)


In this paper, we focus on automatic recognition of telephone conversational speech in scenario, when no amount of genuine telephone recordings is available for training. The training set contains only data from a significantly different domain, such as recording of broadcast news. Significant mismatch arises between training and test conditions, which leads to deteriorated performance of the resulting recognition system. We aim to diminish this mismatch using the data augmentation.

Speech compression and narrow-band spectrum are significant features of the telephone speech. We apply these effects to the training dataset artificially, in order to make it more similar to the desired test conditions. Using such augmented dataset, we subsequently train an acoustic model. Our experiments show that the augmented models achieve accuracy close to the results of a model trained on genuine telephone data. Moreover, when the augmentation is applied to the real-world telephone data, further accuracy gains are achieved.


Compression Data augmentation Multi-conditional training Conversational speech 



This work was supported by the Technology Agency of the Czech Republic (Project No. TH03010018).


  1. 1.
    Amodei, D., et al.: Deep speech 2: End-to-end speech recognition in English and Mandarin. In: International Conference on Machine Learning, pp. 173–182 (2016)Google Scholar
  2. 2.
    Borsky, M., Mizera, P., Pollak, P., Nouza, J.: Dithering techniques in automatic recognition of speech corrupted by MP3 compression: analysis, solutions and experiments. Speech Commun. 86, 75–84 (2017)CrossRefGoogle Scholar
  3. 3.
    Borsky, M., Pollak, P., Mizera, P.: Advanced acoustic modelling techniques in MP3 speech recognition. EURASIP J. Audio Speech Music Process. 2015(1), 20 (2015)CrossRefGoogle Scholar
  4. 4.
    Cui, X., Goel, V., Kingsbury, B.: Data augmentation for deep neural network acoustic modeling. IEEE/ACM Trans. Audio Speech Lang. Process. (TASLP) 23(9), 1469–1477 (2015)CrossRefGoogle Scholar
  5. 5.
    Dahl, G., Yu, D., Deng, L., Acero, A.: Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Trans. Audio Speech Lang. Process. 20(1), 30–42 (2012). Scholar
  6. 6.
    FFmpeg team: Ffmpeg - cross-platform solution to record, convert and stream audio and video. Software version: 20170525–b946bd8.
  7. 7.
    Fraga-Silva, T., et al.: Active learning based data selection for limited resource STT and KWS. In: Sixteenth Annual Conference of the International Speech Communication Association (2015)Google Scholar
  8. 8.
    Fraga-Silva, T., et al.: Improving data selection for low-resource STT and KWS. In: 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), pp. 153–159. IEEE (2015)Google Scholar
  9. 9.
    Garofolo, J.S., et al.: TIMIT acoustic-phonetic continuous speech corpus. Linguist. Data Consortium, 10(5) (1993)Google Scholar
  10. 10.
    Jaitly, N., Hinton, G.E.: Vocal tract length perturbation (VTLP) improves speech recognition. In: Proceeding of the ICML Workshop on Deep Learning for Audio, Speech and Language, pp. 625–660 (2013)Google Scholar
  11. 11.
    Kanda, N., Takeda, R., Obuchi, Y.: Elastic spectral distortion for low resource speech recognition with deep neural networks. In: 2013 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), pp. 309–314. IEEE (2013)Google Scholar
  12. 12.
    Kemp, T., Waibel, A.: Unsupervised training of a speech recognizer: recent experiments. In: Eurospeech (1999)Google Scholar
  13. 13.
    Kinoshita, K., et al.: A summary of the REVERB challenge: state-of-the-art and remaining challenges in reverberant speech processing research. EURASIP J. Adv. Signal Process. 2016(1), 7 (2016)CrossRefGoogle Scholar
  14. 14.
    Kneser, R., Ney, H.: Improved backing-off for m-gram language modeling. In: 1995 International Conference on Acoustics, Speech, and Signal Processing 1995, ICASSP 1995, vol. 1, pp. 181–184. IEEE (1995)Google Scholar
  15. 15.
    Ko, T., Peddinti, V., Povey, D., Khudanpur, S.: Audio augmentation for speech recognition. In: INTERSPEECH, pp. 3586–3589 (2015)Google Scholar
  16. 16.
    Li, J., Deng, L., Gong, Y., Haeb-Umbach, R.: An overview of noise-robust automatic speech recognition. IEEE/ACM Trans. Audio Speech Lang. Process. 22(4), 745–777 (2014)CrossRefGoogle Scholar
  17. 17.
    Ma, J., Schwartz, R.: Unsupervised versus supervised training of acoustic models. In: Ninth Annual Conference of the International Speech Communication Association (2008)Google Scholar
  18. 18.
    Mammone, R.J., Zhang, X., Ramachandran, R.P.: Robust speaker recognition: a feature-based approach. IEEE Signal Process. Mag. 13(5), 58 (1996)CrossRefGoogle Scholar
  19. 19.
    Polacky, J., Jarina, R., Chmulik, M.: Assessment of automatic speaker verification on lossy transcoded speech. In: 2016 4th International Workshop on Biometrics and Forensics (IWBF), pp. 1–6. IEEE (2016)Google Scholar
  20. 20.
    Raghavan, S., et al.: A comparative study on the effect of different codecs on speech recognition accuracy using various acoustic modeling techniques. In: 2017 Twenty-third National Conference on Communications (NCC), pp. 1–6. IEEE (2017)Google Scholar
  21. 21.
    Seltzer, M.L., Yu, D., Wang, Y.: An investigation of deep neural networks for noise robust speech recognition. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7398–7402. IEEE (2013)Google Scholar
  22. 22.
    Siegert, I., Lotz, A.F., Maruschke, M., Jokisch, O., Wendemuth, A.: Emotion intelligibility within codec-compressed and reduced bandwidth speech. In: ITG Symposium, Proceedings of Speech Communication, vol. 12, pp. 1–5. VDE (2016)Google Scholar
  23. 23.
    Torch team: Torch - a scientific computing framework for luajit.
  24. 24.
    Vincent, E., Watanabe, S., Nugraha, A.A., Barker, J., Marxer, R.: An analysis of environment, microphone and data simulation mismatches in robust speech recognition. Comput. Speech Lang. 46, 535–557 (2016)CrossRefGoogle Scholar
  25. 25.
    Xiong, W., et al.: The Microsoft 2016 conversational speech recognition system. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5255–5259. IEEE (2017)Google Scholar
  26. 26.
    Young, S., Young, S.: The HTK hidden Markov model toolkit: design and philosophy. Entrop. Cambridge Res. Lab. Ltd. 2, 2–44 (1994)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Jiří Málek
    • 1
    Email author
  • Jindřich Ždánský
    • 1
  • Petr Červa
    • 1
  1. 1.Institute of Information Technologies and ElectronicsTechnical University of LiberecLiberecCzech Republic

Personalised recommendations