International Journal of Speech Technology

, Volume 22, Issue 4, pp 979–991 | Cite as

Hearing impaired speech recognition: Stockwell features and models

  • A. Revathi
  • N. SasikaladeviEmail author


The development of speech recognition system for recognising the speeches of a reasonable person in various languages usually is in fashion and does not involve challenges to be faced by the researchers. For the past ten years, advances are taking place in analysing and recognising the speeches of the hearing impaired because of the deployment of sophisticated processing methods to study the characteristics of speech production. These technological advancements not only pave the way for developing algorithms for recognising the speeches of healthy persons, also recognising the utterances of the hearing impaired. The study on analysing the oral communication skills of hearing-impaired children has received the attention of the researchers, speech pathologists and audiologists to develop assistive tool/system because inadequacy of such skills dramatically affects the social, educational and career opportunities available to them at large. This paper mainly emphasises the need for the development of a more challenging speaker independent speech recognition system for hearing impaired so that the system can respond to the speech uttered by any HI. In this work, Modified Group Delay Features and Stockwell transform cepstral features are used at the front end and vector quantisation (VQ) and Multivariate Hidden Markov Models (MHMM) at the back end for recognising the speeches uttered by any person with hearing disability. Performance of the system is compared for the three modelling techniques VQ, Fuzzy C Means (FCM) clustering and MHMM for the recognition of isolated digits in Tamil. Recognition accuracy is 89.25% and 79.5% for speaker dependent and independent speech recognition system for the hearing impaired. Performance of the system reveals that this system may be deployed to understand the speeches uttered by any hearing impaired speaker, improve the social status of the people with hearing impairment and mitigate the social stigma in leading a normal life.


Speech recognition Discrete Cosine Stockwell Transform Cepstrum (DCSTC) Hearing impaired (HI) Multivariate Hidden Markov models (MHMM) 



  1. Arunachalam, R. (2019). A strategic approach to recognize the speech of the children with hearing impairment: Different sets of features and models. International Journal on Multimedia Tools and Applications,78(15), 20787–20808.CrossRefGoogle Scholar
  2. Brookes, C. (2000). Speech-to-text systems for deaf, deafened and hard-of-hearing people. In IEE seminar on speech and language processing for disabled and elderly people. Ref. No. 2000/025.
  3. Chee, L. S., Ai, O. C., & Hariharan, M. (2009). MFCC based recognition of repetitions and prolongations in stuttered speech using k-NN and LDA. In IEEE student conference on research and development (SCOReD) (Vol. 1, pp. 146–149).Google Scholar
  4. Deller, J. R., Hsu, D., & Ferrier, L. J. (1991). On the use of hidden Markov modeling for recognition of dysarthric speech. International Journal on Computer Methods and Programs in Biomedicine,35(2), 125–139.CrossRefGoogle Scholar
  5. Girgin, M. C., & Ozsoy, B. (2008). The relationship between formant frequency and duration characteristics of vowels and speech intelligibility in Turkish hearing impaired children. World Applied Sciences Journal,4(6), 891–899.Google Scholar
  6. Gudi, A. B., Shreedhar, H. K., & Nagaraj, H. C. (2010). Signal processing techniques to estimate the speech disability in children. International Journal of Engineering and Technology,2(2), 169–176.Google Scholar
  7. Han, Z., Wang, X., & Wang, J. (2008). Pathological speech deformation degree assessment based on dynamic and static feature integration. In The 2nd international conference on bioinformatics and biomedical engineering (ICBBE) (pp. 2036–2039).
  8. Han, Z., Wang, X., & Wang, J. (2009). Speech recognition system based on visual features and neural network for persons with speech impairments. International Journal on Modeling, Identification and Control,8(3), 240–247.CrossRefGoogle Scholar
  9. Hawley, M. S., Cunningham, S. P., Green, P. D., Enderby, P., Palmer, R., Sehgal, S., et al. (2012). A voice-input voice-output communication aid for people with severe speech Impairment. IEEE Transactions on Neural Systems and Rehabilitation Engineering,21(1), 23–31.CrossRefGoogle Scholar
  10. Healy, E. W., Vasko, J. L., & Wang, D. (2019). The optimal threshold for removing noise from speech is similar across normal and impaired hearing—a time-frequency masking study. The Journal of the Acoustical Society of America,145(6), EL581.CrossRefGoogle Scholar
  11. Hegde, R. M., Murthy, H. A., & Gadde, V. R. R. (2007). Significance of the modified group delay feature in speech recognition. IEEE Transactions on Audio, Speech and Language Processing,15(1), 190–202.CrossRefGoogle Scholar
  12. Jamil, M. H. M., Al-Haddad, S. A. R., & Ng, C. K. (2011). A flexible speech recognition system for cerebral palsy disabled. International Conference on Informatics Engineering and Information Science,251, 42–55.CrossRefGoogle Scholar
  13. Jeyalakshmi, C., Krishnamurthi, V., & Revathi, A. (2010a). Deaf speech assessment using digital processing techniques. International Journal On Signal And Image Processing,1(1), 14–25.CrossRefGoogle Scholar
  14. Jeyalakshmi, C., Krishnamurthi, V., & Revathi, A. (2010). Speech recognition of deaf and hard of hearing people using hybrid neural network. In 2nd International conference on mechanical and electronics engineering (ICMEE) (Vol. 1, pp. 83–87).Google Scholar
  15. Jeyalakshmi, C., Krishnamurthi, V., & Revathi, A. (2014). Development of speech recognition system in native language for hearing impaired. Journal of Engineering Research, 2(2), Article 6.Google Scholar
  16. Jeyalakshmi, C., & Revathi, A. (2018a). Efficient speech recognition system for hearing impaired children in classical Tamil language. International Journal of Biomedical Engineering and Technology (IJBET),26(1), 84.CrossRefGoogle Scholar
  17. Jeyalakshmi, C., & Revathi, A. (2018b). Efficient speech recognition system for hearing impaired children in classical Tamil language. Int. J. Biomedical Engineering and Technology,26(1), 84–100.CrossRefGoogle Scholar
  18. Jeyalakshmi, C., Revathi, A., & Krishnamurthi, V. (2012). Building robust HMM models for speech recognition of hearing impaired. International Journal on EE Times-India (pp. 1–11).Google Scholar
  19. Jeyalakshmi, C., Revathi, A., & Krishnamurthi, V. (2013). Effect of states and mixtures in HMM model and connected word recognition of profoundly deaf and hard of hearing speech. International Journal of Engineering and Technology (IJET),5(6), 4938–4946.Google Scholar
  20. Jeyalakshmi, C., Revathi, A., & Krishnamurthi, V. (2015a). Investigation of voice disorders and recognising the speech of children with hearing impairment in classical Tamil language. International Journal of Biomedical Engineering and Technology,17(4), 356–370.CrossRefGoogle Scholar
  21. Jeyalakshmi, C., Revathi, A., & Krishnamurthi, V. (2015b). Alphabet model-based short vocabulary speech recognition for the assessment of profoundly deaf and hard of hearing speeches. International Journal of Modelling, Identification and Control (IJMIC),23(3), 278–286.CrossRefGoogle Scholar
  22. Karjalainen, M., Boda, P. P., Somervuo, P., & Altosaar, T. (1997). Applications for the hearing impaired: evaluation of Finnish phoneme recognition methods. In 5th European conference on speech communication and technology.
  23. Le Prell, C. G., & Clavier, O. H. (2017). Performance evaluation of the hearing impaired speech recognition in noisy environment. International Journal on Hearing Research,349, 76–89.CrossRefGoogle Scholar
  24. Levitt, H. (1972). Acoustic analysis of deaf speech using digital processing techniques. IEEE Transactions on Audio and Electroacoustics,AU-20(1), 35–41.CrossRefGoogle Scholar
  25. Mahmoudi, Z., Rahati, S., & Ghasemi, M. M. (2010). Classification of voice disorder in children with cochlear implantation and a hearing aid using multiple classifier fusion. In 10th International conference on information sciences signal processing and their applications (ISSPA) (Vol. 1, pp. 304–307).
  26. Mengistu, K. T., & Rudzicz, F. (2011). Adapting acoustic and lexical models to dysarthric speech. In: IEEE international conference on acoustics, speech and signal processing (ICASSP) (Vol. 1, pp. 4924–4927).
  27. Newman, C. W., & Sandridge, S. A. (2004). Hearing loss is often undiscovered, but screening is easy. Cleveland: Audiology Research Laboratory, Department of Otolaryngology and Communicative Disorders, The Cleveland Clinic Foundation.Google Scholar
  28. Pickett, J. (1969). Some applications of speech analysis to communication aids for the deaf. Transactions on Audio and Electroacoustics,17(4), 283–289.CrossRefGoogle Scholar
  29. Pitts, A. B. (2010). Comparing speech assessments: The usefulness of the DEAP as compared tothe GFTA-2. Independent Studies and Capstones, Program in Audiology and Communication Sciences.
  30. Polur, P. D., & Miller, G. E. (2005). Investigation of an HMM/ANN hybrid structure in pattern recognition application using cepstral analysis of dysarthric (distorted) speech signals. Medical Engineering & Physics,28(8), 741–748.CrossRefGoogle Scholar
  31. Rabiner, L., & Juang, B. H. (1993). Fundamentals of speech recognition. Englewood Cliffs: Prentice Hall.Google Scholar
  32. Revathi, A., & Jeyalakshmi, C. (2017). A challenging task in recognizing the speech of the hearing impaired using normal hearing models in classical Tamil language. Journal of Engineering Research,5(2), 110–128.Google Scholar
  33. Saoud, S., Bousselmi, S., Mohamed, B. N., & Cherif, A. (2016). New speech enhancement based on discrete orthonormal Stockwell transform. International Journal of Advanced Computer Science and Applications. Scholar
  34. Stevens, C. G., & Bernstein, J. (1988). Bernstein and Jared, Automatic speech recognition of impaired speech. International Journal of Rehabilitation Research,11(4), 396–397.CrossRefGoogle Scholar
  35. Tseng, S.-C. (2011) Speech production of Mandarin-speaking children with hearing impairment and normal hearing. In 17th International congress of phonetic sciences (Vol. 1, pp. 2030–2033).
  36. Yamada, Y. (2000). Hector Javkin and Karen Youdelman (2000) Assistive speech technology for persons with speech impairments. International Journal on Speech Communication,30(2–3), 179–187.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of ECE/SEEESASTRA Deemed UniversityThanjavurIndia
  2. 2.Department of CSE/SoCSASTRA Deemed UniversityThanjavurIndia

Personalised recommendations