Abstract
Inconspicuous speech enhancement system for laryngectomies using lip-reading is proposed to improve the usability and the speech quality. The proposed system uses a tiny camera on mobile phone and recognize the vowel sequences using lip-reading function. Three types of Japanese vowel recognition algorithms using MLP, CNN, and MobileNets, were investigated. 3,000 image datasets for training and testing were prepared from five persons while uttering discrete vowels. Our preliminary experimental result shows that the MobileNets is appropriate for embedding mobile devices in consideration of a performance both recognition accuracy and calculation cost.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Lin, B.-S., Yao, Y.-H., Liu, C.-F., Lienand, C.-F., Lin, B.-S.: Development of Novel Lip-Reading Recognition Algorithm, pp. 2169–3536. IEEE, 6 March 2017
Kimura, K., et al.: Development of wearable speech enhancement system for laryngectomees. In: NCSP 2016, pp. 339–342, March 2016
King, D.E.: Max-Margin Object Detection, arXiv:1502.00046v1 [cs.CV], 31 January 2015
Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1867–1874 (2014)
Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications, arXiv:1704.04861v1 [cs.CV], 17 April 2017
Rathee, N.: Investigating back propagation neural network for lip reading. In: ICCCA, pp. 373–376 (2016)
Acknowledgment
This work was supported by JSPS KAKENHI Grant-in-Aid for Scientific Research(C) Grant Number 15K01487.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Matsunaga, Y. et al. (2019). Preliminary Study of Mobile Device-Based Speech Enhancement System Using Lip-Reading. In: De La Prieta, F., Omatu, S., Fernández-Caballero, A. (eds) Distributed Computing and Artificial Intelligence, 15th International Conference. DCAI 2018. Advances in Intelligent Systems and Computing, vol 800. Springer, Cham. https://doi.org/10.1007/978-3-319-94649-8_37
Download citation
DOI: https://doi.org/10.1007/978-3-319-94649-8_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-94648-1
Online ISBN: 978-3-319-94649-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)