Abstract
With the development of wireless networks and mobile computing, using speech recognition with wireless networks in mobile terminals to process data has become a new trend in mobile computing and achieved great success. Therefore, how to improve the speed of training speech recognition is still a problem worth studying. Using GPU to accelerate the training of speech recognition based on Deep Belief Network (DBN) has achieved great success, but there exits some problems. Aiming the problems that single GPU can not store huge parameters of DBM at one time and the unreasonable usage of GPU’s memory model, we propose a new method in this paper. We divide the weight matrix into blocks, take the connections between visible units and hidden unit as threads and store the weight matrix into shared memory of GPU, establishing a reasonable memory model. Experimental results show that the optimized GPU implementation achieves 223 times and 1.5 times acceleration compared to single CPU and single GPU in Kaldi respectively, which demonstrate that our method can improve the DBN’s training speed in mobile computing without GPU memory limitation.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Chen, W., Dong, S., et al.: Research on man-machine interaction of handheld mobile computing. Comput. Appl. 25(10), 2219–2223 (2005)
Hinton, G.E., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Seide, F., Li, G., Yu, D.: Conversational speech transcription using context-dependent deep neural networks. In: 12th Conference of the International Speech Communication Association, Florence, pp. 437–440 (2011)
Ly, D.L., Paprotski, V.: Neural networks on GPUs: restricted Boltzmann machines. In: 9th International Conference on Machine Learning and Applications, Washington, pp. 307–312 (2010)
Raina, R., Madhavan, A., Ng, A.Y.: Large-scale deep unsupervised learning using graphics processors. In: 26th International Conference on Machine Learning, Montreal, pp. 873–880 (2009)
Lopes, N., Ribeiro, B.: Towards adaptive learning with improved convergence of deep belief networks on graphics processing units. Pattern Recogn. 47(1), 114–127 (2014)
Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Comput. 14(8), 1771–1800 (2002)
Li, X., Li, C.: Alternating update layers for DBN-DNN fast training method. Appl. Res. Comput. 33(3), 843–847 (2016)
Zhu, Y., Zhang, Y., Pan, Y.: Large-scale restricted Boltzmann machines on single GPU. In: 2013 IEEE International Conference on Big Data, Santa Clara, pp. 169–174 (2013)
Xue, S., Song, Y., et al.: Fast training algorithm for deep neural network using multiple GPUs. J. Tsinghua Univ. (Sci. & Tech.) 53(6), 745–748 (2013)
Acknowledgements
The work described in this paper is supported by Guangdong Provincial Key Laboratory of Petrochemical Equipment Fault Diagnosis, Guangdong University of Petrochemical Technology (GDUPTKLAB201502) and Special Fund for Forest Scientific Research in the Public Welfare (201504307).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Jing, W., Jiang, T., Liu, Y. (2018). An Optimization of DBN/GPU Speech Recognition on Wireless Network Applications. In: Huang, M., Zhang, Y., Jing, W., Mehmood, A. (eds) Wireless Internet. WICON 2016. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 214. Springer, Cham. https://doi.org/10.1007/978-3-319-72998-5_20
Download citation
DOI: https://doi.org/10.1007/978-3-319-72998-5_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-72997-8
Online ISBN: 978-3-319-72998-5
eBook Packages: Computer ScienceComputer Science (R0)