Abstract
Command and control (C&C) speech recognition allows users to interact with a system by speaking commands or asking questions restricted to a fixed grammar containing pre-defined phrases. Whereas C&C interaction has been commonplace in telephony and accessibility systems for many years, only recently have mobile devices had the memory and processing capacity to support client-side speech recognition. Given the personal nature of mobile devices, statistical models that can predict commands based in part on past user behavior hold promise for improving C&C recognition accuracy. For example, if a user calls a spouse at the end of every workday, the language model could be adapted to weight the spouse more than other contacts during that time. In this paper, we describe and assess statistical models learned from a large population of users for predicting the next user command of a commercial C&C application. We explain how these models were used for language modeling, and evaluate their performance in terms of task completion. The best performing model achieved a 26% relative reduction in error rate compared to the base system. Finally, we investigate the effects of personalization on performance at different learning rates via online updating of model parameters based on individual user data. Personalization significantly increased relative reduction in error rate by an additional 5%.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Chickering, D., Heckerman, D., Meek, C.: A Bayesian approach to learning Bayesian networks with local structure. In: Proceedings of Thirteenth Conference on Uncertainty in Artificial Intelligence, Providence, RI, pp. 80–89. Morgan Kaufmann, 1997
Chickering, D., Paek, T.: Personalizing influence diagrams: applying online learning strategies to dialogue management. User Modeling and User-Adaped Interaction, 2005
Chickering, D.M.: The winmine toolkit. Technical Report MSR-TR-2002-103 Microsoft, Redmond, WA, 2002
Dietterich T.G. (1998) Approximate statistical test for comparing supervised classification learning algorithms. Neural Comput. 10(7):1895–1923
Horvitz, E., Paek, T.: Harnessing models of users’ goals to mediate clarification dialog in spoken language systems. In: Proceedings of the Eighth International Conference on User Modeling, pp. 3–13. Sonthofen, Germany, 2001
Horvitz, E., Shwe, M.: In pursuit of effective handsfree decision support: coupling Bayesian inference, speech understanding, and user models. In: Nineteenth Anuual Symposium on Computer Applications in Medical Care. Toward Cost-Effective Clinical Computing, 1995
Hunt, A., McGlashan, S. (eds.): Speech Recognition Grammar Specification Version 1.0, W3C Recommendation (2004) http://www.w3.org/TR/2004/REC-speech-grammar-20040316/.
Jameson A., Klöckner K. (2004) User multitasking with mobile multimodal systems. In: Minker W., Bühler D., Dybkjær L. (eds). Spoken Multimodal Human-Computer Dialogue in Mobile Environments. Kluwer Academic Publishers, Dordrecht, pp. 349–377
Jameson, A., Wittig, F.: Leveraging data about users in general in the learning of individual user models. In: Nebel B., (ed.) Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, pp. 1185–1192. Morgan Kaufmann, San Francisco, CA 2001
Jelinek F. (1997) Statistical Methods for Speech Recognition. MIT Press, Cambridge, MA
Johansson, P.: User modeling in dialog systems. Technical Report Technical Report SAR 02-2, Santa Anna IT Research, 2002
Manning C.D., Schütze H. (1999) Foundations of Statistical Natural Language Processing. The MIT Press, Cambridge Massachusetts
Oviatt S., MacEachern M., Levow G. (1998) Predicting hyperarticulate speech during human-computer error resolution. Speech Commun. 24(2):87–110
Paek, T., Horvitz, E.: Conversation as action under uncertainty. In: Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence, pp. 455–464. Stanford, CA, 2000
Rosenfeld, R.: Two decades of statistical language modeling: where do we go from here? In: Proc. IEEE 88(8), 1270–1278 (2000)
Rosenfeld R., Olsen D., Rudnicky A. (2001) Universal speech interfaces. Interactions 8(6):34–44
Strother, N.: Future cell phones: the big trends, 2005–2010. Technical Report IN0502105WH, In-Stat, Scottsdale, AZ, 2005
Webb G., Pazzani M., Billsus D. (2001) Machine learning for user modeling. User Model. User-Adapted Interac. 11, 19–20
Widmer G., Kubat M. (1996) Learning in the presence of concept drift and hidden contexts. Machine Learning 23, 69–101
Woods, W.A.: Language processing for speech understanding. Computer Speech Processing, pp. 305–334, Prentice Hall, UK (1985)
Yu, D., Wang, K., Mahajan, M., Mau, P., Acero, A.: Improved name recognition with user modeling. In: Proceedings of the Eurospeech Conference, pp. 1229–1232. Geneva, Switzerland, 2003
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Paek, T., Chickering, D.M. Improving command and control speech recognition on mobile devices: using predictive user models for language modeling. User Model User-Adap Inter 17, 93–117 (2007). https://doi.org/10.1007/s11257-006-9021-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11257-006-9021-6