An Online Algorithm for Applying Reinforcement Learning to Handle Ambiguity in Spoken Dialogues
Spoken dialogue systems (SDSs) have been widely used in human-computer communications, including database querying, online trouble shooting advising, etc. A major challenge in building an SDS is to handle ambiguity in natural languages. User queries, questions, descriptions in a natural language may be ambiguous. To be effective in practical applications, an SDS must be able to disambiguate input from its user(s). In our research, we develop an online algorithm for applying reinforcement learning to handle ambiguity in SDSs. We introduce a new user dialogue policy into the framework of reinforcement learning to model user dialogue behavior. Also, differing from the current reinforcement learning algorithms in speech and language processing that are characterized by offline training, our algorithm conducts both offline and online detection of user dialogue behavior. In this paper, we present the online algorithm for reinforcement learning, emphasizing the detection of user dialogue behavior. We also describe the initial implementation and experiments.
Unable to display preview. Download preview PDF.
- 1.Frampton, M., Lemon, O.: Learning more effective dialogue strategies using limited dialogue move features. In: Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pp. 185–192. Association for Computational Linguistics, Sydney (2006)Google Scholar
- 3.Jokinen, K., Kerminen, A., Kaipainen, M., Jauhiainen, T., Wilcock, G., Turunen, M., Hakulinen, J., Kuusisto, J., Lagus, K.: Adaptive dialogue systems - interaction with interact. In: Proceedings of the 3rd SIGdial workshop on Discourse and dialogue, vol. 2, pp. 64–73. Association for Computational Linguistics, Philadelphia (2002)CrossRefGoogle Scholar
- 8.Scheffler, K., Young, S.: Automatic learning of dialogue strategy using dialogue simulation and reinforcement learning. In: Proceedings of the second international conference on Human Language Technology Research, pp. 12–19. Morgan Kaufmann Publishers Inc., San Diego (2002)Google Scholar
- 9.Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (2005)Google Scholar
- 10.Swegles, K., Wang, F.: A hybrid reinforcement learning algorithm for optimizing dialogue strategies in spoken dialogue systems. In: The 26th International Conference on Machine Learning (ICML 2009), Montreal, Canada (submitted, 2009)Google Scholar