Abstract
We propose a method of detecting the task incompletion in spoken dialog systems using N-gram-based dialog features. We used a database created during a field test in which inexperienced users used a client-server music retrieval system with a spoken dialog interface on their own PCs. The dialog for a music retrieval task consisted of a sequence of user and system tags that related their utterances and behaviors. The dialogs were manually classified into two classes: completed and uncompleted music retrieval tasks. We then detected dialogs that did not complete the task using a Support Vector Machine with N-gram feature vectors and interaction parameters trained using manually classified dialogs. Off-line and on-line detection experiments were conducted on a large amount of real data, and the results show that our proposed method achieved good classification performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Engelbrecht KP, Möller S (2010) Sequential classifiers for the prediction of user judgments about spoken dialog systems. Speech Communication 52(10):816–833, DOI 10.1016/j.specom.2010.06.004
Fan RE, Chang KW, Hsieh CJ, Wang XR, Lin CJ (2008) LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research 9:1871–1874
Gibbon D, Mertins I, Moore RK (eds) (2000) Handbook of multimodal and spoken dialogue systems. Kluwer Academic Publishers, Boston
Hara S, Kitaoka N, Takeda K (2010) Automatic detection of task-incompleted dialog for spoken dialog system based on dialog act N-gram. In: Proceedings of INTERSPEECH 2010, pp 3034–3037
Hara S, Kitaoka N, Takeda K (2010) Estimation method of user satisfaction using N-gram-based dialog history model for spoken dialog system. In: Proceedings of LREC 2010, pp 78–83
Herm O, Schmitt A, Liscombe J (2008) When calls go wrong: How to detect problematic calls based on log-files and emotions? In: Proceedings of INTERSPEECH 2008, pp 463–466
Higashinaka R, Nakano M (2009) Ranking multiple dialogue states by corpus statistics to improve discourse understanding in spoken dialogue systems. IEICE Trans Information and Systems E92-D(9):1771–1782
Higashinaka R, Minami Y, Dohsaka K, Meguro T (2010) Issues in predicting user satisfaction transitions in dialogues: Individual differences, evaluation criteria, and prediction models. In: Proceedings of IWSDS 2010, pp 48–60
Hori C, Ohtake K, Misu T, Kashioka H, Nakamura S (2009) Statistical dialog management applied to WFST-based dialog systems. In: Proceedings of ICASSP 2009, pp 4793–4796
ITU-T (2005) Parameters describing the interaction with spoken dialogue systems. Recommendation Series P Suppl. 24, International Telecommunication Union, Geneva
Kim W (2007) Online call quality monitoring for automating agent-based call centers. In: Proceedings of INTERSPEECH 2007, pp 130–133
Möller S (2005) Parameters for quantifying the interaction. In: Proceedings of SIGdial 2005, pp 166–177
Schmitt A, Scholz M, Minker W, Liscombe J, Sündermann D (2010) Is it possible to predict task completion in automated troubleshooters? In: Proceedings of INTERSPEECH 2010, pp 94–97
Vapnik VN (1995) The Nature of Statistical Learning Theory. Springer
Walker MA, Langkilde-Geary I, Hastie HW, Wright J, Gorin A (2002) Automatically training a problematic dialogue predictor for a spoken dialogue system. Journal of Artificial Intelligence Research 16:293–319
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer Science+Business Media, LLC
About this paper
Cite this paper
Hara, S., Kitaoka, N., Takeda, K. (2011). On-line detection of task incompletion for spoken dialog systems using utterance and behavior tag N-gram vectors. In: Delgado, RC., Kobayashi, T. (eds) Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-1335-6_22
Download citation
DOI: https://doi.org/10.1007/978-1-4614-1335-6_22
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-1334-9
Online ISBN: 978-1-4614-1335-6
eBook Packages: EngineeringEngineering (R0)