Estimating Spoken Dialog System Quality with User Models

  • Klaus-Peter Engelbrecht

Part of the T-Labs Series in Telecommunication Services book series (TLABS)

Table of contents

  1. Front Matter
    Pages i-xiv
  2. Klaus-Peter Engelbrecht
    Pages 1-8
  3. Klaus-Peter Engelbrecht
    Pages 9-20
  4. Klaus-Peter Engelbrecht
    Pages 61-85
  5. Klaus-Peter Engelbrecht
    Pages 101-108
  6. Back Matter
    Pages 109-127

About this book


Spoken dialog systems have the potential to offer highly intuitive user interfaces, as they allow systems to be controlled using natural language. However, the complexity inherent in natural language dialogs means that careful testing of the system must be carried out from the very beginning of the design process.   This book examines how user models can be used to support such early evaluations in two ways:  by running simulations of dialogs, and by estimating the quality judgments of users. First, a design environment supporting the creation of dialog flows, the simulation of dialogs, and the analysis of the simulated data is proposed.  How the quality of user simulations may be quantified with respect to their suitability for both formative and summative evaluation is then discussed. The remainder of the book is dedicated to the problem of predicting quality judgments of users based on interaction data. New modeling approaches are presented, which process the dialogs as sequences, and which allow knowledge about the judgment behavior of users to be incorporated into predictions. All proposed methods are validated with example evaluation studies.


Ad-hoc Oser Simulation Adaptive Control of Thought-rational International Telecommunication Union MeMo Workbench Natural Language Understanding Recency Effect System Design Process Telephone-Based Services

Authors and affiliations

  • Klaus-Peter Engelbrecht
    • 1
  1. 1., Quality and Usabilty LabTechnical University BerlinBerlinGermany

Bibliographic information

  • DOI
  • Copyright Information Springer-Verlag Berlin Heidelberg 2013
  • Publisher Name Springer, Berlin, Heidelberg
  • eBook Packages Engineering
  • Print ISBN 978-3-642-31590-9
  • Online ISBN 978-3-642-31591-6
  • Series Print ISSN 2192-2810
  • Series Online ISSN 2192-2829
  • Buy this book on publisher's site
Industry Sectors
Finance, Business & Banking
IT & Software
Consumer Packaged Goods
Oil, Gas & Geosciences