Reinforcement Learning for Adaptive Dialogue Systems

A Data-driven Methodology for Dialogue Management and Natural Language Generation

  • Verena Rieser
  • Oliver Lemon

Table of contents

  1. Front Matter
    Pages i-xv
  2. Verena Rieser, Oliver Lemon
    Pages 1-6
  3. Fundamental Concepts

    1. Front Matter
      Pages 7-7
    2. Verena Rieser, Oliver Lemon
      Pages 9-27
    3. Verena Rieser, Oliver Lemon
      Pages 29-52
    4. Verena Rieser, Oliver Lemon
      Pages 53-70
  4. Policy Learning in Simulated Environments

    1. Front Matter
      Pages 71-71
    2. Verena Rieser, Oliver Lemon
      Pages 85-99
    3. Verena Rieser, Oliver Lemon
      Pages 101-163
  5. Evaluation and Application

    1. Front Matter
      Pages 165-165
    2. Verena Rieser, Oliver Lemon
      Pages 189-204
    3. Verena Rieser, Oliver Lemon
      Pages 205-212
  6. Back Matter
    Pages 213-253

About this book


The past decade has seen a revolution in the field of spoken dialogue systems. As in other areas of Computer Science and Artificial Intelligence, data-driven methods are now being used to drive new methodologies for system development and evaluation.

This book is a unique contribution to that ongoing change. A new  methodology for developing spoken dialogue systems is described in detail. The journey starts and ends with human behaviour in interaction, and explores methods for learning from the data, for building simulation environments for training and testing systems, and for evaluating the results. The detailed material covers: Spoken and Multimodal dialogue systems, Wizard-of-Oz data collection, User Simulation methods, Reinforcement Learning, and Evaluation methodologies.

The book is a research guide for students and researchers with a background in Computer Science, AI, or Machine Learning. It navigates through a detailed case study in data-driven methods for development and evaluation of spoken dialogue systems. Common challenges associated with this approach are discussed and example solutions are provided. This work provides insights, lessons, and inspiration for future research and development – not only for spoken dialogue systems in particular, but for data-driven approaches to human-machine interaction in general.


68-XX, 68Txx, 68T05, 68T50, 68T37, 68T42 Machine Learning Spoken Dialogue Systems User simulation

Authors and affiliations

  • Verena Rieser
    • 1
  • Oliver Lemon
    • 2
  1. 1., School of Mathematical &Heriot-Watt UniversityEdinburghUnited Kingdom
  2. 2., Mathematics and Computer ScienceHeriot Watt UniversityEdinburghUnited Kingdom

Bibliographic information

Industry Sectors
Chemical Manufacturing
IT & Software
Consumer Packaged Goods