© 2010

Adaptive Representations for Reinforcement Learning

  • Recent research in Adaptive Representations for Reinforcement Learning

  • Written by leading experts in this field

  • State-of-the Art book


Part of the Studies in Computational Intelligence book series (SCI, volume 291)

Table of contents

  1. Front Matter
  2. Shimon Whiteson
    Pages 1-5
  3. Shimon Whiteson
    Pages 7-15
  4. Shimon Whiteson
    Pages 17-30
  5. Shimon Whiteson
    Pages 31-46
  6. Shimon Whiteson
    Pages 65-76
  7. Shimon Whiteson
    Pages 77-94
  8. Shimon Whiteson
    Pages 95-104
  9. Back Matter

About this book


This book presents new algorithms for reinforcement learning, a form of machine learning in which an autonomous agent seeks a control policy for a sequential decision task. Since current methods typically rely on manually designed solution representations, agents that automatically adapt their own representations have the potential to dramatically improve performance. This book introduces two novel approaches for automatically discovering high-performing representations. The first approach synthesizes temporal difference methods, the traditional approach to reinforcement learning, with evolutionary methods, which can learn representations for a broad class of optimization problems. This synthesis is accomplished by customizing evolutionary methods to the on-line nature of reinforcement learning and using them to evolve representations for value function approximators. The second approach automatically learns representations based on piecewise-constant approximations of value functions. It begins with coarse representations and gradually refines them during learning, analyzing the current policy and value function to deduce the best refinements. This book also introduces a novel method for devising input representations. This method addresses the feature selection problem by extending an algorithm that evolves the topology and weights of neural networks such that it evolves their inputs too. In addition to introducing these new methods, this book presents extensive empirical results in multiple domains demonstrating that these techniques can substantially improve performance over methods with manual representations.

Authors and affiliations

  1. 1.Instituut voor InformaticaUniversiteit van AmsterdamAmsterdamNetherlands

Bibliographic information

  • Book Title Adaptive Representations for Reinforcement Learning
  • Authors Shimon Whiteson
  • Series Title Studies in Computational Intelligence
  • DOI
  • Copyright Information Springer Berlin Heidelberg 2010
  • Publisher Name Springer, Berlin, Heidelberg
  • eBook Packages Engineering Engineering (R0)
  • Hardcover ISBN 978-3-642-13931-4
  • Softcover ISBN 978-3-642-42231-7
  • eBook ISBN 978-3-642-13932-1
  • Series ISSN 1860-949X
  • Series E-ISSN 1860-9503
  • Edition Number 1
  • Number of Pages XIII, 116
  • Number of Illustrations 0 b/w illustrations, 0 illustrations in colour
  • Topics Computational Intelligence
    Artificial Intelligence
  • Buy this book on publisher's site
Industry Sectors
Chemical Manufacturing
IT & Software
Consumer Packaged Goods
Materials & Steel
Finance, Business & Banking
Energy, Utilities & Environment
Oil, Gas & Geosciences