Skip to main content
  • Book
  • © 2012

Supervised Sequence Labelling with Recurrent Neural Networks

Authors:

  • Recent research in Supervised Sequence Labelling with Recurrent Neural Networks

  • New results in a hot topic

  • Written by leading experts

Part of the book series: Studies in Computational Intelligence (SCI, volume 385)

Buying options

eBook USD 139.00
Price excludes VAT (USA)
  • ISBN: 978-3-642-24797-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book USD 179.99
Price excludes VAT (USA)
Hardcover Book USD 179.99
Price excludes VAT (USA)

This is a preview of subscription content, access via your institution.

Table of contents (9 chapters)

  1. Front Matter

    Pages 1-11
  2. Introduction

    • Alex Graves
    Pages 1-3
  3. Supervised Sequence Labelling

    • Alex Graves
    Pages 5-13
  4. Neural Networks

    • Alex Graves
    Pages 15-35
  5. Long Short-Term Memory

    • Alex Graves
    Pages 37-45
  6. A Comparison of Network Architectures

    • Alex Graves
    Pages 47-56
  7. Hidden Markov Model Hybrids

    • Alex Graves
    Pages 57-60
  8. Connectionist Temporal Classification

    • Alex Graves
    Pages 61-93
  9. Multidimensional Networks

    • Alex Graves
    Pages 95-108
  10. Hierarchical Subsampling Networks

    • Alex Graves
    Pages 109-131
  11. Back Matter

    Pages 0--1

About this book

Supervised sequence labelling is a vital area of machine learning, encompassing tasks such as speech, handwriting and gesture recognition, protein secondary structure prediction and part-of-speech tagging. Recurrent neural networks are powerful sequence learning tools—robust to input noise and distortion, able to exploit long-range contextual information—that would seem ideally suited to such problems. However their role in large-scale sequence labelling systems has so far been auxiliary. 

 

The goal of this book is a complete framework for classifying and transcribing sequential data with recurrent neural networks only. Three main innovations are introduced in order to realise this goal. Firstly, the connectionist temporal classification output layer allows the framework to be trained with unsegmented target sequences, such as phoneme-level speech transcriptions; this is in contrast to previous connectionist approaches, which were dependent on error-prone prior segmentation. Secondly, multidimensional recurrent neural networks extend the framework in a natural way to data with more than one spatio-temporal dimension, such as images and videos. Thirdly, the use of hierarchical subsampling makes it feasible to apply the framework to very large or high resolution sequences, such as raw audio or video.

 

Experimental validation is provided by state-of-the-art results in speech and handwriting recognition.

Keywords

  • Computational Intelligence
  • Neural Networks
  • Recurrent Neural Networks
  • Sequence Labelling

Authors and Affiliations

  • , Department of Computer Science, University of Toronto, Toronto, Canada

    Alex Graves

Bibliographic Information

Buying options

eBook USD 139.00
Price excludes VAT (USA)
  • ISBN: 978-3-642-24797-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book USD 179.99
Price excludes VAT (USA)
Hardcover Book USD 179.99
Price excludes VAT (USA)