Skip to main content
SpringerLink
Log in

Dynamics of Learning In Hierarchical Models – Singularity and Milnor Attractor

  • Conference paper
  • First Online:
Advances in Cognitive Neurodynamics (II)

Abstract

We study the dynamics of learning in a hierarchical model such as multilayer perceptron. Such a model includes singularities, which affects its dynamics seriously. The Milnor type attractors appear, because of the singularity. We will show its trajectories explicitly, and present the topological nature of the singularities.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 259.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 329.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 329.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Cousseau, F., Ozeki, T., Amari, S.: Dynamics of learning in multilayer perceptrons near singularities. IEEE. Trans. Neural. Netw. 19 (2008) 1313–1328.

    Article  PubMed  Google Scholar 

  2. Amari, S.: Natural gradient works efficiently in learning. Neural. Comput. 10 (1998) 251–276.

    Article  Google Scholar 

  3. Amari, S., Park, H., Ozeki, T.: Singularities affect dynamics of learning in neuromanifolds. Neural. Comput. 18 (2006) 1007–1065.

    Article  PubMed  Google Scholar 

  4. Fukumizu, K., Amari, S.: Local minima and plateaus in hierarchical structures of multilayer perceptrons. Neural. Netw. 13 (2000) 317–327.

    Article  CAS  PubMed  Google Scholar 

  5. Wei, H., Zhang, J., Cousseau, F., Ozeki, T., Amari, S.: Dynamics of learning near singularities in layered networks. Neural. Comput. 20 (2008) 813–843.

    Article  PubMed  Google Scholar 

  6. Wei, H., Amari, S.: Dynamics of learning near singularities in radial basis function networks. Neural. Netw. 21 (2008) 989–1005.

    Article  PubMed  Google Scholar 

  7. Milnor, J.: On the Concept of Attractor. Commun. Math. Phys. 99 (1985) 177–195.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. RIKEN Brain Science Institute, Saitama, 351-0198, Japan

    Shun-ichi Amari

  2. Department of Human and Information Science, TOKAI University, Kanagawa, 259-1292, Japan

    Tomoko Ozeki

  3. Graduate School of Frontier Sciences, The University of Tokyo, Chiba, 277-8561, Japan

    Florent Cousseau

  4. Research Institute of Automation, Southeast University, Nanjing, 210096, China

    Haikun Wei

Authors
  1. Shun-ichi Amari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tomoko Ozeki
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Florent Cousseau
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haikun Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shun-ichi Amari .

Editor information

Editors and Affiliations

  1. , Institute for Cognitive Neurodynamics, East China University of Science and Tec, Meilong Road 130, Shanghai, 200237, China, People's Republic

    Rubin Wang

  2. Fudan University, Shanghai, China, People's Republic

    Fanji Gu

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Science+Business Media B.V.

About this paper

Cite this paper

Amari, Si., Ozeki, T., Cousseau, F., Wei, H. (2011). Dynamics of Learning In Hierarchical Models – Singularity and Milnor Attractor. In: Wang, R., Gu, F. (eds) Advances in Cognitive Neurodynamics (II). Springer, Dordrecht. https://doi.org/10.1007/978-90-481-9695-1_1

Download citation

Publish with us

Policies and ethics

Search

Navigation