Skip to main content
SpringerLink
Log in

Resolution of Singularities and Stochastic Complexity of Complete Bipartite Graph-Type Spin Model in Bayesian Estimation

  • Conference paper
Modeling Decisions for Artificial Intelligence (MDAI 2007)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 4617))

  • 1540 Accesses

Abstract

In this paper, we obtain the main term of the average stochastic complexity for certain complete bipartite graph-type spin models in Bayesian estimation. We study the Kullback function of the spin model by using a new method of eigenvalue analysis first and use a recursive blowing up process for obtaining the maximum pole of the zeta function which is defined by using the Kullback function. The papers [1,2] showed that the maximum pole of the zeta function gives the main term of the average stochastic complexity of the hierarchical learning model.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Watanabe, S.: Algebraic analysis for nonidentifiable learning machines. Neural Computation 13(4), 899–933 (2001)

    Article  MATH  Google Scholar 

  2. Watanabe, S.: Algebraic geometrical methods for hierarchical learning machines. Neural Networks 14(8), 1049–1060 (2001)

    Article  Google Scholar 

  3. Akaike, H.: A new look at the statistical model identification. IEEE Trans. on Automatic Control 19, 716–723 (1974)

    Article  MATH  MathSciNet  Google Scholar 

  4. Takeuchi, K.: Distribution of an information statistic and the criterion for the optimal model. Mathematical Science 153, 12–18 (1976)

    Google Scholar 

  5. Hannan, E.J., Quinn, B.G.: The determination of the order of an autoregression. Journal of Royal Statistical Society Series B 41, 190–195 (1979)

    MATH  MathSciNet  Google Scholar 

  6. Murata, N.J., Yoshizawa, S.G., Amari, S.: Network information criterion - determining the number of hidden units for an artificial neural network model. IEEE Trans. on Neural Networks 5(6), 865–872 (1994)

    Article  Google Scholar 

  7. Schwarz, G.: Estimating the dimension of a model. Annals of Statistics 6(2), 461–464 (1978)

    Article  MATH  MathSciNet  Google Scholar 

  8. Rissanen, J.: Universal coding, information, prediction, and estimation. IEEE Trans. on Information Theory 30(4), 629–636 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  9. Hironaka, H.: Resolution of Singularities of an algebraic variety over a field of characteristic zero. Annals of Math. 79, 109–326 (1964)

    Article  MathSciNet  Google Scholar 

  10. Yamazaki, K., Watanabe, S.: Singularities in Complete Bipartite Graph-Type Boltzmann Machines and Upper Bounds of Stochastic Complexities. IEEE Trans. on Neural Networks 16(2), 312–324 (2005)

    Article  Google Scholar 

  11. Nishiyama, Y., Watanabe, S.: Asymptotic Behavior of Free Energy of General Boltzmann Machines in Mean Field Approximation. Technical report of IEICE NC2006 38, 1–6 (2006)

    Google Scholar 

  12. Aoyagi, M., Watanabe, S.: Resolution of Singularities and the Generalization Error with Bayesian Estimation for Layered Neural Network. IEICE Trans. J88-D-II,10, 2112–2124 (2005) (English version: Systems and Computers in Japan John Wiley & Sons Inc. 2005) (in press)

    Google Scholar 

  13. Aoyagi, M.: The zeta function of learning theory and generalization error of three layered neural perceptron. RIMS Kokyuroku, Recent Topics on Real and Complex Singularities (in press)

    Google Scholar 

  14. Aoyagi, M., Watanabe, S.: Stochastic Complexities of Reduced Rank Regression in Bayesian Estimation. Neural Networks 18, 924–933 (2005)

    Article  MATH  Google Scholar 

  15. Watanabe, S., Hagiwara, K., Akaho, S., Motomura, Y., Fukumizu, K., Okada, M., Aoyagi, M.: Theory and Application of Learning System. Morikita, p. 195 (Japanese) (2005)

    Google Scholar 

  16. Nagata, K., Watanabe, S.: A proposal and effectiveness of the optimal approximation for Bayesian posterior distribution. In: Workshop on Information-Based Induction Sciences, pp. 99–104 (2005)

    Google Scholar 

  17. Takamatsu, S., Nakajima, S., Watanabe, S.: Generalization Error of Localized Bayes Estimation in Reduced Rank Regression. In: Workshop on Information-Based Induction Sciences, pp. 81–86 (2005)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Precision and Intelligence Laboratory, Tokyo Institute of Technology, 4259 Nagatsuda, Midori-ku, R2-5, Yokohama, 226-8503, Japan

    Miki Aoyagi & Sumio Watanabe

Authors
  1. Miki Aoyagi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sumio Watanabe
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Vicenç Torra Yasuo Narukawa Yuji Yoshida

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Aoyagi, M., Watanabe, S. (2007). Resolution of Singularities and Stochastic Complexity of Complete Bipartite Graph-Type Spin Model in Bayesian Estimation. In: Torra, V., Narukawa, Y., Yoshida, Y. (eds) Modeling Decisions for Artificial Intelligence. MDAI 2007. Lecture Notes in Computer Science(), vol 4617. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73729-2_42

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-73729-2_42

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-73728-5

  • Online ISBN: 978-3-540-73729-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation