Mutual Learning with Many Linear Perceptrons: On-Line Learning Theory

  • Kazuyuki Hara
  • Yoichi Nakayama
  • Seiji Miyoshi
  • Masato Okada
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5768)


We propose a new mutual learning using many weak learner (or student) which converges into the identical state of Bagging that is kind of ensemble learning, within the framework of on-line learning, and have analyzed its asymptotic property through the statistical mechanics method. Mutual learning involving more than three students is essential compares to two student case from a viewpoint of variety of selection of a student acting as teacher. The proposed model consists of two learning steps: many students independently learn from a teacher, and then the students learn from others through the mutual learning. In mutual learning, students learn from other students and the generalization error is improved even if the teacher has not taken part in the mutual learning. We demonstrate that the learning style of selecting a student to act as teacher randomly is superior to that of cyclic order by using principle component analysis.


Weight Vector Thermodynamic Limit Generalization Error Ensemble Learning Cyclic Order 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Klein, E., et al.: Synchronization of neural networks by mutual learning and its application to cryptography. Proc. Neural Inf. Pro. Sys. 17, 689–696 (2004)Google Scholar
  2. 2.
    Mislovaty, R., Klein, E., Taunter, I., Kinzel, W.: Public channel cryptography by synchronization of neural networks and chaotic maps. Phys. Rev. Lett. 91(11), 118701 (2003)CrossRefGoogle Scholar
  3. 3.
    Bran, L.: Bagging predictors. Machine Learning 24, 123–140 (1996)zbMATHGoogle Scholar
  4. 4.
    Freund, Y., Shari’a, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 119–139 (1997)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Hara, K., Okada, M.: Statistical mechanics of mutual learning with latent teacher. J. Phys. Soc. Jpn. 76, 014001 (2007)CrossRefGoogle Scholar
  6. 6.
    Hara, K., Yamada, T.: Optimization of the asymptotic property of mutual learning involving an integration mechanism of ensemble learning. J. Phys. Soc. Jpn. 76, 024005 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Kazuyuki Hara
    • 1
  • Yoichi Nakayama
    • 2
  • Seiji Miyoshi
    • 3
  • Masato Okada
    • 4
    • 5
  1. 1.Tokyo Metropolitan College of Industrial TechnologyTokyoJapan
  2. 2.Tokyo Metropolitan College of TechnologyTokyoJapan
  3. 3.Faculty of Engineering ScienceKansai UniversityOsakaJapan
  4. 4.Graduate School of Frontier SciencesThe University of TokyoChibaJapan
  5. 5.Brain Science Institute, RikenSaitamaJapan

Personalised recommendations