Semi-unsupervised Weighted Maximum-Likelihood Estimation of Joint Densities for the Co-training of Adaptive Activation Functions

  • Ilaria Castelli
  • Edmondo Trentin
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7081)


The paper presents an explicit maximum-likelihood algorithm for the estimation of the probabilistic-weighting density functions that are associated with individual adaptive activation functions in neural networks. A partially unsupervised technique is devised which takes into account the joint distribution of input features and target outputs. Combined with the training algorithm introduced in the companion paper [2], the solution proposed herein realizes a well-defined, specific instance of the novel learning machine. The extension of the overall training method to more-than-one hidden layer architectures is pointed out, as well. A preliminary experimental demonstration is given, outlining how the algorithm works.


Expectation maximization partially unsupervised learning co-training adaptive activation function 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification. Wiley (2001)Google Scholar
  2. 2.
    Castelli, I., Trentin, E.: Supervised and Unsupervised Co-Training of Adaptive Activation Functions in Neural Nets. In: Schwenker, F., Trentin, E. (eds.) PSL 2011. LNCS (LNAI), vol. 7081, pp. 52–61. Springer, Heidelberg (2012)Google Scholar
  3. 3.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Information Science and Statistics. Springer, Heidelberg (2007)Google Scholar
  4. 4.
    Trentin, E., Gori, M.: Inversion-based nonlinear adaptation of noisy acoustic parameters for a neural/HMM speech recognizer. Neurocomputing 70(1-3), 398–408 (2006)CrossRefGoogle Scholar
  5. 5.
    Linden, A., Kindermann, J.: Inversion of multilayer nets. In: Proc. of IJCNN 1989, Washington DC, pp. 425–430 (1989)Google Scholar
  6. 6.
    Hertz, J.A., Palmer, R.G., Krogh, A.: Introduction to the Theory of Neural Computation. Santa Fe Institute Studies in the Sciences of Complexity. Westview Press (1991)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Ilaria Castelli
    • 1
  • Edmondo Trentin
    • 1
  1. 1.Dipartimento di Ingegneria dell’InformazioneUniversità di SienaSienaItaly

Personalised recommendations