Projectional Learning Laws for Differential Neural Networks Based on Double-Averaged Sub-Gradient Descent Technique

  • Isaac Chairez
  • Alexander PoznyakEmail author
  • Alexander Nazin
  • Tatyana Poznyak
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11554)


A new method to design learning laws for neural networks with continuous dynamics is proposed in this study. The learning method is based on the so-called double-averaged descendant technique (DASGDT), which is a variant of the gradient-descendant method. The learning law implements a double averaged algorithm which filters the effect of uncertainties of the states, which are continuously measurable. The learning law overcomes the classical assumption on the strict convexity of the functional with respect to the weights. The photocatalytic ozonation process of a single contaminant is estimated using the learning law design proposed in this study.


Differential neural networks Double-averaged subgradient Optimization Projection Ozonation processes 


  1. 1.
    Andreozzi, R., Caprio, V., Insola, A., Marotta, R.: Advanced oxidation processes (AOP) for water purification and recovery. Catal. Today 53(1), 51–59 (1999)Google Scholar
  2. 2.
    Beck, A., Teboulle, M.: Mirror descent and nonlinear projected subgradient methods for convex optimization. Oper. Res. Lett. 31(3), 167–175 (2003)Google Scholar
  3. 3.
    Cotter, N.: The stone-weierstrass theorem and its application to neural networks. IEEE Trans. Neural Netw. 1, 290–295 (1990)Google Scholar
  4. 4.
    Cybenko, G.: Approximation by superpositions of sigmoidal function. Math. Control Sig. Syst. 1989(2), 303–314 (1989)Google Scholar
  5. 5.
    Haddad, W., Chellaboina, V.: Nonlinear Dynamical Systems and Control. Princeton University Press, Princeton (2008)Google Scholar
  6. 6.
    Haykin, S.: Neural Networks and Learning Machines. Prentice Hall, Upper Saddle River (2009)Google Scholar
  7. 7.
    Kim, C.T., Lee, J.J.: Training two-layered feedforward networks with variable projection method. IEEE Trans. Neural Netw. 19(2), 371–375 (2008)Google Scholar
  8. 8.
    Malato, S., Oller, I., Fernández-Ibánez, P., Fuerhacker, M.: Technologies for advanced wastewater treatment in the mediterranean region. In: Barcelá, D., Petrovic, M. (eds.) Waste Water Treatment and Reuse in the Mediterranean Region, pp. 1–28. Springer, Heidelberg (2010). Scholar
  9. 9.
    Nazin, A.V.: Algorithms of inertial mirror descent in convex problems of stochastic optimization. Autom. Remote Control 79(1), 78–88 (2018)Google Scholar
  10. 10.
    Nemirovsky, A., Yudin, D.: Problem Complexity and Optimization Method Efficiency. Nauka, Moscow (1979)Google Scholar
  11. 11.
    Pérez-Sánchez, B., Fontenla-Romero, O., Guijarro-Berdiñas, B.: A review of adaptive online learning for artificial neural networks. Artif. Intell. Rev. 49(2), 281–299 (2018)Google Scholar
  12. 12.
    Poznyak, A.: Advanced Mathematical Tools for Automatic Control Engineers: Volume 1: Deterministic Systems. Elsevier Science, Amsterdam (2008)Google Scholar
  13. 13.
    Poznyak, A., Sanchez, E., Yu, W.: Differential Neural Networks for Robust Nonlinear Control (Identification, State Estimation and Trajectory Tracking). World Scientific, Singapore (2001)Google Scholar
  14. 14.
    Poznyak, T., Chairez, I., Poznyak, A.: Ozonation and Biodegradation in Environmental Engineering. Dynamic Neural Network Approach. Elsevier, Amsterdam (2019)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Isaac Chairez
    • 1
  • Alexander Poznyak
    • 2
    Email author
  • Alexander Nazin
    • 3
  • Tatyana Poznyak
    • 4
  1. 1.Biprocesses DepartamentUPIBI-Instituto Politecnico NacionalMexico CityMexico
  2. 2.Automatic Control DepartmentCINVESTAV-IPNMexico CityMexico
  3. 3.Trapeznikov Institute of Control Sciences Russian Academy of SciencesMoscowRussia
  4. 4.SEPI, ESIQIE-Instituto Politecnico NacionalMexico CityMexico

Personalised recommendations