Feed-Forward Neural Networks Using Hermite Polynomial Activation Functions

Abstract

In this paper feed-forward neural networks are introduced where hidden units employ orthogonal Hermite polynomials for their activation functions. The proposed neural networks have some interesting properties: (i) the basis functions are invariant under the Fourier transform, subject only to a change of scale, and (ii) the basis functions are the eigenstates of the quantum harmonic oscillator, and stem from the solution of Schrödinger’s diffusion equation. The proposed neural networks demonstrate the particle-wave nature of information and can be used in nonparametric estimation. Possible applications of neural networks with Hermite basis functions include system modelling and image processing.