PERFUNC: Learning Continuous Functions with Back-Propagation

Abstract

As discussed in Sect. 6.4, feed-forward networks with hidden layers can represent any smooth continuous function ℝⁿ → ℝ^m. As an illustration the program perfunc is designed to solve the following task: when presented with the values of a specimen g(x) out of a dass of one-dimensional functions at a set of points x _i it predicts the function values at the output points x′ _i . This is achieved by training the network with the error back-propagation algorithm by presenting various ‘patterns’ σ ^μ _i where μ refers to specific values of the continuous parameters on which the function g(x) depends. The training method is essentially the same as was used for Boolean-function learning, so that its description need not be repeated. As a slight variation, the couplings w _ij to the output layer are fed into linear transfer functions $\tilde f\left( x \right)\,\, = \,\,x$. In contrast to the bounded sigmoidal function f(x) = tanh(βx) used for the interior layers, this enables the output signals S _i to take on arbitrarily large values.