Neural Model Selection: the Minimum Prediction Risk Principle

Overview

The model selection procedure we employ as part of this methodology is a flexible framework for acquiring consistent estimators; it divides model selection into well-defined tasks, without restraining the implementation choices (see Figure 4.1). Asymptotic consistency for the ‘true’ parameter vector w₀ is an essential requirement of the neural model identification methodology presented. In many respects it is quite similar to Vapnik’s and (in particular) Moody’s approaches, which use a heuristic search (a hierarchy of nested model classes) and are based on prediction risk estimates (bounds for prediction risk in the case of Vapnik). For prediction risk e use a somewhat different and more frequently used analytical formulation from Moody’s, along the lines of Murata et al (1993) and Amari (1995), and also a different empirical estimate (bootstrap). Our notational conventions are as in Amari (1995).