Robust Parameter Estimation

Abstract

When applying HMMs in practice one — as always in the field of statistical pattern recognition — is faced with the problem to robustly estimate the parameters on the available training samples. Yet the situation is not as severe as in the case of n-gram models, for which no useful models could be created without suitable measures (cf. section 6.5). However, also with HMMs one is confronted with the so-called sparse data problem when working with more complex model architectures. Then chances are high, that the model parameters can either no longer be computed due to numerical reasons, or that an overfitting of the model to the sample data considered occurs. This can in extreme cases lead to the situation that the models estimated have learned the sample set “by heart”, i.e. they describe nothing but known samples. Such a behavior can, however, be diagnosed by an accompanying evaluation on an independent cross-validation set and the training procedure can then be stopped at a suitable position.