Automatic Model Selection by Modelling the Distribution of Residuals
Many problems in computer vision involve a choice of the most suitable model for a set of data. Typically one wishes to choose a model which best represents the data in a way that generalises to unseen data without overfitting. We propose an algorithm in which the quality of a model match can be determined by calculating how well the distribution of model residuals matches a distribution estimated from the noise on the data. The distribution of residuals has two components - the measurement noise, and the noise caused by the uncertainty in the model parameters. If the model is too complex to be supported by the data, then there will be large uncertainty in the parameters. We demonstrate that the algorithm can be used to select appropriate model complexity in a variety of problems, including polynomial fitting, and selecting the number of modes to match a shape model to noisy data.
KeywordsModel Selection Gaussian Noise Measurement Noise Shape Model Unseen Data
- 1.N. A.J. Lacey and N.L. Seed. Feature tracking and motion classification using a switchable model kalman filter. In E. Hancock, editor, 5th British Machine Vison Conference, pages 599–608. BMVA Press, Sept. 1994.Google Scholar
- 3.T. F. Cootes, C. J. Taylor, D. Cooper, and J. Graham. Active shape models-their training and application. Computer Vision and Image Understanding, 61(1):38–59, Jan. 1995.Google Scholar
- 4.D. Schuurmans. A new metric-based approach to model selection. In National Conf. on Artificial Intelligence (AAAI97), 1997.Google Scholar
- 5.D. Schuurmans and F. Southey. Metric-based methods for adaptive model selection and regularization. Machine Learning, page To Appear, 2001.Google Scholar
- 7.K. Kanatani. Statistical Optimization for Geometric Computation: Theory and Practise. Elsevier Science, Amsterdam, 1996.Google Scholar
- 8.N.A. Thacker, F. Ahearne, and P.I. Rockett. The bhattacharryya metric as an absolute similarity measure for frequency coded data. Kybernetika, 34(4):363–368, 1997.Google Scholar
- 9.O. Chapelle, V. Vapnik, and Y. Bengio. Model selection for small sample regression. In NIPS2000 Workshop: Cross-Validation, Bootstrap and Model Selection, 2000.Google Scholar
- 10.W. Press, S. Teukolsky, W. Vetterling, and B. Flannery. Numerical Recipes in C (2nd Edition). Cambridge University Press, 1992.Google Scholar
- 12.N. Thacker, D. Prendergast, and P.I. Rockett. B-fitting: An estimation technique with automatic parameter selection. In 7 th British Machine Vison Conference, pages 283–292, Edinburgh, UK, 1996.Google Scholar
- 13.P. Torr. Model selection for two view geometry. Technical report, http://research.microsoft.com/-philtorr, 1998.
- 14.P. H. S. Torr. Geometric motion segmentation and model selection. In J. Lasenby, A. Zisserman, R. Cipolla, and H. Longuet-Higgins, editors, Philosophical Transactions of the Royal Society A, pages 1321–1340. Roy Soc, 1998.Google Scholar