Abstract
In this paper, we present a novel deformation-aware discriminative model for handwritten digit recognition. Unlike previous approaches our model directly considers image deformations and allows discriminative training of all parameters, including those accounting for non-linear transformations of the image. This is achieved by extending a log-linear framework to incorporate a latent deformation variable. The resulting model has an order of magnitude less parameters than competing approaches to handling image deformations. We tune and evaluate our approach on the USPS task and show its generalization capabilities by applying the tuned model to the MNIST task. We gain interesting insights and achieve highly competitive results on both tasks.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
DeCoste, D., Schölkopf, B.: Training invariant support vector machines. Machine Learning 46, 161–190 (2002)
Haasdonk, B., Keysers, D.: Tangent distance kernels for support vector machines. In: ICPR, Quebec City, Canada, pp. 864–868 (2002)
Simard, P.: Best practices for convolutional neural networks applied to visual document analysis. In: ICDAR, Edinburgh, Scotland, pp. 958–962 (2003)
Keysers, D., Deselaers, T., Gollan, C., Ney, H.: Deformation models for image recognition. PAMI 29, 1422–1435 (2007)
Keysers, D., Macherey, W., Ney, H., Dahmen, J.: Adaptation in statistical pattern recognition using tangent vectors. PAMI 26, 269–274 (2004)
Memisevic, R., Hinton, G.: Unsupervised learning of image transformations. In: CVPR, Minneapolis, MN, USA (2007)
Lafferty, J., McCallum, A., Pereira., F.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: ICML (2001)
Quattoni, A., Wang, S., Morency, L.P., Collins, M., Darrell, T.: Hidden conditional random fields. PAMI 29, 1848–1852 (2007)
Uchida, S., Sakoe, H.: A survey of elastic matching techniques for handwritten character recognition. Trans. Information and Systems E88-D, 1781–1790 (2005)
Mori, S., Yamamoto, K., Yasuda, M.: Research on machine recognition of handprinted characters. PAMI 6, 386–405 (1984)
Keysers, D., Och, F.J., Ney, H.: Maximum entropy and gaussian models for image object recognition. In: Van Gool, L. (ed.) DAGM 2002. LNCS, vol. 2449, pp. 498–506. Springer, Heidelberg (2002)
Heigold, G., Deselaers, T., Schlüter, R., Ney, H.: GIS-like estimation of log-linear models with hidden variables. In: ICASSP, Las Vegas, NV, USA, pp. 4045–4048 (2008)
Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropagation learning: The RPROP algorithm. In: ICNN, San Francisco, CA, USA (1993)
Haasdonk, B.: Transformation Knowledge in Pattern Analysis with Kernel Methods. PhD thesis, Albert-Ludwigs-Universität Freiburg (2005)
Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Computation 18, 1527–1554 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gass, T., Deselaers, T., Ney, H. (2009). Deformation-Aware Log-Linear Models. In: Denzler, J., Notni, G., Süße, H. (eds) Pattern Recognition. DAGM 2009. Lecture Notes in Computer Science, vol 5748. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03798-6_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-03798-6_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03797-9
Online ISBN: 978-3-642-03798-6
eBook Packages: Computer ScienceComputer Science (R0)