Abstract
Numerous statistical learning methods have been developed for visual recognition tasks. Few attempts, however, have been made to address theoretical issues, and in particular, study the suitability of different learning algorithms for visual recognition. Large margin classifiers, such as SNoW and SVM, have recently demonstrated their success in object detection and recognition. In this paper, we present a theoretical account of these two learning approaches, and their suitability to visual recognition. Using tools from computational learning theory, we show that the main difference between the generalization bounds of SVM and SNoW depends on the properties of the data. We argue that learning problems in the visual domain have sparseness characteristics and exhibit them by analyzing data taken from face detection experiments. Experimental results exhibit good generalization and robustness properties of the SNoW-based method, and conform to the theoretical analysis.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
M. Aizerman, E. Braverman, and L. Rozonoer. Theoretical foundations of the potential function method in pattern recognition learning. Automation and Remote Control, 25:821–837, 1964.
Y. Amit and D. Geman. A computational model for visual selection. Neural Computation, 11(7):1691–1715, 1999.
S. Ben-David and H. U. Simon. Efficient learning of linear perceptron. In T. K. Leen, T. G. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems 13, pages 189–195. MIT Press, 2001.
A. Blum. Learning boolean functions in an infinite attribute space. Machine Learning, 9(4):373–386, 1992.
B. E. Boser, I. M. Guyon, and V. Vapnik. A training algorithm for optimal margin classifiers. In Proceedings of the Fifth Annual Workshop on Computational Learning Theory, pages 144–152, 1992.
A. Carleson, C. Cumby, J. Rosen, and D. Roth. The SNoW learning architecture. Technical Report UIUCDCS-R-99-2101, UIUC Computer Science Department, 1999.
N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines and other Kernel-based learning methods. Cambridge University Press, 2000.
J. De Bonet and P. Viola. Texture recognition using a non-parametric multi-scale statistical model. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pages 641–647, 1998.
G. Donato, M. S. Bartlett, J. C. Hager, P. Ekman, and T. J. Sejnowski. Classifying facial actions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(10):974–989, 2000.
Y. Freund and R. Schapire. Large margin classification using the perceptron. Machine Learning, 37(3):277–296, 1999.
T. Graepel, R. Herbrich, and R. C. Williamson. From margin to sparsity. In Advances in Neural Information Processing Systems 13, pages 210–216. MIT Press, 2001.
B. Heisele, T. Poggio, and M. Pontil. Face detection in still gray images. Technical Report AI Memo 1687, MIT AI Lab, 2000.
J. Kivinen, M. K. Warmuth, and P. Auer. The Perceptron algorithm vs. Winnow: linear vs. logarithmic mistake bound when few input variables are relevant. Artificial Intelligence, 1–2:325–343, 1997.
Y. Le Cun, L. Jackel, L. Bottou, A. Brunot, C. Cortes, J. Denker, H. Drucker, I. Guyon, U. Müller, E. Säckinger, P. Simard, and V. Vapnik. Comparison of learning algorithms for handwritten digit recognition. In Proceedings of International Conference on Artificial Neural Networks, pages 53–60, 1995.
N. Littlestone. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Machine Learning, 2:285–318, 1988.
N. Littlestone. Redundant noisy attributes, attribute errors, and linear threshold learning using winnow. In Proceedings of the fourth Annual Workshop on Computational Learning Theory, pages 147–156, 1991.
B. W. Mel and J. Fiser. Minimizing binding errors using learned conjunctive features. Neural Computation, 12:247–278, 2000.
A. Mohan, C. Papageorgiou, and T. Poggio. Example-based object detection in images by components. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(4):349–361, 2001.
E. Osuna, R. Freund, and F. Girosi. Training support vector machines: An application to face detection. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pages 130–136, 1997.
C. Papageorgiou and T. Poggio. A trainable system for object detection. International Journal of Computer Vision, 38(1):15–33, 2000.
P. Penev and J. Atick. Local feature analysis: A general statistical theory for object representation. Network: Computation in Neural Systems, 7(3):477–500, 1996.
T. Poggio and S. Edelman. A network that learns to recognize 3D objects. Nature, 343:263–266, 1990.
M. Pontil and A. Verri. Support vector machines for 3d object recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(6):637–646, 1998.
T. Rikert, M. Jones, and P. Viola. A cluster-based statistical model for object detection. In Proceedings of the Seventh IEEE International Conference on Computer Vision, pages 1046–1053, 1999.
F. Rosenblatt. The Perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65:386–407, 1958.
D. Roth. Learning to resolve natural language ambiguities: A unified approach. In Proceedings of the Fifteenth National Conference on Artificial Intelligence, pages 806–813, 1998.
D. Roth, M.-H. Yang, and N. Ahuja. Learning to recognize objects. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, volume 1, pages 724–731, 2000.
H. Rowley, S. Baluja, and T. Kanade. Neural network-based face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(1):23–38, 1998.
H. Schneiderman. A Statistical Approach to 3D Object Detection Applied to Faces and Cars. PhD thesis, Carnegie Mellon University, 2000.
H. Schneiderman and T. Kanade. Probabilistic modeling of local appearance and spatial relationships for object recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pages 45–51, 1998.
K.-K. Sung and T. Poggio. Example-based learning for view-based human face detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(1):39–51, 1998.
Y. W. Teh and G. E. Hinton. Rate-coded restricted Boltzmann machines for face recognition. In T. K. Leen, T. G. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems 13, pages 908–914. MIT Press, 2001.
K. Tieu and P. Viola. Boosting image retrieval. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, volume 1, pages 228–235, 2000.
M. Turk and A. Pentland. Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1):71–86, 1991.
L. G. Valiant. A theory of the learnable. Communications of the ACM, 27(11):1134–1142, Nov. 1984.
V. N. Vapnik. The Nature of Statistical Learning Theory. Springer-Verlag, New York, 1995.
M.-H. Yang, D. Roth, and N. Ahuja. Learning to recognize 3D objects with SNoW. In Proceedings of European Conference on Computer Vision, volume 1, pages 439–454, 2000.
M.-H. Yang, D. Roth, and N. Ahuja. A SNoW-based face detector. In S. A. Solla, T. K. Leen, and K.-R. Müller, editors, Advances of Neural Information Processing Systems, pages 855–861. MIT Press, 2000.
T. Zhang. Some theoretical results concerning the convergence of compositions of regularized linear functions. In S. A. Solla, T. K. Leen, and K.-R. Müller, editors, Advances in Neural Information Processing Systems 12, pages 370–376. MIT Press, 2000.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yang, MH., Roth, D., Ahuja, N. (2002). A Tale of Two Classifiers: SNoW vs. SVM in Visual Recognition. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds) Computer Vision — ECCV 2002. ECCV 2002. Lecture Notes in Computer Science, vol 2353. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-47979-1_46
Download citation
DOI: https://doi.org/10.1007/3-540-47979-1_46
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43748-2
Online ISBN: 978-3-540-47979-6
eBook Packages: Springer Book Archive