Support Vector Machines: A Large Scale QP Problem

  • Massimiliano Pontil
  • Stefano Rogai
  • Alessandro Verri
Part of the Applied Optimization book series (APOP, volume 24)

Abstract

In this paper a new type of learning machines, named Support Vector Machines (SVMs), are discussed. In essence, given a training set — i.e., a number of previously classified patterns —, SVMs perform effective pattern recognition on a set of previously unseen patterns. We first review the theory of SCMs and some of their mathematical properties in detail. Then, we describe a few methods for the implementation of SVMs, which in the general case of large training sets requires the solution of large scale Quadratic Programming (QP) problem. Finally, we report the experimental results of the application of SCMs for the solution of a computer vision problem, appearance based 3-D object recognition from single image.

Keywords

Nite 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    M Bazaraa and C.M. Shetty, Nonlinear programming, John Wiley, New York, 1979.MATHGoogle Scholar
  2. 2.
    B. Boser, I. Guyoni, and V. Vapnik, training algorithm for optimal margin classifier, Fifth Annual Workshop on Computational Learning Theory, pp. 144–152. Pittsburg, PA, 1992.Google Scholar
  3. 3.
    C. Cortes and V.N. Vapnik, Support Vector Network. Machine learning, Vol. 20, 1995, pp. 1–25.Google Scholar
  4. 4.
    R. Courant and D. Hilbert. Methods of Mathematical Physics. John Wiley, New York, 1959.Google Scholar
  5. 5.
    L. Kaufman. Solving the Quadratic Programming problem arising in support vector classification. Technical report Bell Labs 1997.Google Scholar
  6. 6.
    J. Moré and G. Toraldo. On the solution of large scale quadratic programming with bound constraints. SIAM J. Optimization, 1 pp. 93–113, 1991.MATHCrossRefGoogle Scholar
  7. 7.
    E. Osuna, R. Freund, and F. Girosi. Support Vector Machines: training and applications, A.I. Memo 1602, MIT A.I. Lab., 1997.Google Scholar
  8. 8.
    E. Osuna, R. Freund and F. Girosi, An improved training algorithm for Support Vector Machines, IEEE Workshop on Neural Networks and Signal Processing, Amelia Island, FL, 1997.Google Scholar
  9. 9.
    C. Papageorgiou, M. Oren and T. Poggio. A General Framework for Object Detection. Proceedings of the International Conference on Computer Vision, Bombay, India, 1998.Google Scholar
  10. 10.
    M. Pontil and A. Verri, Object recognition with support vector machines. IEEE Trans. Pattern Anal. Mach. Intell., 20, pp. 637–646, 1998.CrossRefGoogle Scholar
  11. 11.
    M. Pontil and A. Verri, Properties of Support Vector Machines. Neural Computation, 1998, 10, pp. 955–974, 1998.CrossRefGoogle Scholar
  12. 12.
    M. Schmit, Identifying speakers with support vectors machines. Proceedings of Interface, Sydney, 1996.Google Scholar
  13. 13.
    Vanderbei, R.J. 1997. LOQO User’s Manual — Version 3.04. Statistical and Operations Research Tech. Rep. SOR-97-07, 1997.Google Scholar
  14. 14.
    V.N. Vapnik, The Nature of Statistical Learning Theory, Springer-Verlag, New York, 1995.MATHGoogle Scholar
  15. 15.
    V.N. Vapnik and A.Ja. Chervonenkis, On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab Appl., 16, pp. 264–280, 1971.MATHCrossRefGoogle Scholar

Copyright information

© Kluwer Academic Publishers, Boston 1998

Authors and Affiliations

  • Massimiliano Pontil
    • 1
    • 2
  • Stefano Rogai
    • 3
  • Alessandro Verri
    • 3
  1. 1.Center for Biological and Computational LearningMassachusetts Institute of TechnologyCambridgeUSA
  2. 2.INFM — Dipartimento di Fisica dell’ Università di GenovaItaly
  3. 3.DISI — Dipartimento di Informatica e Scienze dell’InformazioneUniversità di GenovaItaly

Personalised recommendations