Model Selection for Support Vector Machines via Adaptive Step-Size Tabu Search

  • Gavin C. Cawley


The generalisation properties of a support vector classification network are typically governed by a regularisation parameter, C, and a small number of parameters specifying the kernel function. The process by which the optimal values of these parameters are obtained is known as model selection. This paper describes an automated model selection procedure based on minimisation of an upper bound on the leave-one-out cross-validation error, via a simple tabu search strategy with adaptive step size adjustment.


Support Vector Machine Support Vector Tabu Search Model Selection Procedure Tabu Search Heuristic 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    J. C. Platt, “Sequential minimal optimization: A fast algorithm for training support vector machines” Tech. Rep. MSR-TR-98-14, Microsoft Research 1998.Google Scholar
  2. [2]
    B. Boser, I. Guyon, and V. N. Vapnik, “A training algorithm for optimal margin classifiers” in Proceedings of the fifth annual workshop on computational learning theory, (Pittsburgh), pp. 144–152, ACM, 1992.Google Scholar
  3. [3]
    C. Cortes and V. Vapnik, “Support vector networks” Machine Learning, vol. 20, pp. 1–25, 1995.Google Scholar
  4. [4]
    V. N. Vapnik, The Nature of Statistical Learning Theory. New York: Springer-Verlag, 1995.CrossRefMATHGoogle Scholar
  5. [5]
    N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines (and other kernel-based learning methods). Cambridge, U.K.: Cambridge University Press, 2000.Google Scholar
  6. [6]
    T. Joachims, “Estimating the generalization performance of a SVM efficiently” Tech. Rep. LS-8 number 25, Univerität Dortmund, Fachbereich Informatik, 1999.Google Scholar
  7. [7]
    F. Glover and M. Laguna, “Tabu search,” in Modern Heuristic Techniques for Combinatorial Problems (C. R. Reeves, ed.), Advanced Topics in Computer Science, ch. 3, pp. 70–150, Mc Graw Hill, 1995.Google Scholar
  8. [8]
    M. Riedmiller and H. Braun, “A direct adaptive method for faster back propagation learning: The RPROP algorithm” in Proceedings of the IEEE International Conference on Neural Networks (H. Ruspini, ed.), (San Francisco, CA), pp. 586–591, 1993.Google Scholar
  9. [9]
    R. A. Fisher, “The use of multiple measurements in taxonomie problems” Annual Eugenics, vol. 7, no. II, pp. 179–188, 1936.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Wien 2001

Authors and Affiliations

  • Gavin C. Cawley
    • 1
  1. 1.School of Information SystemsUniversity of East AngliaNorwich, NorfolkUK

Personalised recommendations