Some Improvements to a Parallel Decomposition Technique for Training Support Vector Machines

  • Thomas Serafini
  • Luca Zanni
  • Gaetano Zanghirati
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3666)


We consider a parallel decomposition technique for solving the large quadratic programs arising in training the learning methodology Support Vector Machine. At each iteration of the technique a subset of the variables is optimized through the solution of a quadratic programming subproblem. This inner subproblem is solved in parallel by a special gradient projection method. In this paper we consider some improvements to the inner solver: a new algorithm for the projection onto the feasible region of the optimization subproblem and new linesearch and steplength selection strategies for the gradient projection scheme. The effectiveness of the proposed improvements is evaluated, both in terms of execution time and relative speedup, by solving large-scale benchmark problems on a parallel architecture.


Support vector machines quadratic programs decomposition techniques gradient projection methods parallel computation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Chang, C.C., Lin, C.J.: LIBSVM: a Library for Support Vector Machines (2002),
  2. 2.
    Collobert, R., Benjo, S.: SVMTorch: Support Vector Machines for Large-Scale Regression Problems. Journal of Machine Learning Research 1, 143–160 (2001)CrossRefGoogle Scholar
  3. 3.
    Dai, Y.H., Fletcher, R.: New Algorithms for Singly Linearly Constrained Quadratic Programs Subject to Lower and Upper Bounds, Research Report NA/216. Department of Mathematics, University of Dundee (2003)Google Scholar
  4. 4.
    Joachims, T.: Making Large-Scale SVM Learning Practical. In: Schölkopf, B., et al. (eds.) Advances in Kernel Methods. MIT Press, Cambridge (1998)Google Scholar
  5. 5.
    Lin, C.J.: On the Convergence of the Decomposition Method for Support Vector Machines. IEEE Transactions on Neural Networks 12(6), 1288–1298 (2001)CrossRefGoogle Scholar
  6. 6.
    Pardalos, P.M., Kovoor, N.: An Algorithm for a Singly Constrained Class of Quadratic Programs Subject to Upper and Lower Bounds. Math. Programming 46, 321–328 (1990)zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Ruggiero, V., Zanni, L.: A Modified Projection Algorithm for Large Strictly Convex Quadratic Programs. J. Optim. Theory Appl. 104(2), 281–299 (2000)zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Ruggiero, V., Zanni, L.: Variable Projection Methods for Large Convex Quadratic Programs. In: Trigiante, D. (ed.) Recent Trends in Numerical Analysis. Advances in the Theory of Computational Mathematics 3, pp. 299–313. Nova Science Publ. (2000)Google Scholar
  9. 9.
    Serafini, T.: Gradient Projection Methods for Quadratic Programs and Applications in Training Support Vector Machines. Ph.D. Thesis, Dept. of Mathematics, University of Modena and Reggio Emilia (2005)Google Scholar
  10. 10.
    Serafini, T., Zanghirati, G., Zanni, L.: Gradient Projection Methods for Large Quadratic Programs and Applications in Training Support Vector Machines. Optim. Meth. and Soft. 20, 353–378 (2005)zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    Serafini, T., Zanghirati, G., Zanni, L.: Parallel Decomposition Approaches for Training Support Vector Machines. In: Joubert, G.R., Nagel, W.E., Peters, F.J., Walter, W.V. (eds.) Parallel Computing: Software Technology, Algorithms, Architectures and Applications. Advances in Parallel Computing, vol. 13, pp. 259–266. Elsevier, Amsterdam (2004)CrossRefGoogle Scholar
  12. 12.
    Serafini, T., Zanni, L.: On the working set selection in Gradient Projection-based Decomposition Techniques for Support Vector Machines.Optim. Meth. and Soft. (2005) (to appear),
  13. 13.
    Vapnik, V.N.: Statistical Learning Theory. John Wiley and Sons, New York (1998)zbMATHGoogle Scholar
  14. 14.
    Zanghirati, G., Zanni, L.: A Parallel Solver for Large Quadratic Programs in Training Support Vector Machines. Parallel Computing 29, 535–551 (2003)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Thomas Serafini
    • 1
  • Luca Zanni
    • 1
  • Gaetano Zanghirati
    • 2
  1. 1.Department of MathematicsUniversity of Modena and Reggio Emilia 
  2. 2.Department of MathematicsUniversity of Ferrara 

Personalised recommendations