Skip to main content
SpringerLink
Log in

Two-Class Support Vector Machines

  • Chapter
  • First Online:
Support Vector Machines for Pattern Classification

Part of the book series: Advances in Pattern Recognition ((ACVPR))

Abstract

In training a classifier, usually we try to maximize classification performance for the training data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    One of the main ideas is, like support vector machines, to add a regularization term, which controls the generalization ability, to the objective function.

  2. 2.

    This definition is imprecise. As shown in Definition 2.1 on p. 72, there are data that satisfy \(y_i (\textbf{w}^{\top}\,\textbf{x} + b) = 1\) but that can be deleted without changing the optimal separating hyperplane. Support vectors are defined using the solution of the dual problem, as discussed later.

  3. 3.

    In the definition of support vectors, we exclude the data in which both \(\alpha_i = 0\) and \(y_i \left(\textbf{w}^{\top}\,\textbf{x}_i+b\right)=1\) hold.

  4. 4.

    If we use a training method with fixed-size chunking such as SMO (see Section 5.2), the values of b calculated for x i in the working set and those in the fixed set may be different. In such a case it is better to take the average. But if a training method with variable-size chunking is used, in which all the nonzero α i are in the working set, the average is not necessary.

  5. 5.

    Orsenigo and Vercellis [6] formulate the discrete support vector machines that maximize the margin and minimize the number of misclassifications. This results in a linear mixed integer programming problem.

  6. 6.

    For the interpretation of indefinite kernels for classification, please see [7].

  7. 7.

    In Section 2.3.4, we will show that the mapping functions associated with (2.73) are many to one for even d, which is unfavorable.

  8. 8.

    In [10], neural network kernels are shown to be indefinite.

  9. 9.

    This assumption is satisfied when the input variables are nonnegative and \(K(\textbf{x}, \textbf{x}^{\prime})=(\textbf{x}^{\top}\textbf{x})^d\).

  10. 10.

    Conditionally positive semidefiniteness, which is positive semidefiniteness under an equality constraint discussed in Appendix D.1, is equivalent to positive semidefiniteness of \(K_\textrm{L1}\).

  11. 11.

    The formulation given by (2.306) and (2.307) is the same as that of the fuzzy support vector machine discussed in [70].

  12. 12.

    Here, to simplify discussions, we exclude the case where \(R_\textrm{{emp}}(\textbf{w},b)\) is not zero for a small value of C.

References

  1. C. M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, Oxford, 1995.

    Google Scholar 

  2. S. Abe. Neural Networks and Fuzzy Systems: Theory and Applications. Kluwer Academic Publishers, Norwell, MA, 1997.

    Google Scholar 

  3. T. Evgeniou, M. Pontil, and T. Poggio. Regularization networks and support vector machines. Advances in Computational Mathematics, 13(1):1–50, 2000.

    Article  MATH  MathSciNet  Google Scholar 

  4. V. N. Vapnik. The Nature of Statistical Learning Theory. Springer-Verlag, New York, 1995.

    Google Scholar 

  5. V. Cherkassky and F. Mulier. Learning from Data: Concepts, Theory, and Methods. John Wiley & Sons, New York, 1998.

    Google Scholar 

  6. C. Orsenigo and C. Vercellis. Discrete support vector decision trees via tabu search. Computational Statistics & Data Analysis, 47(2):311–322, 2004.

    Article  MATH  MathSciNet  Google Scholar 

  7. B. Haasdonk. Feature space interpretation of SVMs with indefinite kernels. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(4):482–492, 2005.

    Article  Google Scholar 

  8. M. G. Genton. Classes of kernels for machine learning: A statistics perspective. Journal of Machine Learning Research, 2:299–312, 2001.

    Article  Google Scholar 

  9. J.-H. Chen. M-estimator based robust kernels for support vector machines. In Proceedings of the Seventeenth International Conference on Pattern Recognition (ICPR 2004), volume 1, pages 168–171, Cambridge, 2004.

    Google Scholar 

  10. B. Schölkopf and A. J. Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge, MA, 2002.

    Google Scholar 

  11. T. Nishikawa and S. Abe. Maximizing margins of multilayer neural networks. In Proceedings of the Ninth International Conference on Neural Information Processing (ICONIP '02), volume 1, pages 322–326, Singapore, 2002.

    Google Scholar 

  12. J. A. K. Suykens and J. Vandewalle. Training multilayer perceptron classifiers based on a modified support vector method. IEEE Transactions on Neural Networks, 10(4):907–911, 1999.

    Article  Google Scholar 

  13. Y. Grandvalet and S. Canu. Adaptive scaling for feature selection in SVMs. In S. Becker, S. Thrun, and K. Obermayer, editors, Advances in Neural Information Processing Systems 15, pages 569–576. MIT Press, Cambridge, MA, 2003.

    Google Scholar 

  14. S. Abe. Pattern Classification: Neuro-Fuzzy Methods and Their Comparison. Springer-Verlag, London, 2001.

    Google Scholar 

  15. S. Abe. Training of support vector machines with Mahalanobis kernels. In W. Duch, J. Kacprzyk, E. Oja, and S. Zadrożny, editors, Artificial Neural Networks: Formal Models and Their Applications (ICANN 2005)–-Proceedings of Fifteenth International Conference, Part II, Warsaw, Poland, pages 571–576. Springer-Verlag, Berlin, Germany, 2005.

    Google Scholar 

  16. Y. Kamada and S. Abe. Support vector regression using Mahalanobis kernels. In F. Schwenker and S. Marinai, editors, Artificial Neural Networks in Pattern Recognition: Proceedings of Second IAPR Workshop, ANNPR 2006, Ulm, Germany, pages 144–152. Springer-Verlag, Berlin, Germany, 2006.

    Google Scholar 

  17. S. Chen, A. Wolfgang, C. J. Harris, and L. Hanzo. Symmetric kernel detector for multiple-antenna aided beamforming systems. In Proceedings of the 2007 International Joint Conference on Neural Networks (IJCNN 2007), pages 2486–2491, Orlando, FL, 2007.

    Google Scholar 

  18. M. Espinoza, J. A. K. Suykens, and B. De Moor. Imposing symmetry in least squares support vector machines regression. In Proceedings of the Forty-Fifth IEEE Conference on Decision and Control and European Control Conference 2005 (CDC-ECC'05), pages 5716–5721, Orlando, FL, 2005.

    Google Scholar 

  19. V. N. Vapnik. Statistical Learning Theory. John Wiley & Sons, New York, 1998.

    Google Scholar 

  20. S. R. Gunn and M. Brown. SUPANOVA: A sparse, transparent modelling approach. In Neural Networks for Signal Processing IX–-Proceedings of the 1999 IEEE Signal Processing Society Workshop, pages 21–30, 1999.

    Google Scholar 

  21. K. K. Lee, S. R. Gunn, C. J. Harris, and P. A. S. Reed. Classification of imbalanced data with transparent kernels. In Proceedings of International Joint Conference on Neural Networks (IJCNN '01), volume 4, pages 2410–2415, Washington, DC, 2001.

    Google Scholar 

  22. T. Howley and M. G. Madden. An evolutionary approach to automatic kernel construction. In S. Kollias, A. Stafylopatis, W. Duch, and E. Oja, editors, Artificial Neural Networks (ICANN 2006)–-Proceedings of the Sixteenth International Conference, Athens, Greece, Part II, pages 417–426. Springer-Verlag, Berlin, Germany, 2006.

    Google Scholar 

  23. N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), volume 1, pages 886–893, San Diego, CA, 2005.

    Google Scholar 

  24. B. Schölkopf, P. Simard, A. Smola, and V. Vapnik. Prior knowledge in support vector kernels. In M. I. Jordan, M. J. Kearns, and S. A. Solla, editors, Advances in Neural Information Processing Systems 10, pages 640–646. MIT Press, Cambridge, MA, 1998.

    Google Scholar 

  25. V. L. Brailovsky, O. Barzilay, and R. Shahave. On global, local, mixed and neighborhood kernels for support vector machines. Pattern Recognition Letters, 20(11–13):1183–1190, 1999.

    Article  Google Scholar 

  26. A. Barla, E. Franceschi, F. Odone, and A. Verri. Image kernels. In S.-W. Lee and A. Verri, editors, Pattern Recognition with Support Vector Machines: Proceedings of First International Workshop, SVM 2002, Niagara Falls, Canada, pages 83–96. Springer-Verlag, Berlin, Germany, 2002.

    Google Scholar 

  27. K. Hotta. Support vector machine with local summation kernel for robust face recognition. In Proceedings of the Seventeenth International Conference on Pattern Recognition (ICPR 2004), volume 3, pages 482–485, Cambridge, UK, 2004.

    Google Scholar 

  28. K. Grauman and T. Darrell. The pyramid match kernel: Efficient learning with sets of features. Journal of Machine Learning Research, 8:725–760, 2007.

    Google Scholar 

  29. T. Joachims. Learning to Classify Text Using Support Vector Machines: Methods, Theory and Algorithms. Kluwer Academic Publishers, Norwell, MA, 2002.

    Google Scholar 

  30. H. Lodhi, J. Shawe-Taylor, N. Cristianini, and C. Watkins. Text classification using string kernels. In T. K. Leen, T. G. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems 13, pages 563–569, 2001.

    Google Scholar 

  31. H. Lodhi, C. Saunders, J. Shawe-Taylor, N. Cristianini, and C. Watkins. Text classification using string kernels. Journal of Machine Learning Research, 2:419–444, 2002.

    Article  MATH  Google Scholar 

  32. C. Leslie, E. Eskin, and W. S. Noble. The spectrum kernel: A string kernel for SVM protein classification. In Proceedings of the Pacific Symposium on Biocomputing, pages 564–575, 2002.

    Google Scholar 

  33. C. Leslie, E. Eskin, J. Weston, and W. S. Noble. Mismatch string kernels for SVM protein classification. In S. Becker, S. Thrun, and K. Obermayer, editors, Advances in Neural Information Processing Systems 15, pages 1441–1448. MIT Press,Cambridge, MA, 2003.

    Google Scholar 

  34. C. S. Leslie, E. Eskin, A. Cohen, J. Weston, and W. S. Noble. Mismatch string kernels for discriminative protein classification. Bioinformatics, 20(4):467–476, 2004.

    Article  Google Scholar 

  35. H. Saigo, J.-P. Vert, N. Ueda, and T. Akutsu. Protein homology detection using string alignment kernels. Bioinformatics, 20(11):1682–1689, 2004.

    Article  Google Scholar 

  36. R. I. Kondor and J. Lafferty. Diffusion kernels on graphs and other discrete structures. In C. Sammut and A. Günther and Hoffmann, editors, Machine Learning, Proceedings of the Nineteenth International Conference (ICML 2002), Sydney, Australia, pages 315–322. Morgan Kaufmann Publishers, July 2002.

    Google Scholar 

  37. A. J. Smola and R. Kondor. Kernels and regularization on graphs. In Schölkopf and M. K. Warmuth, editors, Learning Theory and Kernel Machines: Proceedings of Sixteenth Annual Conference on Learning Theory and Seventh Kernel Workshop, COLT/Kernel 2003, Washington, DC, pages 144–158. Springer-Verlag, Berlin, Germany, 2003.

    Google Scholar 

  38. T. Ito, M. Shimbo, T. Kudo, and Y. Matsumoto. Application of kernels to link analysis. In KDD-2005: Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 586–592, Chicago, IL, August 2005.

    Google Scholar 

  39. F. Fouss, L. Yen, A. Pirotte, and M. Saerens. An experimental investigation of graph kernels on a collaborative recommendation task. In Proceedings of the Sixth IEEE International Conference on Data Mining (ICDM 2006), pages 863–868, Hong Kong, China, 2006.

    Google Scholar 

  40. T. Gärtner, P. Flach, and S. Wrobel. On graph kernels: Hardness results and efficient alternatives. In Schölkopf and M. K. Warmuth, editors, Learning Theory and Kernel Machines: Proceedings of Sixteenth Annual Conference on Computational Learning Theory and Seventh Kernel Workshop, COLT/Kernel 2003, Washington, DC, pages 129–143. Springer-Verlag, Berlin, Germany, 2003.

    Google Scholar 

  41. H. Kashima, K. Tsuda, and A. Inokuchi. Kernels for graphs. In B. Schölkopf, K. Tsuda, and J.-P. Vert, editors, Kernel Methods in Computational Biology, pages 155–170. MIT Press, Cambridge, MA, 2004.

    Google Scholar 

  42. K. Riesen, M. Neuhaus, and H. Bunke. Graph embedding in vector spaces by means of prototype selection. In F. Escolano and M. Vento, editors, Graph-Based Representations in Pattern Recognition: Proceedings of Sixth IAPR-TC-15 International Workshop, GbRPR 2007, Alicante, Spain, pages 383–393. Springer-Verlag, Berlin, Germany, June 2007.

    Google Scholar 

  43. K. Riesen and H. Bunke. Kernel k-means clustering applied to vector space embedding of graphs. In L. Prevost, S. Marinai, and F. Schwenker, editors, Artificial Neural Networks in Pattern Recognition: Proceedings of Third IAPR Workshop, ANNPR 2008, Paris, France, pages 24–35. Springer-Verlag, Berlin, Germany, 2008.

    Google Scholar 

  44. L. Ralaivola, S. J. Swamidass, H. Saigo, and P. Baldi. Graph kernels for chemical informatics. Neural Networks, 18(8):1093–1110, 2005.

    Article  Google Scholar 

  45. K. M. Borgwardt, C. S. Ong, S. Schönauer, S. V. N. Vishwanathan, A. J. Smola, and H.-P. Kriegel. Protein function prediction via graph kernels. Bioinformatics, 21(Suppl. 1):i47–i56, 2005.

    Article  Google Scholar 

  46. E. Pękalska and R. P. W. Duin. The Dissimilarity Representation for Pattern Recognition: Foundations and Applications. World Scientific Publishing, Singapore, 2005.

    Book  Google Scholar 

  47. H. Shimodaira, K. Noma, M. Nakai, and S. Sagayama. Dynamic time-alignment kernel in support vector machine. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14, volume 2, pages 921–928, MIT Press, Cambridge, MA, 2002.

    Google Scholar 

  48. N. Smith and M. Gales. Speech recognition using SVMs. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14, volume 2, pages 1197–1204, MIT Press, Cambridge, MA, 2002.

    Google Scholar 

  49. C. Cortes, P. Haffner, and M. Mohri. Rational kernels. In S. Becker, S. Thrun, and K. Obermayer, editors, Advances in Neural Information Processing Systems 15, pages 617–624. MIT Press, Cambridge, MA, 2003.

    Google Scholar 

  50. H. Shin and S. Cho. Invariance of neighborhood relation under input space to feature space mapping. Pattern Recognition Letters, 26(6):707–718, 2005.

    Article  Google Scholar 

  51. S. R. Gunn. Support vector machines for classification and regression. Technical Report ISIS-1-98, School of Electronics and Computer Science, University of Southampton, 1998.

    Google Scholar 

  52. T. M. Huang and V. Kecman. Bias term b in SVMs again. In Proceedings of the Twelfth European Symposium on Artificial Neural Networks (ESANN 2004), pages 441–448, Bruges, Belgium, 2004.

    Google Scholar 

  53. H. Xiong, M. N. S. Swamy, and M. O. Ahmad. Optimizing the kernel in the empirical feature space. IEEE Transactions on Neural Networks, 16(2):460–474, 2005.

    Article  Google Scholar 

  54. S. Abe. Sparse least squares support vector training in the reduced empirical feature space. Pattern Analysis and Applications, 10(3):203–214, 2007.

    Article  MathSciNet  Google Scholar 

  55. R. Herbrich. Learning Kernel Classifiers: Theory and Algorithms. MIT Press, Cambridge, MA, 2002.

    Google Scholar 

  56. M. Pontil and A. Verri. Properties of support vector machines. Neural Computation, 10(4):955–974, 1998.

    Article  Google Scholar 

  57. C. J. C. Burges and D. J. Crisp. Uniqueness of the SVM solution. In S. A. Solla, T. K. Leen, and K.-R. Müller, editors, Advances in Neural Information Processing Systems 12, pages 223–229. MIT Press, Cambridge, MA, 2000.

    Google Scholar 

  58. S. Abe. Analysis of support vector machines. In H. Bourlard, T. Adali, S. Bengio, J. Larsen, and S. Douglas, editors, Neural Networks for Signal Processing XII–-Proceedings of the 2002 IEEE Signal Processing Society Workshop, pages 89–98, 2002.

    Google Scholar 

  59. T. Downs, K. E. Gates, and A. Masters. Exact simplification of support vector solutions. Journal of Machine Learning Research, 2:293–297, 2001.

    Article  Google Scholar 

  60. P. M. L. Drezet and R. F. Harrison. A new method for sparsity control in support vector classification and regression. Pattern Recognition, 34(1):111–125, 2001.

    Article  MATH  Google Scholar 

  61. C. J. C. Burges. Simplified support vector decision rules. In L. Saitta, editor, Machine Learning, Proceedings of the Thirteenth International Conference (ICML '96), Bari, Italy, pages 71–77. Morgan Kaufmann, San Francisco, 1996.

    Google Scholar 

  62. D. Mattera, F. Palmieri, and S. Haykin. Simple and robust methods for support vector expansions. IEEE Transactions on Neural Networks, 10(5):1038–1047, 1999.

    Article  Google Scholar 

  63. S. Chen, S. R. Gunn, and C. J. Harris. The relevance vector machine technique for channel equalization application. IEEE Transactions on Neural Networks, 12(6):1529–1532, 2001.

    Article  Google Scholar 

  64. S. Chen, S. R. Gunn, and C. J. Harris. Errata to “The relevance vector machine technique for channel equalization application. ”IEEE Transactions on Neural Networks, 13(4):1024, 2002.

    Google Scholar 

  65. S. S. Keerthi, S. K. Shevade, C. Bhattacharyya, and K. R. K. Murthy. A fast iterative nearest point algorithm for support vector machine classifier design. IEEE Transactions on Neural Networks, 11(1):124–136, 2000.

    Article  Google Scholar 

  66. D. Tsujinishi, Y. Koshiba, and S. Abe. Why pairwise is better than one-against-all or all-at-once. In Proceedings of International Joint Conference on Neural Networks (IJCNN 2004), volume 1, pages 693–698, Budapest, Hungary, 2004.

    Google Scholar 

  67. R. M. Rifkin, M. Pontil, and A. Verri. A note on support vector machine degeneracy. In O. Watanabe and T. Yokomori, editors, Algorithmic Learning Theory: Proceedings of the Tenth International Conference on Algorithmic Learning Theory (ALT '99), Tokyo, Japan, pages 252–263. Springer-Verlag, Berlin, Germany, 1999.

    Google Scholar 

  68. R. Fernández. Behavior of the weights of a support vector machine as a function of the regularization parameter C. In Proceedings of the Eighth International Conference on Artificial Neural Networks (ICANN '98), volume 2, pages 917–922, Skövde, Sweden, 1998.

    Google Scholar 

  69. G. C. Cawley and N. L. C. Talbot. Manipulation of prior probabilities in support vector classification. In Proceedings of International Joint Conference on Neural Networks (IJCNN '01), volume 4, pages 2433–2438, Washington, DC, 2001.

    Google Scholar 

  70. C.-F. Lin and S.-D. Wang. Fuzzy support vector machines. IEEE Transactions on Neural Networks, 13(2):464–471, 2002.

    Article  Google Scholar 

  71. K. Veropoulos, C. Campbell, and N. Cristianini. Controlling the sensitivity of support vector machines. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence (IJCAI-99), Workshop ML3, pages 55–60, Stockholm, Sweden, 1999.

    Google Scholar 

  72. P. Xu and A. K. Chan. Support vector machines for multi-class signal classification with unbalanced samples. In Proceedings of International Joint Conference on Neural Networks (IJCNN 2003), volume 2, pages 1116–1119, Portland, OR, 2003.

    Google Scholar 

  73. G. Wu and E. Y. Chang. KBA: Kernel boundary alignment considering imbalanced data distribution. IEEE Transactions on Knowledge and Data Engineering, 17(6):786–795, 2005.

    Article  Google Scholar 

  74. T. Van Gestel, J. A. K. Suykens, J. De Brabanter, B. De Moor, and J. Vandewalle. Least squares support vector machine regression for discriminant analysis. In Proceedings of International Joint Conference on Neural Networks (IJCNN '01), volume 4, pages 2445–2450, Washington, DC, 2001.

    Google Scholar 

  75. C. Yuan and D. Casasent. Support vector machines for class representation and discrimination. In Proceedings of International Joint Conference on Neural Networks (IJCNN 2003), volume 2, pages 1611–1616, Portland, OR, 2003.

    Google Scholar 

  76. V. Vapnik and O. Chapelle. Bounds on error expectation for SVM In A. J. Smola, P. L. Bartlett, B. Schölkopf, and D. Schuurmans, editors, Advances in Large Margin Classifiers, pages 261–280. MIT Press, Cambridge, MA, 2000.

    Google Scholar 

  77. G. Cauwenberghs and T. Poggio. Incremental and decremental support vector machine learning. In T. K. Leen, T. G. Dietterich, and V. Tresp, editors, Advances in Neural Information Processing Systems 13, pages 409–415. MIT Press, Cambridge, MA, 2001.

    Google Scholar 

  78. K. Saadi, G. C. Cawley, and N. L. C. Talbot. Fast exact leave-one-out cross-validation of least-squares support vector machines. In Proceedings of the Tenth European Symposium on Artificial Neural Networks (ESANN 2002), pages 149–154, Bruges, Belgium, 2002.

    Google Scholar 

  79. Z. Ying and K. C. Keong. Fast leave-one-out evaluation and improvement on inference for LS-SVMs. In Proceedings of the Seventeenth International Conference on Pattern Recognition (ICPR 2004), volume 3, pages 494–497, Cambridge, UK, 2004.

    Google Scholar 

  80. B. Schölkopf, C. J. C. Burges, and A. J. Smola, editors. Advances in Kernel Methods: Support Vector Learning. MIT Press, Cambridge, MA, 1999.

    Google Scholar 

  81. C. J. C. Burges. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2(2):121–167, 1998.

    Article  Google Scholar 

  82. T. Joachims. Estimating the generalization performance of an SVM efficiently. In Proceedings of the Seventeenth International Conference on Machine Learning (ICML-2000), pages 431–438, Stanford, CA, 2000.

    Google Scholar 

  83. O. Chapelle and V. Vapnik. Model selection for support vector machines. In S. A. Solla, T. K. Leen, and K.-R. Müller, editors, Advances in Neural Information Processing Systems 12, pages 230–236. MIT Press, Cambridge, MA, 2000.

    Google Scholar 

  84. K. Duan, S. S. Keerthi, and A. N. Poo. An empirical evaluation of simple performance measures for tuning SVM hyperparameters. In Proceedings of the Eighth International Conference on Neural Information Processing (ICONIP-2001), Paper ID# 159, Shanghai, China, 2001.

    Google Scholar 

  85. D. Anguita, A. Boni, and S. Ridella. Evaluating the generalization ability of support vector machines through the bootstrap. Neural Processing Letters, 11(1):51–58, 2000.

    Article  Google Scholar 

  86. B. Efron and R. J. Tibshirani. An Introduction to the Bootstrap. Chapman & Hall/CRC Press, Boca Raton, FL, 1993.

    Google Scholar 

  87. N. Cristianini and C. Campbell. Dynamically adapting kernels in support vector machines. In M. S. Kearns, S. A. Solla, and D. A. Cohn, editors, Advances in Neural Information Processing Systems 11, pages 204–210. MIT Press, Cambridge, MA, 1999.

    Google Scholar 

  88. B. Schölkopf, J. Shawe-Taylor, A. J. Smola, and R. C. Williamson. Kernel-dependent support vector error bounds. In Proceedings of the Ninth International Conference on Artificial Neural Networks (ICANN '99), volume 1, pages 103–108, Edinburgh, UK, 1999.

    Google Scholar 

  89. M. Seeger. Bayesian model selection for support vector machines, Gaussian processes and other kernel classifiers. In S. A. Solla, T. K. Leen, and K.-R. Müller, editors, Advances in Neural Information Processing Systems 12, pages 603–609. MIT Press, Cambridge, MA, 2000.

    Google Scholar 

  90. J. T.-Y. Kwok. The evidence framework applied to support vector machines. IEEE Transactions on Neural Networks, 11(5):1162–1173, 2000.

    Article  Google Scholar 

  91. P. Sollich. Bayesian methods for support vector machines: Evidence and predictive class probabilities. Machine Learning, 46(1–3):21–52, 2002.

    Article  MATH  Google Scholar 

  92. L. Wang and K. L. Chan. Learning kernel parameters by using class separability measure. In Sixth Kernel Machines Workshop, In conjunction with Neural Information Processing Systems (NIPS), 2002.

    Google Scholar 

  93. O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee. Choosing multiple parameters for support vector machines. Machine Learning, 46(1–3):131–159, 2002.

    Article  MATH  Google Scholar 

  94. C. S. Ong and A. J. Smola. Machine learning using hyperkernels. In T. Fawcett and N. Mishra, editors, Machine Learning, Proceedings of the Twentieth International Conference (ICML 2003), Washington, DC, pages 568–575. AAAI Press, Menlo Park, CA, 2003.

    Google Scholar 

  95. C. S. Ong, A. J. Smola, and R. C. Williamson. Hyperkernels. In S. Thrun S. Becker and K. Obermayer, editors, Advances in Neural Information Processing Systems 15, pages 495–502. MIT Press, Cambridge, MA, 2003.

    Google Scholar 

  96. S. S. Keerthi and C.-J. Lin. Asymptotic behaviors of support vector machines with Gaussian kernel. Neural Computation, 15(7):1667–1689, 2003.

    Article  MATH  Google Scholar 

  97. F. Friedrichs and C. Igel. Evolutionary tuning of multiple SVM parameters. In Proceedings of the Twelfth European Symposium on Artificial Neural Networks (ESANN 2004), pages 519–524, Bruges, Belgium, 2004.

    Google Scholar 

  98. G. Lebrun, C. Charrier, and H. Cardot. SVM training time reduction using vector quantization. In Proceedings of the Seventeenth International Conference on Pattern Recognition (ICPR 2004), volume 1, pages 160–163, Cambridge, UK, 2004.

    Google Scholar 

  99. T. Hastie, S. Rosset, R. Tibshirani, and J. Zhu. The entire regularization path for the support vector machine. Journal of Machine Learning Research, 5:1391–1415, 2004.

    MathSciNet  Google Scholar 

  100. C. S. Ong, A. J. Smola, and R. C. Williamson. Learning the kernel with hyperkernels. Journal of Machine Learning Research, 6:1043–1071, 2005.

    MathSciNet  Google Scholar 

  101. C. Gold, A. Holub, and P. Sollich. Bayesian approach to feature selection and parameter tuning for support vector machine classifiers. Neural Networks, 18(5–6):693–701, 2005.

    Article  MATH  Google Scholar 

  102. K.-P. Wu and S.-D. Wang. Choosing the kernel parameters of support vector machines according to the inter-cluster distance. In Proceedings of the 2006 International Joint Conference on Neural Networks (IJCNN 2006), pages 2184–2190, Vancouver, Canada, 2006.

    Google Scholar 

  103. G. Gasso, K. Zapien, and S. Canu. Computing and stopping the solution paths for ν-SVR. In Proceedings of the Fifteenth European Symposium on Artificial Neural Networks (ESANN 2007), pages 253–258, 2007.

    Google Scholar 

  104. A. Rakotomamonjy and M. Davy. One-class SVM regularization path and comparison with alpha seeding. In Proceedings of the Fifteenth European Symposium on Artificial Neural Networks (ESANN 2007), pages 271–276, 2007.

    Google Scholar 

  105. M. M. Beigi and A. Zell. A novel kernel-based method for local pattern extraction in random process signals. In Proceedings of the Fifteenth European Symposium on Artificial Neural Networks (ESANN 2007), pages 265–270, 2007.

    Google Scholar 

  106. S. Abe. Optimizing kernel parameters by second-order methods. In Proceedings of the Fifteenth European Symposium on Artificial Neural Networks (ESANN 2007), pages 259–264, 2007.

    Google Scholar 

  107. C. J. C. Burges. Geometry and invariance in kernel based methods. In B. Schölkopf, C. J. C. Burges, and A. J. Smola, editors, Advances in Kernel Methods: Support Vector Learning, pages 89–116. MIT Press, Cambridge, MA, 1999.

    Google Scholar 

  108. Presented at the Fourth International Conference on Intelligent Data Engineering and Learning (IDEAL 2003), but not included in the proceedings (http://www2.kobe-u.ac.jp/abe/pdf/ideal2003.pdf), 2003.

Download references

Author information

Authors and Affiliations

  1. Graduate School of Engineering, Kobe University, 1-1 Rokkodai-cho, Nada-ku, Kobe, 657-8501, Japan

    Prof. Dr. Shigeo Abe

Authors
  1. Prof. Dr. Shigeo Abe
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shigeo Abe .

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag London Limited

About this chapter

Cite this chapter

Abe, S. (2010). Two-Class Support Vector Machines. In: Support Vector Machines for Pattern Classification. Advances in Pattern Recognition. Springer, London. https://doi.org/10.1007/978-1-84996-098-4_2

Download citation

  • DOI: https://doi.org/10.1007/978-1-84996-098-4_2

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-84996-097-7

  • Online ISBN: 978-1-84996-098-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation