Skip to main content
SpringerLink
Log in

Double SVMSBagging: A Subsampling Approach to SVM Ensemble

  • Chapter
  • First Online:
Intelligent Automation and Computer Engineering

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 52))

Abstract

In ensemble methods, pooling the decisions of multiple unstable classifiers often lead to improvements in the generalization performance substantially in many applications. We propose here a new ensemble method, Double SVMSBagging, which is a variant of double bagging. In this method we have used subsampling in order to make the out-of-bag samples larger and trained support vector machine as the additional classifier on these out-of-bag samples. The underlying base classifier is the decision tree. We have used radial basis function kernel, expecting that the new classifier can perform efficiently in both linear and non-linear feature space. We have studied the performance of the proposed ensemble method in several benchmark datasets with different subsampling rate (SSR). We have applied the proposed method in partial discharge classification of the gas insulated switchgear (GIS). We compare the performance of double SVMsbagging with other well-known classifier ensemble methods in condition diagnosis; the double SVMsbagging performed better than other ensemble method in this case. We applied the double SVMsbagging in 15 UCI benchmark datasets and compare its accuracy with other ensemble methods, e.g., Bagging, Adaboost, Random Forest and Rotation Forest. The performance of this method with optimum SSR generate significantly lower prediction error than Rotation Forest and Adaboost for most of the datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Blake. C., & Merz, C. (1999). UCI repository of machine learning databases, http://www.ics.uci.edu/mlearn/MLRepository.html.

  2. Breiman, L., Friedman, J., Olshen, R., & Stone, C. (1984). Classification and regression trees. Belmont, CA: Wadsworth.

    MATH  Google Scholar 

  3. Breiman, L. (1996). Bagging predictors. Machine Learning, 24(2), 123–140.

    MathSciNet  MATH  Google Scholar 

  4. Breiman, L. (1998). Arcing classifiers. Annals of Statistics, 26(3), 801–849.

    Article  MathSciNet  MATH  Google Scholar 

  5. Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.

    Article  MATH  Google Scholar 

  6. Breiman, L. (2001). Statistical modeling: the two cultures. Statistical Science, 16(3), 199–231 (with discussion).

    Google Scholar 

  7. Burges, C. (1998). A tutorial on support vector machines for pattern recognition, Data Mining and Knowledge Discovery, 2, 121–167.

    Article  Google Scholar 

  8. Chang, C., & Lin, C. (2001). LIBSVM: a library for support vector machines, software available at http://www.csie.ntu.edu.tw/cjlin/libsvm.

  9. Cortes, C., & Vapnik, V. (1995). support-vector networks, Machine Learning, 20, 273–297.

    MATH  Google Scholar 

  10. Freund Y., & Schapire, R. (1996). Experiments with a new boosting algorithm. Proceedings of the thirteenth international conference machine learning (pp. 148–156). San Francisco, MA: Morgan Kaufmann.

    Google Scholar 

  11. Friedman, J., Hastie, T., & Tibshirani, R. (2000). Additive logistic regression: a statistical view of boosting. Annals of Statistics, 28, 337–407(with discussion).

    Google Scholar 

  12. Gestel, T., Suykens, J., Baesens, B., Viaene, S., Vanthienen, J., Dedene, G., Moor, B., & Vandewalle, J. (2001). Bench marking least squares support vector machine classifiers, Machine Learning, 54(1), 5–32.

    Article  Google Scholar 

  13. Gulski, E. (1995). Digital analysis of partial discharges. IEEE Transactions on Dielectrics and Electrical Insulation, 2(5), 822–837.

    Article  Google Scholar 

  14. Hirose, H., Matsuda, S., & H., Hikita. (2006). Electrical insulation diagnosing using a new statistical classification method. Proceedings of the 8th international conference on properties and applications of dielectric materials (ICPADM2006) (pp. 698–701).

    Google Scholar 

  15. Hirose, H., Hikita, M., Ohtsuka, S., Tsuru, S., & Ichimaru, J. , (2008). Diagnosing the electric power apparatuses using the decision tree method. IEEE Transactions on Dielectrics and Electrical Insulation, 15(5), 1252–1261.

    Article  Google Scholar 

  16. Hirose, H., Zaman, F., Tsuru, K., Tsuboi, T., & Okabe, S. (2008). Diagnosis accuracy in electric power apparatuses conditions using the classification methods. IEICE Technical Report, 108(243), 39–44.

    Google Scholar 

  17. Hothorn, T., & Lausen, B. (2003). Double-bagging: combining classifiers by bootstrap aggregation, Pattern Recognition, 36(6), 1303–1309.

    Article  MATH  Google Scholar 

  18. Iba, W., & Langley, R. (1992). Induction of one-level decision trees. Proceedings of the nineteenth international conference on machine learning, Aberdeen, Scotland.

    Google Scholar 

  19. Joachims, T. (1999). Making large-scale support vector machine learning practical. Advances in kernel methods: support vector machines (pp. 169–184). Cambridge, MA: MIT Press.

    Google Scholar 

  20. Li, Y., Cal, Y., Yin, R., & Xu, X. (2005). Fault diagnosis based on support vector machine ensemble. Proceedings of the 2005 international conference on machine Learning. Cybernet, 6, 3309–3314.

    Google Scholar 

  21. Lin, H., Lin, C., & Wen, R. (2007). A Note on Platt’s Probabilistic outputs for support vector machines, Machine Learning, 68(3), 267–276.

    Article  Google Scholar 

  22. Meyer, D., Leisch, F., & Hornik, K. (2003). The support vector machine under test. Neurocomputing, 55, 169–186.

    Article  Google Scholar 

  23. Patton, R., Lopez-Toribio, C., & Uppal, F. (1999). Artificial intelligence approaches to fault diagnosis, condition monitoring. IEE Colloquium on Machinery, External Structures and Health (Ref. No. 1999/034), 5/1–5/18.

    Google Scholar 

  24. Platt, J. (1999). Fast training of support vector machines using sequential minimal optimization. In B. Scholkopf, C.J.C. Burges, & A.J. Smola (Eds.), Advances Kernel methods – support vector learning (pp. 185–208). Cambridge, MA: MIT.

    Google Scholar 

  25. Platt, J. (2000). Probabilistic outputs for support vector machines and comparison to regularized likelihood methods. In A.J. Smola, P. Bartlett, B. Scholkopf, & D. Schuurmans, (Eds.), Advances in large margin classifiers (pp. 61–74). Cambridge, MA: MIT Press.

    Google Scholar 

  26. Rodríguez, J., Kuncheva, L., & Alonso. C. (2006). Rotation forest: a new classifier ensemble method. PAMI, 28(10), 1619–1630.

    Article  Google Scholar 

  27. Scholkopf, B., Smola, A., & Muller, K. (1998). Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10(5), 1299–1319.

    Article  Google Scholar 

  28. Scholkopf, B., Burges, C., & Smola, A. (1999). Introduction to support vector learning. In B. Scholkopf, C.J.C. Burges, & A.J. Smola (Eds.), Advances in kernel methods: support vector learning (pp. 1–15). Cambridge, MA: MIT Press.

    Google Scholar 

  29. Sorsa, T. (1995). Neural network approach to fault diagnosis. Doctoral Thesis, Tampere University of Technology Publications 153.

    Google Scholar 

  30. Wu, T., Lin, C., & Weng, R. (2004). Probability estimates for multi-class classification by pairwise coupling. Journal of Machine Learning and Research, 5(Aug.), 975–1005.

    Google Scholar 

  31. Zaman, F., & Hirose, H. (2009). A new double bagging via the support vector machine with application to the condition diagnosis for the electric power apparatus. International Conference on Data Mining and Applications (ICDMA’09) (pp. 654–660).

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Systems Design and Informatics, Kyushu Institute of Technology, Kyushu, Japan

    Faisal Zaman & Hideo Hirose

Authors
  1. Faisal Zaman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hideo Hirose
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Faisal Zaman .

Editor information

Editors and Affiliations

  1. School of Information Sciences &, Engineering, University of Canberra, Canberra, 2601, Australia

    Xu Huang

  2. International Association of Engineers, Hung To Road 37-39, Hong Kong, Hong Kong/PR China

    Sio-Iong Ao

  3. Dept. Computer Science, Tijuana Institute of Technology, Chula Vista, USA

    Oscar Castillo

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer Science+Business Media B.V.

About this chapter

Cite this chapter

Zaman, F., Hirose, H. (2009). Double SVMSBagging: A Subsampling Approach to SVM Ensemble. In: Huang, X., Ao, SI., Castillo, O. (eds) Intelligent Automation and Computer Engineering. Lecture Notes in Electrical Engineering, vol 52. Springer, Dordrecht. https://doi.org/10.1007/978-90-481-3517-2_30

Download citation

  • DOI: https://doi.org/10.1007/978-90-481-3517-2_30

  • Published:

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-90-481-3516-5

  • Online ISBN: 978-90-481-3517-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation