Skip to main content
SpringerLink
Log in

Distributed Support Vector Machines: An Overview

  • Chapter
  • First Online:
Solving Large Scale Learning Tasks. Challenges and Algorithms

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9580))

Abstract

Support Vector Machines (SVM) have a strong theoretical foundation and a wide variety of applications. However, the underlying optimization problems can be highly demanding in terms of runtime and memory consumption. With ever increasing usage of mobile and embedded systems, energy becomes another limiting factor. Distributed versions of the SVM solve at least parts of the original problem on different networked nodes. Methods trying to reduce the overall running time and memory consumption usually run in high performance compute clusters, assuming high bandwidth connections and an unlimited amount of available energy. In contrast, pervasive systems consisting of battery-powered devices, like wireless sensor networks, usually require algorithms whose main focus is on the preservation of energy. This work elaborates on this distinction and gives an overview of various existing distributed SVM approaches developed in both kinds of scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 16.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bennett, K.P., Campbell, C.: Support vector machines: hype or hallelujah? SIGKDD Explor. Newsl. 2(2), 1–13 (2000)

    Article  Google Scholar 

  2. Bhaduri, K., Stolpe, M.: Distributed data mining in sensor networks. In: Aggarwal, C.C. (ed.) Managing and Mining Sensor Data, pp. 211–236. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  3. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  MATH  Google Scholar 

  4. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, New York (2004)

    Book  MATH  Google Scholar 

  5. Caragea, C., Caragea, D., Honavar, V.: Learning support vector machines from distributed data sources. In: Proceedings of the 20th National Conference on Artificial Intelligence (AAAI), vol. 4, pp. 1602–1603. AAAI Press (2005)

    Google Scholar 

  6. Caragea, D., Silvescu, A., Honavar, V.: Towards a theoretical framework for analysisand synthesis of agents that learn from distributed dynamic data sources. In: Proceedings of the Workshop on Distributed and Parallel Knowledge Discovery. ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining (KDD) (2000)

    Google Scholar 

  7. Caragea, D., Silvescu, A., Honavar, V.: Agents that learn from distributed dynamic data sources. In: Proceedings of the Workshop on Learning Agents (2000)

    Google Scholar 

  8. Collobert, R., Bengio, S., Bengio, Y.: A parallel mixture of SVMs for very large scale problems. Neural Comput. 14(5), 1105–1114 (2002)

    Article  MATH  Google Scholar 

  9. Das, K., Bhaduri, K., Votava, P.: Distributed anomaly detection using 1-class SVM for vertically partitioned data. Stat. Anal. Data Min. 4(4), 393–406 (2011)

    Article  MathSciNet  Google Scholar 

  10. Datta, S., Kargupta, H.: Uniform data sampling from a peer-to-peer network. In: Proceedings of the 27th International Conference on Distributed Computing Systems (ICDCS), pp. 1–8 (June 2007)

    Google Scholar 

  11. Do, T.N., Poulet, F.: Classifying one billion data with a new distributed SVM algorithm. In: International Conference on Research, Innovation and Vision for the Future, pp. 59–66 (February 2006)

    Google Scholar 

  12. Flouri, K., Beferull-Lozano, B., Tsakalides, P.: Training a SVM-based classifier in distributed sensor networks. In: EUSIPCO 2006 (2006)

    Google Scholar 

  13. Flouri, K., Beferull-Lozano, B., Tsakalides, P.: Distributed consensus algorithms for SVM training in wireless sensor networks. In: EUSIPCO (2008)

    Google Scholar 

  14. Flouri, K., Beferull-Lozano, B., Tsakalides, P.: Optimal gossip algorithm for distributed consensus SVM training in wireless sensor networks. In: 16th International Conference on Digital Signal Processing, pp. 1–6 (July 2009)

    Google Scholar 

  15. Forero, P.A., Cano, A., Giannakis, G.B.: Consensus-based distributed support vector machines. J. Mach. Learn. Res. 11, 1663–1707 (2010)

    MathSciNet  MATH  Google Scholar 

  16. Graf, H.P., Cosatto, E., Bottou, L., Dourdanovic, I., Vapnik, V.: Parallel support vector machines: the cascade SVM. In: Saul, L., Weiss, Y., Bottou, L. (eds.) Advances in Neural Information Processing Systems, vol. 17, pp. 521–528. MIT Press, Cambridge (2005)

    Google Scholar 

  17. Hazan, T., Man, A., Shashua, A.: A parallel decomposition solver for SVM: distributed dual ascend using fenchel duality. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (June 2008)

    Google Scholar 

  18. Joachims, T.: Making large-scale support vector machine learning practical. In: Advances in Kernel Methods, pp. 169–184. MIT Press, Cambridge (1999)

    Google Scholar 

  19. Lee, S., Stolpe, M., Morik, K.: Separable approximate optimization of support vector machines for distributed sensing. In: Flach, P.A., De Bie, T., Cristianini, N. (eds.) ECML PKDD 2012, Part II. LNCS, vol. 7524, pp. 387–402. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  20. Lu, Y., Roychowdhury, V.P.: Parallel randomized support vector machine. In: Ng, W.-K., Kitsuregawa, M., Li, J., Chang, K. (eds.) PAKDD 2006. LNCS (LNAI), vol. 3918, pp. 205–214. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  21. Mangasarian, O.L., Wild, E.W., Fung, G.M.: Privacy-preserving classification of vertically partitioned data via random kernels. ACM Trans. Knowl. Discov. Data 2(3), 12:1–12:16 (2008)

    Article  Google Scholar 

  22. Moya, M., Koch, M., Hostetler, L.: One-class classifier networks for target recognition applications. In: Proceedings of World Congress on Neural Networks, pp. 797–801. International Neural Network Society (1993)

    Google Scholar 

  23. Osuna, E., Freund, R., Girosi, F.: An improved training algorithm for support vector machines, pp. 276–285. IEEE (1997)

    Google Scholar 

  24. Pechyony, D., Shen, L., Jones, R.: Solving large scale linear SVM with distributed block minimization. In: NIPS 2011 Workshop on Big Learning: Algorithms, Systems and Tools for Learning at Scale (2011)

    Google Scholar 

  25. Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. In: Advances in kernel methods, pp. 185–208. MIT Press, Cambridge (1999)

    Google Scholar 

  26. Rüping, S.: Incremental learning with support vector machines. In: Proceedings of the 2001 IEEE International Conference on Data Mining (ICDM), pp. 641–642 (2001)

    Google Scholar 

  27. Schölkopf, B., Platt, J.C., Shawe-Taylor, J.C., Smola, A.J., Williamson, R.C.: Estimating the support of a high-dimensional distribution. Neural Comp. 13(7), 1443–1471 (2001)

    Article  MATH  Google Scholar 

  28. Stolpe, M., Bhaduri, K., Das, K., Morik, K.: Anomaly detection in vertically partitioned data by distributed core vector machines. In: Blockeel, H., Kersting, K., Nijssen, S., Železný, F. (eds.) ECML PKDD 2013, Part III. LNCS, vol. 8190, pp. 321–336. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  29. Suykens, J., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9(3), 293–300 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  30. Syed, N.A., Huan, S., Kah, L., Sung, K.: Incremental learning with support vector machines. In: International Joint Conference on Artificial Intelligence (IJCAI) (1999)

    Google Scholar 

  31. Tanenbaum, A., van Steen, M.: Distributed Systems: Principles and Paradigms, 2nd edn. Prentice Hall, Upper Saddle River (2006)

    MATH  Google Scholar 

  32. Tax, D.M.J., Duin, R.P.W.: Support vector data description. Mach. Learn. 54, 45–66 (2004)

    Article  MATH  Google Scholar 

  33. Tsang, I., Kwok, J., Cheung, P.: Core vector machines: fast svm training on very large data sets. J. Mach. Learn. Res. 6, 363–392 (2005)

    MathSciNet  MATH  Google Scholar 

  34. Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)

    Book  MATH  Google Scholar 

  35. Yu, H.F., Hsieh, C.J., Chang, K.W., Lin, C.J.: Large linear classification when data cannot fit in memory. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 833–842. ACM, New York (2010)

    Google Scholar 

  36. Yu, H., Vaidya, J., Jiang, X.: Privacy-preserving svm classification on vertically partitioned data. In: Ng, W.-K., Kitsuregawa, M., Li, J., Chang, K. (eds.) PAKDD 2006. LNCS (LNAI), vol. 3918, pp. 647–656. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  37. Yunhong, H., Liang, F., Guoping, H.: Privacy-preserving SVM classification on vertically partitioned data without secure multi-party computation. In: 5th International Conference on Natural Computation (ICNC), vol. 1, pp. 543–546 (August 2009)

    Google Scholar 

  38. Zanghirati, G., Zanni, L.: A parallel solver for large quadratic programs in training support vector machines. Parallel Comput. 29(4), 535–551 (2003)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work has been supported by the DFG, Collaborative Research Center SFB 876 (http://sfb876.tu-dortmund.de/), project B3.

Author information

Authors and Affiliations

  1. TU Dortmund, Computer Science, LS 8, 44221, Dortmund, Germany

    Marco Stolpe

  2. Netflix Inc., Los Gatos, CA, 94032, USA

    Kanishka Bhaduri

  3. UARC, NASA Ames, Mountain View, CA, 94035, USA

    Kamalika Das

Authors
  1. Marco Stolpe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kanishka Bhaduri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kamalika Das
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marco Stolpe .

Editor information

Editors and Affiliations

  1. TU Dortmund , Dortmund, Germany

    Stefan Michaelis

  2. TU Dortmund , Dortmund, Germany

    Nico Piatkowski

  3. TU Dortmund , Dortmund, Germany

    Marco Stolpe

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Stolpe, M., Bhaduri, K., Das, K. (2016). Distributed Support Vector Machines: An Overview. In: Michaelis, S., Piatkowski, N., Stolpe, M. (eds) Solving Large Scale Learning Tasks. Challenges and Algorithms. Lecture Notes in Computer Science(), vol 9580. Springer, Cham. https://doi.org/10.1007/978-3-319-41706-6_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-41706-6_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-41705-9

  • Online ISBN: 978-3-319-41706-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation