Advertisement

Fast and Provably Effective Multi-view Classification with Landmark-Based SVM

  • Valentina ZantedeschiEmail author
  • Rémi Emonet
  • Marc Sebban
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11052)

Abstract

We introduce a fast and theoretically founded method for learning landmark-based SVMs in a multi-view classification setting which leverages the complementary information of the different views and linearly scales with the size of the dataset. The proposed method – called MVL-SVM – applies a non-linear projection to the dataset through multi-view similarity estimates w.r.t. a small set of randomly selected landmarks, before learning a linear SVM in this latent space joining all the views. Using the uniform stability framework, we prove that our algorithm is robust to slight changes in the training set leading to a generalization bound depending on the number of views and landmarks. We also show that our method can be easily adapted to a missing-view scenario by only reconstructing the similarities to the landmarks. Empirical results, both in complete and missing view settings, highlight the superior performances of our method, in terms of accuracy and execution time, w.r.t. state of the art techniques. Code related to this paper is available at: https://github.com/vzantedeschi/multiviewLSVM.

Keywords

Multi-view learning Linear SVM Landmark induced latent space Uniform stability Missing views 

Notes

Acknowledgments

This work has been funded by the ANR projects LIVES (ANR-15-CE23-0026-03) and SOLSTICE (ANR-13-BS02-01).

References

  1. 1.
    Amini, M., Usunier, N., Goutte, C.: Learning from multiple partially observed views-an application to multilingual text categorization. In: Advances in Neural Information Processing Systems, pp. 28–36 (2009)Google Scholar
  2. 2.
    Bakır, G., Bottou, L., Weston, J.: Breaking SVM complexity with cross training. In: Advances in Neural Information Processing Systems, vol. 17, pp. 81–88 (2005)Google Scholar
  3. 3.
    Bellet, A., Habrard, A., Sebban, M.: A survey on metric learning for feature vectors and structured data (2013). http://arxiv.org/abs/1306.6709
  4. 4.
    Bhadra, S., Kaski, S., Rousu, J.: Multi-view kernel completion. Mach. Learn. 106(5), 713–739 (2017)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: Proceedings of the Eleventh Annual Conference on Computational Learning Theory, pp. 92–100. ACM (1998)Google Scholar
  6. 6.
    Bordes, A., Bottou, L., Gallinari, P.: SGD-QN: careful quasi-newton stochastic gradient descent. J. Mach. Learn. Res. 10, 1737–1754 (2009)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Bousquet, O., Elisseeff, A.: Stability and generalization. J. Mach. Learn. Res. 2, 499–526 (2002)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Chen, Y., et al.: The UCR time series classification archive, July 2015. www.cs.ucr.edu/~eamonn/time_series_data/
  9. 9.
    Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)zbMATHGoogle Scholar
  10. 10.
    Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: LIBLINEAR: a library for large linear classification. J. Mach. Learn. Res. 9, 1871–1874 (2008)zbMATHGoogle Scholar
  11. 11.
    Farquhar, J., Hardoon, D., Meng, H., Shawe-taylor, J.S., Szedmak, S.: Two view learning: SVM-2K, theory and practice. In: Advances in Neural Information Processing Systems, pp. 355–362 (2006)Google Scholar
  12. 12.
    Gönen, M., Alpaydın, E.: Multiple kernel learning algorithms. J. Mach. Learn. Res. 12, 2211–2268 (2011)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Goyal, A., Morvant, E., Germain, P., Amini, M.-R.: PAC-Bayesian analysis for a two-step hierarchical multiview learning approach. In: Ceci, M., Hollmén, J., Todorovski, L., Vens, C., Džeroski, S. (eds.) ECML PKDD 2017. LNCS (LNAI), vol. 10535, pp. 205–221. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-71246-8_13CrossRefGoogle Scholar
  14. 14.
    Huusari, R., Kadri, H., Capponi, C.: Multi-view metric learning in vector-valued kernel spaces. In: Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, pp. 415–424 (2018)Google Scholar
  15. 15.
    Kadri, H., Ayache, S., Capponi, C., Koço, S., Dupé, F.X., Morvant, E.: The multi-task learning view of multimodal data. In: Asian Conference on Machine Learning, pp. 261–276 (2013)Google Scholar
  16. 16.
    Ladicky, L., Torr, P.: Locally linear support vector machines. In: Proceedings of the 28th International Conference on Machine Learning (ICML-11), pp. 985–992 (2011)Google Scholar
  17. 17.
    Micchelli, C.A., Pontil, M.: On learning vector-valued functions. Neural Comput. 17(1), 177–204 (2005)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Minh, H.Q., Bazzani, L., Murino, V.: A unifying framework for vector-valued manifold regularization and multi-view learning. In: ICML (2), pp. 100–108 (2013)Google Scholar
  19. 19.
    Minh, H.Q., Bazzani, L., Murino, V.: A unifying framework in vector-valued reproducing kernel Hilbert spaces for manifold regularization and co-regularized multi-view learning. J. Mach. Learn. Res. 17(25), 1–72 (2016)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Steinwart, I.: Sparseness of support vector machines. J. Mach. Learn. Res. 4, 1071–1105 (2003)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Sun, S.: Multi-view Laplacian support vector machines. In: Tang, J., King, I., Chen, L., Wang, J. (eds.) ADMA 2011. LNCS (LNAI), vol. 7121, pp. 209–222. Springer, Heidelberg (2011).  https://doi.org/10.1007/978-3-642-25856-5_16CrossRefGoogle Scholar
  22. 22.
    Sun, S.: A survey of multi-view machine learning. Neural Comput. Appl. 23(7–8), 2031–2038 (2013)CrossRefGoogle Scholar
  23. 23.
    Tang, J., Tian, Y., Zhang, P., Liu, X.: Multiview privileged support vector machines. IEEE Trans. Neural Netw. Learn. Syst. (2017)Google Scholar
  24. 24.
    Trivedi, A., Rai, P., Daumé III, H., DuVall, S.L.: Multiview clustering with incomplete views. In: NIPS Workshop (2010)Google Scholar
  25. 25.
    Williams, C.K., Seeger, M.: Using the Nyström method to speed up kernel machines. In: Advances in Neural Information Processing Systems, pp. 682–688 (2001)Google Scholar
  26. 26.
    Xu, C., Tao, D., Xu, C.: A survey on multi-view learning. CoRR abs/1304.5634 (2013), http://arxiv.org/abs/1304.5634
  27. 27.
    Yang, L., Jin, R.: Distance metric learning: a comprehensive survey. Michigan State Univ. 2(2), 4 (2006)Google Scholar
  28. 28.
    Zantedeschi, V., Emonet, R., Sebban, M.: \(L^3\)-SVMs: landmarks-based linear local support vectors machines. arXiv preprint arXiv:1703.00284 (2017)
  29. 29.
    Zhao, J., Xie, X., Xu, X., Sun, S.: Multi-view learning overview: recent progress and new challenges. Inf. Fusion 38, 43–54 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Valentina Zantedeschi
    • 1
    Email author
  • Rémi Emonet
    • 1
  • Marc Sebban
    • 1
  1. 1.Univ Lyon, UJM-Saint-Etienne, CNRS, Institut d Optique Graduate School, Laboratoire Hubert Curien UMR 5516Saint-EtienneFrance

Personalised recommendations