Online Metric Learning Methods Using Soft Margins and Least Squares Formulations

  • Adrian Perez-Suay
  • Francesc J. Ferri
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7626)


Online metric learning using margin maximization has been introduced as a way to learn appropriate dissimilarity measures in an efficient way when information as pairs of examples is given to the learning system in a progressive way. These schemes have several practical advantages with regard to global ones in which a training set needs to be processed. On the other hand, they may suffer from a poor performance depending on the quality of the examples and the particular tuning or other implementation details. This paper formulates several online metric learning alternatives using a passive-aggressive schema. A new formulation of the online problem using least squares is also introduced. The relative behavior of the different alternatives is studied and comparative experimentation is carried out to put forward the benefits and weaknesses of each alternative.


  1. 1.
    Globerson, A., Roweis, S.T.: Metric learning by collapsing classes. [13]Google Scholar
  2. 2.
    Weinberger, K.Q., Blitzer, J., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. [13]Google Scholar
  3. 3.
    Chechik, G., Sharma, V., Shalit, U., Bengio, S.: Large scale online learning of image similarity through ranking. Journal of Machine Learning Research 11, 1109–1135 (2010)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Shalev-Shwartz, S., Singer, Y., Ng, A.Y.: Online and batch learning of pseudo-metrics. In: Brodley, C.E. (ed.) ACM International Conference Proceeding Series, ICML, vol. 69, ACM (2004)Google Scholar
  5. 5.
    Davis, J.V., Kulis, B., Jain, P., Sra, S., Dhillon, I.S.: Information-theoretic metric learning. In: Ghahramani, Z. (ed.) ICML. ACM International Conference Proceeding Series, vol. 227, pp. 209–216. ACM (2007),
  6. 6.
    Shalev-Shwartz, S., Crammer, K., Dekel, O., Singer, Y.: Online passive-aggressive algorithms. In: Thrun, S., Saul, L.K., Schölkopf, B. (eds.) NIPS. MIT Press (2003)Google Scholar
  7. 7.
    Scholkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge (2001)Google Scholar
  8. 8.
    Perez-Suay, A., Ferri, F.J., Albert, J.V.: An Online Metric Learning Approach through Margin Maximization. In: Vitrià, J., Sanches, J.M., Hernández, M. (eds.) IbPRIA 2011. LNCS, vol. 6669, pp. 500–507. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  9. 9.
    Suykens, J.A.K., Vandewalle, J.: Least squares support vector machine classifiers. Neural Process. Lett. 9, 293–300 (1999)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Duin, R.P.W.: Prtools version 3.0: A matlab toolbox for pattern recognition. In: Proc. of SPIE, p. 1331 (2000)Google Scholar
  11. 11.
    Asuncion, A., Newman, D.J.: UCI machine learning repository (2007)Google Scholar
  12. 12.
    Arevalillo-Herrez, M., Ferri, F.J., Domingo, J.: A naive relevance feedback model for content-based image retrieval using multiple similarity measures. Pattern Recognition 43(3), 619–629 (2010)CrossRefGoogle Scholar
  13. 13.
    Advances in Neural Information Processing Systems 18 [Neural Information Processing Systems, NIPS 2005, December 5-8, 2005, Vancouver, British Columbia, Canada]. In: NIPS (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Adrian Perez-Suay
    • 1
  • Francesc J. Ferri
    • 1
  1. 1.Dept. InformàticaUniversitat de ValènciaBurjassotSpain

Personalised recommendations