Cost-Sensitive Learning of SVM for Ranking

  • Jun Xu
  • Yunbo Cao
  • Hang Li
  • Yalou Huang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4212)


In this paper, we propose a new method for learning to rank. ‘Ranking SVM’ is a method for performing the task. It formulizes the problem as that of binary classification on instance pairs and performs the classification by means of Support Vector Machines (SVM). In Ranking SVM, the losses for incorrect classifications of instance pairs between different rank pairs are defined as the same. We note that in many applications such as information retrieval the negative effects of making errors between higher ranks and lower ranks are larger than making errors among lower ranks. Therefore, it is natural to bring in the idea of cost-sensitive learning to learning to rank, or more precisely, to set up different losses for misclassification of instance pairs between different rank pairs. Given a cost-sensitive loss function we can construct a Ranking SVM model on the basis of the loss function. Simulation results show that our method works better than Ranking SVM in practical settings of ranking. Experimental results also indicate that our method can outperform existing methods including Ranking SVM on real information retrieval tasks such as document search and definition search.


Support Vector Machine Information Retrieval Support Vector Machine Classifier Support Vector Machine Model Ranking Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Abe, N., Zadrozny, B., Langford, J.: An Iterative Method for Multi-class Cost-sensitive Learning. In: Proc. of 10th Inter. Conf. on KDD, Seattle, Washington, USA, pp. 3–11 (2004)Google Scholar
  2. 2.
    Baeza-Yates, R.A., Ribeiro-Neto, B.: Modern Information Retrieval. Addison-Wesley Longman Publishing Co., Inc, Boston (1999)Google Scholar
  3. 3.
    Bradford, J., Kunz, C., Kohavi, R., Brunk, C., Brodley, C.: Pruning decision trees with misclassification costs. In: Proc. of ECML, Chemnitz, Germany, pp. 131–136 (1998)Google Scholar
  4. 4.
    Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., Hullender, G.: Learning to Rank using Gradient Descent. In: Proc. of 22nd ICML, Bonn, Germany (2005)Google Scholar
  5. 5.
    Cao, Y., Xu, J., Liu, T., Li, H., Huang, Y., Hon, H.W.: An Ordinal Regression Method for Document Retrieval. In: Proc. of 29th Inter. ACM SIGIR Conf. (to appear, 2006)Google Scholar
  6. 6.
    Chu, W., Keerthi, S.: New Approaches to Support Vector Ordinal Regression. In: Proc. of ICML, Bonn, Germany, pp. 145–152 (2005)Google Scholar
  7. 7.
    Crammer, K., Singer, Y.: PRanking with Ranking. In: Advances in NIPS 14, pp. 641–647. MIT Press, Cambridge (2002)Google Scholar
  8. 8.
    Domingos, P.: MetaCost: A general method for making classifiers cost sensitive. In: Proc. of the 5th KDD, San Diego, CA, USA, pp. 155–164 (1999)Google Scholar
  9. 9.
    Elkan, C.: The Foundations of Cost-Sensitive Learning. In: Proc. of the 17th Inter. Joint Conf. on Artificial Intelligence, pp. 973–978 (2001)Google Scholar
  10. 10.
    Frank, E., Hall, M.: A Simple Approach to Ordinal Classification. In: Proc. of ECML, Freiburg, Germany, pp. 145–165 (2001)Google Scholar
  11. 11.
    Geibel, P., Wysotzki, F.: Perceptron based learning with example dependent and noisy costs. In: Proc. of 12th ICML, Washington, DC, USA (2003)Google Scholar
  12. 12.
    Har-Peled, S., Roth, D., Zimak, D.: Constraint classification: A new approach to multiclass classification and ranking. In: Advances in NIPS 15 (2002) Google Scholar
  13. 13.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: data mining, inference and prediction. Springer, Heidelberg (2001)zbMATHGoogle Scholar
  14. 14.
    Herbrich, R., Graepel, T., Obermayer, K.: Large Margin Rank Boundaries for Ordinal Regression. In: Advances in Large Margin Classifiers, pp. 115–132 (2000) Google Scholar
  15. 15.
    Hersh, W.R., Buckley, C., Leone, T.J., Hickam, D.H.: OHSUMED: An interactive retrieval evaluation and new large test collection for research. In: Proc. of the 17th Inter. ACM SIGIR Conf., pp. 192–201 (1994)Google Scholar
  16. 16.
    Jarvelin, K., Kekalainen, J.: IR evaluation methods for retrieving highly relevant documents. In: Proc. of the 23rd Inter. ACM SIGIR Conf., pp. 41–48 (2000)Google Scholar
  17. 17.
    Kramer, S., Widmer, G., Pfahringer, B., Degroeve, M.: Prediction of ordinal classes using regression trees. Fundamenta Informaticae 47, 1–13 (2001)zbMATHMathSciNetGoogle Scholar
  18. 18.
    Monard, M.C., Batista, G.E.A.P.A.: Learning with Skewed Class Distribution. Advances in Logic, Artificial Intelligence and Robotics, Sao Paulo, SP, 173–180 (2002)Google Scholar
  19. 19.
    Nallapati, R.: Discriminative models for information retrieval. In: Proc. of the 27th Inter. ACM SIGIR Conf., Sheffield, United Kingdom, pp. 64–71 (2004)Google Scholar
  20. 20.
    Ponte, J., Croft, W.B.: A language model approach to information retrieval. In: Proc. of the Inter. ACM SIGIR Conf., pp. 275–281 (1998)Google Scholar
  21. 21.
    Robertson, S., Hull, D.A.: The TREC-9 Filtering Track Final Report. In: Proc. of the 9th TREC, pp. 25–40 (2000)Google Scholar
  22. 22.
    Spink, A., Jansen, B.J., Wolfram, D., Saracevic, T.: From e-sex to e-commerce: Web search changes. IEEE Computer 35(3), 107–109 (2002)Google Scholar
  23. 23.
    Xu, J., Cao, Y., Li, H., Zhao, M.: Ranking Definitions with Supervised Learning Methods. In: Proc. of the 14th Inter. Conf. on World Wide Web, pp. 811–819 (2005)Google Scholar
  24. 24.
    Zadrozny, B., Elkan, C.: Learning and making decisions when costs and probabilities are both unknown. In: Proc. of the 7th Inter. Conf. on KDD, pp. 204–213 (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Jun Xu
    • 1
  • Yunbo Cao
    • 2
  • Hang Li
    • 2
  • Yalou Huang
    • 1
  1. 1.College of SoftwareNankai UniversityTianjinChina
  2. 2.Microsoft Research AsiaBeijingChina

Personalised recommendations