Abstract
A widespread idea to attack ranking works by reducing it into a set of binary preferences and applying well studied classification techniques. The basic question addressed in this paper relates to whether an accurate classifier would transfer directly into a good ranker. In particular, we explore this reduction for subset ranking, which is based on optimization of DCG metric (Discounted Cumulated Gain), a standard position-sensitive performance measure. We propose a consistent reduction framework, guaranteeing that the minimal DCG regret is achievable by learning pairwise preferences assigned with importance weights. This fact allows us to further develop a novel upper bound on the DCG regret in terms of pairwise regrets. Empirical studies on benchmark datasets validate the proposed reduction approach with improved performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ailon, N., Mohri, M.: An efficient reduction of ranking to classification. In: Proc. 21st COLT, pp. 87–98 (2008)
Balcan, M.-F., Bansal, N., Beygelzimer, A., Coppersmith, D., Langford, J., Sorkin, G.B.: Robust reductions from ranking to classification. In: Bshouty, N.H., Gentile, C. (eds.) COLT. LNCS (LNAI), vol. 4539, pp. 604–619. Springer, Heidelberg (2007)
Beygelzimer, A., Dani, V., Hayes, T., Langford, J., Zadrozny, B.: Error limiting reductions between classification tasks. In: Proc. 22nd ICML, pp. 49–56 (2005)
Beygelzimer, A., Langford, J., Ravikumar, P.: Error-correcting tournaments. In: Gavaldà , R., Lugosi, G., Zeugmann, T., Zilles, S. (eds.) ALT 2009. LNCS, vol. 5809, pp. 247–262. Springer, Heidelberg (2009)
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., Hullender, G.: Learning to rank using gradient descent. In: Proc. 22nd ICML, pp. 89–96 (2005)
Burges, C.J.C., Ragno, R., Le, Q.V.: Learning to rank with non-smooth cost functions. In: Proc. 19th NIPS, pp. 193–200. MIT Press, Cambridge (2006)
Cortes, C., Mohri, M., Rastogi, A.: Magnitude-preserving ranking algorithms. In: Proc. 24th ICML, pp. 169–176 (2007)
Cossock, D., Zhang, T.: Statistical analysis of bayes optimal subset ranking. IEEE Transactions on Information Theory 54, 5140–5154 (2008)
Freund, Y., Iyer, R., Schapire, R.E., Singer, Y.: An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research 4, 933–969 (2003)
Herbrich, R., Graepel, T., Obermayer, K.: Support vector learning for ordinal regression. In: Proc. 9th ICANN, pp. 97–102 (1999)
Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems 20, 422–446 (2002)
Joachims, T.: Optimizing search engines using clickthrough data. In: Proc. 8th KDD, pp. 133–142. ACM Press, New York (2002)
Kendall, M.G.: A new measure of rank correlation. Biometrika 30, 81–93 (1938)
Langford, J., Beygelzimer, A.: Sensitive error correcting output codes. In: Auer, P., Meir, R. (eds.) COLT 2005. LNCS (LNAI), vol. 3559, pp. 158–172. Springer, Heidelberg (2005)
Le, Q.V., Smola, A.J.: Direct optimization of ranking measures. In: CoRR, abs/0704.3359 (2007)
Marshall, A., Olkin, I.: Inequalities: Theory of majorization and its applications. Mathematics in Science and Engineering, vol. 143 (1979)
Asia, M.R.: Letor3.0: benchmark datasets for learning to rank. Microsoft Corporation (2008)
Sun, Z.Y., Qin, T., Tao, Q., Wang, J.: Robust sparse rank learning for non-smooth ranking measures. In: Proc. 32nd SIGIR, pp. 259–266 (2009)
Zhang, T.: Statistical behavior and consistency of classification methods based on convex risk minimization. The Annuals of Statistics 32, 56–85 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sun, Z., Jin, W., Wang, J. (2011). Reducing Position-Sensitive Subset Ranking to Classification. In: Butz, C., Lingras, P. (eds) Advances in Artificial Intelligence. Canadian AI 2011. Lecture Notes in Computer Science(), vol 6657. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-21043-3_48
Download citation
DOI: https://doi.org/10.1007/978-3-642-21043-3_48
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-21042-6
Online ISBN: 978-3-642-21043-3
eBook Packages: Computer ScienceComputer Science (R0)