Skip to main content
SpringerLink
Log in
Book cover

International Conference on Algorithmic Learning Theory

ALT 2008: Algorithmic Learning Theory pp 7–21Cite as

Generalization Bounds for Some Ordinal Regression Algorithms

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 5254))

Abstract

The problem of ordinal regression, in which the goal is to learn a rule to predict labels from a discrete but ordered set, has gained considerable attention in machine learning in recent years. We study generalization properties of algorithms for this problem. We start with the most basic algorithms that work by learning a real-valued function in a regression framework and then rounding off a predicted real value to the closest discrete label; our most basic bounds for such algorithms are derived by relating the ordinal regression error of the resulting prediction rule to the regression error of the learned real-valued function. We end with a margin-based bound for the state-of-the-art ordinal regression algorithm of Chu & Keerthi (2007).

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. McCullagh, P., Nelder, J.A.: Generalized Linear Models, 2nd edn. Chapman and Hall, Boca Raton (1989)

    MATH  Google Scholar 

  2. Herbrich, R., Graepel, T., Obermayer, K.: Large margin rank boundaries for ordinal regression. In: Advances in Large Margin Classifiers, pp. 115–132. MIT Press, Cambridge (2000)

    Google Scholar 

  3. Kramer, S., Pfahringer, B., Widmer, G., Groeve, M.D.: Prediction of ordinal classes using regression trees. Fundamenta Informaticae 47, 1001–1013 (2001)

    Google Scholar 

  4. Frank, E., Hall, M.: A simple approach to ordinal classification. In: Proceedings of the 12th European Conference on Machine Learning, pp. 145–156 (2001)

    Google Scholar 

  5. Crammer, K., Singer, Y.: Online ranking by projecting. Neural Computation 17(1), 145–175 (2005)

    Article  MATH  Google Scholar 

  6. Shashua, A., Levin, A.: Ranking with large margin principle: Two approaches. In: Advances in Neural Information Processing Systems, vol. 15, pp. 937–944. MIT Press, Cambridge (2003)

    Google Scholar 

  7. Harrington, E.F.: Online ranking/collaborative filtering using the perceptron algorithm. In: Proceedings of the 20th International Conference on Machine Learning, pp. 250–257 (2003)

    Google Scholar 

  8. Chu, W., Ghahramani, Z.: Gaussian processes for ordinal regression. Journal of Machine Learning Research 6, 1019–1041 (2005)

    MathSciNet  Google Scholar 

  9. Rennie, J.D.M., Srebro, N.: Loss functions for preference levels: Regression with discrete ordered labels. In: Proc. IJCAI Multidisciplinary Workshop on Advances in Preference Handling (2005)

    Google Scholar 

  10. Chu, W., Keerthi, S.S.: Support vector ordinal regression. Neural Computation 19(3), 792–815 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  11. Cardoso, J.S., da Costa, J.F.P.: Learning to classify ordinal data: The data replication method. Journal of Machine Learning Research 8, 1393–1429 (2007)

    Google Scholar 

  12. Waegeman, W., De Baets, B., Boullart, L.: ROC analysis in ordinal regression learning. Pattern Recognition Letters 29(1), 1–9 (2008)

    Article  Google Scholar 

  13. Mathieson, M.J.: Ordinal models for neural networks. In: Neural Networks in Financial Engineering, pp. 523–536. World Scientific, Singapore (1996)

    Google Scholar 

  14. Crammer, K., Singer, Y.: Pranking with ranking. In: Advances in Neural Information Processing Systems, vol. 14, pp. 641–647. MIT Press, Cambridge (2002)

    Google Scholar 

  15. Shashua, A., Levin, A.: Taxonomy of large margin principle algorithms for ordinal regression problems. Technical Report 2002-39, Leibniz Center for Research, School of Computer Science and Engg., The Hebrew University of Jerusalem (2002)

    Google Scholar 

  16. Rajaram, S., Agarwal, S.: Generalization bounds for k-partite ranking. In: Proceedings of the NIPS-2005 Workshop on Learning to Rank (2005)

    Google Scholar 

  17. Bousquet, O., Elisseeff, A.: Stability and generalization. Journal of Machine Learning Research 2, 499–526 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  18. Anthony, M., Bartlett, P.L.: Learning in Neural Networks: Theoretical Foundations. Cambridge University Press, Cambridge (1999)

    Google Scholar 

  19. Agarwal, S., Niyogi, P.: Stability and generalization of bipartite ranking algorithms. In: Proceedings of the 18th Annual Conference on Learning Theory (2005)

    Google Scholar 

  20. Zhang, T.: Covering number bounds of certain regularized linear function classes. Journal of Machine Learning Research 2, 527–550 (2002)

    Article  MATH  Google Scholar 

  21. Zhang, T.: Statistical behavior and consistency of classification methods based on convex risk minimization. The Annals of Statistics 32, 56–85 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  22. Bartlett, P.L., Jordan, M.I., McAuliffe, J.D.: Convexity, classification, and risk bounds. Journal of the American Statistical Association 101(473), 138–156 (2006)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Massachusetts Institute of Technology, Cambridge, MA 02139, USA

    Shivani Agarwal

Authors
  1. Shivani Agarwal
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Computer Science and Engineering, University of California, San Diego, USA

    Yoav Freund

  2. Department of Computer Science and Information Theory, Department of Computer Science and Budapest University of Technology and Economics, Stoczek u. 2, 1521, Budapest, Hungary

    László Györfi

  3. Department of Math., Stat. and Comp. Sci,, University of Illinois, 851 S. Morgan, IL 60607-7045, Chicago, USA

    György Turán

  4. Division of Computer Science, Hokkaido University, N-14, W-9, 060-0814, Sapporo, Japan

    Thomas Zeugmann

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Agarwal, S. (2008). Generalization Bounds for Some Ordinal Regression Algorithms. In: Freund, Y., Györfi, L., Turán, G., Zeugmann, T. (eds) Algorithmic Learning Theory. ALT 2008. Lecture Notes in Computer Science(), vol 5254. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87987-9_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-87987-9_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-87986-2

  • Online ISBN: 978-3-540-87987-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation