Abstract
We consider the problem of ordinal classification, in which a value set of the decision attribute (output, dependent variable) is finite and ordered. This problem shares some characteristics of multi-class classification and regression, however, in contrast to the former, the order between class labels cannot be neglected, and, in the contrast to the latter, the scale of the decision attribute is not cardinal. In the paper, following the theoretical framework for ordinal classification, we introduce two algorithms based on gradient descent approach for learning ensemble of base classifiers being decision rules. The learning is performed by greedy minimization of so-called threshold loss, using a forward stagewise additive modeling. Experimental results are given that demonstrate the usefulness of the approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Błaszczyński, J., Dembczyński, K., Kotłowski, W., Słowiński, R., Szeląg, M.: Ensembles of Decision Rules. Foundations of Computing and Decision Sciences 31, 221–232 (2006)
Błaszczyński, J., Dembczyński, K., Kotłowski, W., Słowiński, R., Szeląg, M.: Ensembles of Decision Rules for Solving Binary Classification Problems in the Presence of Missing Values. In: Greco, S., Hata, Y., Hirano, S., Inuiguchi, M., Miyamoto, S., Nguyen, H.S., Słowiński, R. (eds.) RSCTC 2006. LNCS (LNAI), vol. 4259, pp. 224–234. Springer, Heidelberg (2006)
Chu, W., Keerthi, S.S.: New approaches to support vector ordinal regression. In: Proc. of 22nd International Conference on Machine Learning, pp. 321–328 (2005)
Clémençon, S., Lugosi, G., Vayatis, N.: Ranking and empirical minimization of U-statistics (to appear)
Cohen, W., Singer, Y.: A simple, fast, and effective rule learner. In: Proc. of 16th National Conference on Artificial Intelligence, pp. 335–342 (1999)
Frank, E., Hall, M.: A simple approach to ordinal classification. In: Flach, P.A., De Raedt, L. (eds.) ECML 2001. LNCS (LNAI), vol. 2167, pp. 145–157. Springer, Heidelberg (2001)
Freund, Y., Iyer, R., Schapire, R., Singer, Y.: An efficient boosting algorithm for combining preferences. J. of Machine Learning Research 4, 933–969 (2003)
Freund, Y., Schapire, R.: A decision-theoretic generalization of on-line learning and an application to boosting. J. of Computer and System Sciences 55(1), 119–139 (1997)
Friedman, J.: Greedy function approximation: A gradient boosting machine. The Annals of Statistics 29(5), 1189–1232 (2001)
Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: a statistical view of boosting. Annals of Statistics 28(2), 337–407 (2000)
Friedman, J., Popescu, B.: Predictive learning via rule ensembles. Research report, Dept. of Statistics, Stanford University (2005)
Hastie, T., Tibshirani, R., Friedman, J.: Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, Heidelberg (2003)
Herbrich, R., Graepel, T., Obermayer, K.: Regression models for ordinal data: A machine learning approach. Technical report TR-99/03, TU Berlin (1999)
Lin, H.T., Li, L.: Large-margin thresholded ensembles for ordinal regression: Theory and practice. In: Balcázar, J.L., Long, P.M., Stephan, F. (eds.) ALT 2006. LNCS (LNAI), vol. 4264, pp. 319–333. Springer, Heidelberg (2006)
Lin, H.T., Li, L.: Ordinal regression by extended binary classifications. Advances in Neural Information Processing Systems 19, 865–872 (2007)
Netflix prize, http://www.netflixprize.com
Rennie, J., Srebro, N.: Loss functions for preference levels: Regression with discrete ordered labels. In: Proc. of the IJCAI Multidisciplinary Workshop on Advances in Preference Handling (2005)
Shashua, A., Levin, A.: Ranking with large margin principle: Two approaches. Advances in Neural Information Processing Systems 15 (2003)
Weiss, S., Indurkhya, N.: Lightweight rule induction. In: Proc. of 17th International Conference on Machine Learning, pp. 1135–1142 (2000)
Witten, I., Frank, E.: Data Mining: Practical machine learning tools and techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dembczyński, K., Kotłowski, W., Słowiński, R. (2008). Ordinal Classification with Decision Rules. In: Raś, Z.W., Tsumoto, S., Zighed, D. (eds) Mining Complex Data. MCD 2007. Lecture Notes in Computer Science(), vol 4944. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-68416-9_14
Download citation
DOI: https://doi.org/10.1007/978-3-540-68416-9_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-68415-2
Online ISBN: 978-3-540-68416-9
eBook Packages: Computer ScienceComputer Science (R0)