Abstract
In past papers, we have introduced Empirical Model Learning (EML) as a method to enable Combinatorial Optimization on real world systems that are impervious to classical modeling approaches. The core idea in EML consists in embedding a Machine Learning model in a traditional combinatorial model. So far, the method has been demonstrated by using Neural Networks and Constraint Programming (CP). In this paper we add one more technique to the EML arsenal, by devising methods to embed Decision Trees (DTs) in CP. In particular, we propose three approaches: 1) a simple encoding based on meta-constraints; 2) a method using attribute discretization and a global table constraint; 3) an approach based on converting a DT into a Multi-valued Decision Diagram, which is then fed to an mdd constraint. We finally show how to embed in CP a Random Forest, a powerful type of ensemble classifier based on DTs. The proposed methods are compared in an experimental evaluation, highlighting their strengths and their weaknesses.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bartolini, A., Cacciari, M., Tilli, A., Benini, L.: Thermal and energy management of high-performance multicores: Distributed and self-calibrating model-predictive controller. IEEE Trans. Parallel Distrib. Syst. 24(1), 170–183 (2013)
Bartolini, A., Lombardi, M., Milano, M., Benini, L.: Neuron constraints to model complex real-world problems. In: Proc. of CP, pp. 115–129 (2011)
Battiti, R., Brunato, M.: The LION way: Machine Learning plus Intelligent Optimization. University of Trento, LIONlab (2014)
Beldiceanu, N., Simonis, H.: A model seeker: extracting global constraint models from positive examples. In: Proc. of CP, pp. 141–157 (2012)
Bennett, K.P., Parrado-Hernández, E.: The interplay of optimization and machine learning research. Journal of Machine Learning Research 7, 1265–1281 (2006)
Benoist, T., Estellon, B., Gardi, F., Megel, R., Nouioua, K.: Localsolver 1.x: a black-box local-search solver for 0–1 programming. 4OR 9(3), 299–316 (2011)
Bessière, C., Coletta, R., Freuder, E.C., O’Sullivan, B.: Leveraging the learning power of examples in automated constraint acquisition. In: Proc. of CP, pp. 123–137 (2004)
Bessière, C., Coletta, R., Hebrard, E., Katsirelos, G., Lazaar, N., Narodytska, N., Quimper, C., Walsh, T.: Constraint acquisition via partial queries. In: Proc. of IJCAI (2013)
Bessière, C., Coletta, R., O’Sullivan, B., Paulin, M.: Query-driven constraint acquisition. In: Proc. of IJCAI, pp. 50–55 (2007)
Breiman, L.: Random forests. Machine learning 45(1), 5–32 (2001)
Cheng, K.C.K., Yap, R.H.C.: An mdd-based generalized arc consistency algorithm for positive and negative table constraints and some global constraints. Constraints 15(2), 265–304 (2010)
Gardi, F., Benoist, T., Darlay, J., Estellon, B., Megel, R.: Mathematical Programming Solver Based on Local Search. John Wiley & Sons (2014)
Gent, I.P., Jefferson, C., Miguel, I., Nightingale, P.: Data structures for generalised arc consistency for extensional constraints. In: Proc. of AAAI, pp. 191–197 (2007)
Glover, F., Kelly, J.P., Laguna, M.: New Advances for Wedding optimization and simulation. In: Proc. of WSC, pp. 255–260 (1999)
Gopalakrishnan, K., Asce, A.M.: Neural Network - Swarm Intelligence Hybrid Nonlinear Optimization Algorithm for Pavement Moduli Back-Calculation. Journal of Transportation Engineering 136(6), 528–536 (2009)
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The weka data mining software: an update. ACM SIGKDD explorations newsletter 11(1), 10–18 (2009)
Hernando, Leticia, Mendiburu, Alexander, Lozano, Jose A.: Generating Customized Landscapes in Permutation-Based Combinatorial Optimization Problems. In: Nicosia, Giuseppe, Pardalos, Panos (eds.) LION 7. LNCS, vol. 7997, pp. 299–303. Springer, Heidelberg (2013)
Ho, T.K.: Random decision forests. In: Proc. of ICDAR, p. 278 (1995)
Howard, J., Dighe, S., et al.: A 48-Core IA-32 message-passing processor with DVFS in 45nm CMOS. In: Proc. of ISSCC, pp. 108–109, February 2010
Hutter, F., Xu, L., Hoos, H.H., Leyton-Brown, K.: Algorithm runtime prediction: Methods & evaluation. Artif. Intell. 206, 79–111 (2014)
Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. Journal of Global optimization 13(4), 455–492 (1998)
Kadioglu, S., Malitsky, Y., Sellmann, M., Tierney, K.: ISAC - instance-specific algorithm configuration. In: Proc. of ECAI, pp. 751–756 (2010)
Katsirelos, G., Walsh, T.: A compression algorithm for large arity extensional constraints. In: Proc. of CP, pp. 379–393 (2007)
Kotthoff, L., Gent, I.P., Miguel, I.: An evaluation of machine learning in algorithm selection for search problems. AI Commun. 25(3), 257–270 (2012)
Perron, L.: Operations Research and Constraint Programming at Google. In: Proc. of CP, p. 2 (2011)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann (1993)
Shaw, P.: Using constraint programming and local search methods to solve vehicle routing problems. In: Proc. of CP, pp. 417–431 (1998)
Sra, S., Nowozin, S., Wright, S.J.: Optimization for machine learning. MIT Press(2012)
Zaabab, A.H., Zhang, Q., Nakhla, M.: A neural network modeling approach to circuit optimization and statistical design. IEEE Transactions on Microwave Theory and Techniques 43(6), 1349–1358 (1995)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Bonfietti, A., Lombardi, M., Milano, M. (2015). Embedding Decision Trees and Random Forests in Constraint Programming. In: Michel, L. (eds) Integration of AI and OR Techniques in Constraint Programming. CPAIOR 2015. Lecture Notes in Computer Science(), vol 9075. Springer, Cham. https://doi.org/10.1007/978-3-319-18008-3_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-18008-3_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-18007-6
Online ISBN: 978-3-319-18008-3
eBook Packages: Computer ScienceComputer Science (R0)