Recent research in areas such as SAT solving and Integer Linear Programming has shown that the performances of a single arbitrarily efficient solver can be significantly outperformed by a portfolio of possibly slower on-average solvers. We report an empirical evaluation and comparison of portfolio approaches applied to Constraint Satisfaction Problems (CSPs). We compared models developed on top of off-the-shelf machine learning algorithms with respect to approaches used in the SAT field and adapted for CSPs, considering different portfolio sizes and using as evaluation metrics the number of solved problems and the time taken to solve them. Results indicate that the best SAT approaches have top performances also in the CSP field and are slightly more competitive than simple models built on top of classification algorithms.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Amadini, R., Gabbrielli, M., Mauro, J.: An Empirical Evaluation of Portfolios Approaches for solving CSPs. ArXiv e-prints (December 2012),
  2. 2.
    Arlot, S., Celisse, A.: A survey of cross-validation procedures for model selection (July 2009)Google Scholar
  3. 3.
    Gomes, C.P., Selman, B.: Algorithm portfolios. Artif. Intell. 126(1-2), 43–62 (2001)MathSciNetzbMATHCrossRefGoogle Scholar
  4. 4.
    Inc. Gurobi Optimization. Gurobi Optimizer Reference Manual (2012)Google Scholar
  5. 5.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: an update. SIGKDD Explor. Newsl. 11(1), 10–18 (2009)CrossRefGoogle Scholar
  6. 6.
    Kadioglu, S., Malitsky, Y., Sabharwal, A., Samulowitz, H., Sellmann, M.: Algorithm selection and scheduling. In: Lee, J. (ed.) CP 2011. LNCS, vol. 6876, pp. 454–469. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  7. 7.
    Kadioglu, S., Malitsky, Y., Sellmann, M., Tierney, K.: ISAC - Instance-Specific Algorithm Configuration. In: ECAI, pp. 751–756 (2010)Google Scholar
  8. 8.
    Kiziltan, Z., Mandrioli, L., Mauro, J., O’Sullivan, B.: A classification-based approach to managing a solver portfolio for CSPs. In: AICS (2011)Google Scholar
  9. 9.
    Leyton-Brown, K., Nudelman, E., Shoham, Y.: Learning the Empirical Hardness of Optimization Problems: The Case of Combinatorial Auctions. In: Van Hentenryck, P. (ed.) CP 2002. LNCS, vol. 2470, pp. 556–572. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  10. 10.
    Malitsky, Y., Sellmann, M.: Instance-Specific Algorithm Configuration as a Method for Non-Model-Based Portfolio Generation. In: Beldiceanu, N., Jussien, N., Pinson, É. (eds.) CPAIOR 2012. LNCS, vol. 7298, pp. 244–259. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  11. 11.
    Morara, M., Mauro, J., Gabbrielli, M.: Solving XCSP problems by using Gecode. In: CILC, pp. 401–405 (2011)Google Scholar
  12. 12.
    Nethercote, N., Stuckey, P.J., Becket, R., Brand, S., Duck, G.J., Tack, G.: MiniZinc: Towards a standard CP modelling language. In: Bessière, C. (ed.) CP 2007. LNCS, vol. 4741, pp. 529–543. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  13. 13.
    O’Mahony, E., Hebrard, E., Holland, A., Nugent, C., O’Sullivan, B.: Using case-based reasoning in an algorithm portfolio for constraint solving. In: AICS 2008 (2009)Google Scholar
  14. 14.
    Roussel, O., Lecoutre, C.: XML Representation of Constraint Networks: Format XCSP 2.1. CoRR, abs/0902.2362 (2009)Google Scholar
  15. 15.
    Silverthorn, B., Miikkulainen, R.: Latent class models for algorithm portfolio methods. In: Fox, M., Poole, D. (eds.) AAAI. AAAI Press (2010)Google Scholar
  16. 16.
    Xu, L., Hoos, H., Leyton-Brown, K.: Hydra: Automatically Configuring Algorithms for Portfolio-Based Selection. In: AAAI (2010)Google Scholar
  17. 17.
    Xu, L., Hutter, F., Hoos, H., Leyton-Brown, K.: Evaluating Component Solver Contributions to Portfolio-Based Algorithm Selectors. In: Cimatti, A., Sebastiani, R. (eds.) SAT 2012. LNCS, vol. 7317, pp. 228–241. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  18. 18.
    Xu, L., Hutter, F., Hoos, H.H., Leyton-Brown, K.: SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT. In: Bessière, C. (ed.) CP 2007. LNCS, vol. 4741, pp. 712–727. Springer, Heidelberg (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Roberto Amadini
    • 1
  • Maurizio Gabbrielli
    • 1
  • Jacopo Mauro
    • 1
  1. 1.Department of Computer Science and Engineering/Lab. Focus INRIAUniversity of BolognaItaly

Personalised recommendations