A Supervised Learning Approach to Construct Hyper-heuristics for Constraint Satisfaction

  • José Carlos Ortiz-Bayliss
  • Hugo Terashima-Marín
  • Santiago Enrique Conant-Pablos
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7914)


Hyper-heuristics are methodologies that choose from a set of heuristics and decide which one to apply given some properties of the current instance. When solving a constraint satisfaction problem, the order in which the variables are selected to be instantiated has implications in the complexity of the search. In this paper we propose a logistic regression model to generate hyper-heuristics for variable ordering within constraint satisfaction problems. The first step in our approach requires to generate a training set that maps any given instance, expressed in terms of some of their features, to one suitable variable ordering heuristic. This set is used later to train the system and generate a hyper-heuristic that decides which heuristic to apply given the current features of the instances at hand at different steps of the search. The results suggest that hyper-heuristics generated through this methodology allow us to exploit the strengths of the heuristics to minimize the cost of the search.


Constraint Satisfaction Hyper-heuristics Logistic Regression 


  1. 1.
    Bilgin, B., Özcan, E., Korkmaz, E.E.: An experimental study on hyper-heuristics and exam timetabling. In: Burke, E.K., Rudová, H. (eds.) PATAT 2007. LNCS, vol. 3867, pp. 394–412. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  2. 2.
    Bitner, J.R., Reingold, E.M.: Backtrack programming techniques. Communications of the ACM 18(11), 651–656 (1975)zbMATHCrossRefGoogle Scholar
  3. 3.
    Bittle, S.A., Fox, M.S.: Learning and using hyper-heuristics for variable and value ordering in constraint satisfaction problems. In: Proceedings of the 11th Annual Conference Companion on Genetic and Evolutionary Computation Conference: Late Breaking Papers, pp. 2209–2212. ACM (2009)Google Scholar
  4. 4.
    Boussemart, F., Hemery, F., Lecoutre, C., Sais, L.: Boosting systematic search by weighting constraints. In: European Conference on Artificial Intelligence (ECAI 2004), pp. 146–150 (2004)Google Scholar
  5. 5.
    Burke, E., Hart, E., Kendall, G., Newall, J., Ross, P., Shulenburg, S.: Hyper-heuristics: an emerging direction in modern research technology. In: Handbook of Metaheuristics, pp. 457–474. Kluwer Academic Publishers (2003)Google Scholar
  6. 6.
    Burke, E.K., Hyde, M., Kendall, G., Ochoa, G., Özcan, E., Woodward, J.R.: A classification of hyper-heuristic approaches. In: Gendreau, M., Potvin, J.Y. (eds.) Handbook of Metaheuristics. International Series in Operations Research and Management Science, vol. 146, pp. 449–468. Springer (2010)Google Scholar
  7. 7.
    Cowling, P., Kendall, G., Soubeiga, E.: Hyperheuristics: A robust optimisation method applied to nurse scheduling. In: Guervós, J.J.M., Adamidis, P.A., Beyer, H.-G., Fernández-Villacañas, J.-L., Schwefel, H.-P. (eds.) PPSN 2002. LNCS, vol. 2439, pp. 851–860. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  8. 8.
    Fisher, H., Thompson, G.L.: Probabilistic learning combinations of local job-shop scheduling rules. In: Factory Scheduling Conference. Carnegie Institute of Technology (1961)Google Scholar
  9. 9.
    Freuder, E.C., Mackworth, A.K.: Constraint-Based Reasoning. MIT/Elsevier (1994)Google Scholar
  10. 10.
    Freuder, E.C.: Synthesizing constraint expressions. Communications of the ACM 21(11), 958–966 (1978)MathSciNetzbMATHCrossRefGoogle Scholar
  11. 11.
    Gaschnig, J.G.: A general backtrack algorithm that eliminates most redundant tests. In: Proceedings of the 5th International Joint Conference on Artificial Intelligence, vol. 1, p. 457. Morgan Kaufmann Publishers (1977)Google Scholar
  12. 12.
    Gent, I., MacIntyre, E., Prosser, P., Smith, B., Wals, T.: An empirical study of dynamic variable ordering heuristics for the constraint satisfaction problem. In: Freuder, E.C. (ed.) CP 1996. LNCS, vol. 1118, pp. 179–193. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  13. 13.
    Haralick, R.M., Elliott, G.L.: Increasing tree search efficiency for constraint satisfaction problems. Artificial Intelligence 14, 263–313 (1980)CrossRefGoogle Scholar
  14. 14.
    Kumar, V.: Algorithms for constraint satisfaction: a survey. AI Magazine 13(1), 32–44 (1992)Google Scholar
  15. 15.
    Lecoutre, C., Boussemart, F., Hemery, F.: Backjump-based techniques versus conflict-directed heuristics. In: Proceedings of the 16th IEEE International Conference on Tools with Artificial Intelligence, ICTAI 2004, pp. 549–557. IEEE Computer Society, Washington, DC (2004)Google Scholar
  16. 16.
    Minton, S., Phillips, A., Laird, P.: Solving large-scale CSP and scheduling problems using a heuristic repair method. In: Proceedings of the 8th AAAI Conference, pp. 17–24 (1990)Google Scholar
  17. 17.
    Montanari, U.: Networks of constraints: fundamentals properties and applications to picture processing. Information Sciences 7, 95–132 (1974)MathSciNetzbMATHCrossRefGoogle Scholar
  18. 18.
    Ortiz-Bayliss, J., Terashima-Marín, H., Conant-Pablos, S.: Neural networks to guide the selection of heuristics within constraint satisfaction problems. In: Martínez-Trinidad, J., Carrasco-Ochoa, J., Ben-Youssef Brants, C., Hancock, E. (eds.) MCPR 2011. LNCS, vol. 6718, pp. 250–259. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  19. 19.
    Ortiz-Bayliss, J.C., Özcan, E., Parkes, A.J., Terashima-Marín, H.: Mapping the performance of heuristics for constraint satisfaction. In: Proceedings of the 2010 IEEE Congress on Evolutionary Computation (CEC 2010), pp. 1–8. IEEE Press (2010)Google Scholar
  20. 20.
    Ortiz-Bayliss, J.C., Terashima-Marín, H., Conant-Pablos, S.E.: Learning vector quantization for variable ordering in constraint satisfaction problems. Pattern Recogn. Lett. 34(4), 423–432 (2013)CrossRefGoogle Scholar
  21. 21.
    Özcan, E., Bilgin, B., Korkmaz, E.E.: A comprehensive analysis of hyper-heuristics. Intelligent Data Analysis 12(1), 3–23 (2008)Google Scholar
  22. 22.
    Purdom, P.W.: Search rearrangement backtracking and polynomial average time. Artificial Intelligence 21, 117–133 (1983)CrossRefGoogle Scholar
  23. 23.
    Rossi, F., Petrie, C., Dhar, V.: On the equivalence of constraint satisfaction problems. In: Proceedings of the 9th European Conference on Artificial Intelligence, pp. 550–556 (1990)Google Scholar
  24. 24.
    Russell, S., Norvig, P.: Artificial Intelligence A Modern Approach. Prentice Hall (1995)Google Scholar
  25. 25.
    Smith, B.M.: Locating the phase transition in binary constraint satisfaction problems. Artificial Intelligence 81, 155–181 (1996)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Storer, R.H., Wu, S.D., Vaccari, R.: New search spaces for sequencing problems with application to job shop scheduling. Management Science 38(10), 1495–1509 (1992)zbMATHCrossRefGoogle Scholar
  27. 27.
    Terashima-Marín, H., Ortiz-Bayliss, J.C., Ross, P., Valenzuela-Rendón, M.: Hyper-heuristics for the dynamic variable ordering in constraint satisfaction problems. In: Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation (GECCO 2008), pp. 571–578. ACM (2008)Google Scholar
  28. 28.
    Terashima-Marín, H., Ross, P., Valenzuela-Rendón, M.: Evolution of constraint satisfaction strategies in examination timetabling. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 1999), pp. 635–642. Morgan Kaufmann (1999)Google Scholar
  29. 29.
    Tsang, E., Kwan, A.: Mapping constraint satisfaction problems to algorithms and heuristics. Tech. Rep. CSM-198, Department of Computer Sciences, University of Essex (1993)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • José Carlos Ortiz-Bayliss
    • 1
  • Hugo Terashima-Marín
    • 2
  • Santiago Enrique Conant-Pablos
    • 2
  1. 1.Automated Scheduling, Optimisation and Planning (ASAP), School of Computer ScienceUniversity of NottinghamUK
  2. 2.Tecnológico de MonterreyMexico

Personalised recommendations