Neural Networks to Guide the Selection of Heuristics within Constraint Satisfaction Problems

  • José Carlos Ortiz-Bayliss
  • Hugo Terashima-Marín
  • Santiago Enrique Conant-Pablos
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6718)


Hyper-heuristics are methodologies used to choose from a set of heuristics and decide which one to apply given some properties of the current instance. When solving a Constraint Satisfaction Problem, the order in which the variables are selected to be instantiated has implications in the complexity of the search. We propose a neural network hyper-heuristic approach for variable ordering within Constraint Satisfaction Problems. The first step in our approach requires to generate a pattern that maps any given instance, expressed in terms of constraint density and tightness, to one adequate heuristic. That pattern is later used to train various neural networks which represent hyper-heuristics. The results suggest that neural networks generated through this methodology represent a feasible alternative to code hyper-heuristic which exploit the strengths of the heuristics to minimise the cost of finding a solution.


Constraint Satisfaction Neural Networks Hyper-heuristics 


  1. 1.
    Bilgin, B., Özcan, E., Korkmaz, E.E.: An experimental study on hyper-heuristics and exam timetabling. In: Proceedings of the 6th International Conference on Practice and Theory of Automated Timetabling, pp. 123–140 (2006)Google Scholar
  2. 2.
    Bitner, J.R., Reingold, E.M.: Backtrack programming techniques. Commun. ACM 18, 651–656 (1975)CrossRefzbMATHGoogle Scholar
  3. 3.
    Bittle, S.A., Fox, M.S.: Learning and using hyper-heuristics for variable and value ordering in constraint satisfaction problems. In: Proceedings of the 11th Annual Conference Companion on Genetic and Evolutionary Computation Conference: Late Breaking Papers, GECCO 2009, pp. 2209–2212. ACM Press, New York (2009)Google Scholar
  4. 4.
    Burke, E., Hart, E., Kendall, G., Newall, J., Ross, P., Shulenburg, S.: Hyper-heuristics: an emerging direction in modern research technology. In: Handbook of metaheuristics, pp. 457–474. Kluwer Academic Publishers, Dordrecht (2003)CrossRefGoogle Scholar
  5. 5.
    Chakhlevitch, K., Cowling, P.: Hyperheuristics: Recent developments. In: Cotta, C., Sevaux, M., Sörensen, K. (eds.) Adaptive and Multilevel Metaheuristics, Studies in Computational Intelligence, vol. 136, pp. 3–29. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  6. 6.
    Crowston, W.B., Glover, F., Thompson, G.L., Trawick, J.D.: Probabilistic and parametric learning combinations of local job shop scheduling rules, p. 117 (1963)Google Scholar
  7. 7.
    Denzinger, J., Fuchs, M., Fuchs, M., Informatik, F.F., Munchen, T.: High performance atp systems by combining several ai methods. In: Proc. Fifteenth International Joint Conference on Artificial Intelligence (IJCAI 1997), pp. 102–107. Morgan Kaufmann, San Francisco (1997)Google Scholar
  8. 8.
    Fisher, H., Thompson, G.L.: Probabilistic learning combinations of local job-shop scheduling rules. In: Factory Scheduling Conference, Carnegie Institute of Technology (1961)Google Scholar
  9. 9.
    Freuder, E.C., Mackworth, A.K.: Constraint-Based Reasoning. MIT/Elsevier, Cambridge (1994)zbMATHGoogle Scholar
  10. 10.
    Garey, M.R., Johnson, D.S.: Computers and Intractability; A Guide to the Theory of NP-Completeness. W. H. Freeman & Co., New York (1979)zbMATHGoogle Scholar
  11. 11.
    Gent, I., MacIntyre, E., Prosser, P., Smith, B.: T.Walsh.: An empirical study of dynamic variable ordering heuristics for the constraint satisfaction problem. In: Proceedings of CP 1996, pp. 179–193 (1996)Google Scholar
  12. 12.
    Haralick, R.M., Elliott, G.L.: Increasing tree search efficiency for constraint satisfaction problems. Artificial Intelligence 14, 263–313 (1980)CrossRefGoogle Scholar
  13. 13.
    Jönsson, H., Söderberg, B.: An information-based neural approach to generic constraint satisfaction. Artificial Intelligence 142(1), 1–17 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Mackworth, A.K.: Consistency in networks of relations. Artificial Intelligence 8(1), 99–118 (1977)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Minton, S., Johnston, M.D., Phillips, A., Laird, P.: Minimizing conflicts: A heuristic repair method for csp and scheduling problems. Artificial Intellgence 58, 161–205 (1992)CrossRefzbMATHGoogle Scholar
  16. 16.
    Minton, S., Phillips, A., Laird, P.: Solving large-scale csp and scheduling problems using a heuristic repair method. In: Proceedings of the 8th AAAI Conference, pp. 17–24 (1990)Google Scholar
  17. 17.
    Montanari, U.: Networks of constraints: fundamentals properties and applications to picture processing. Information Sciences 7, 95–132 (1974)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Nadel, B.A.: Algorithms for constraint satisfaction: a survey. AI Magazine 13(1), 32–44 (1992)Google Scholar
  19. 19.
    Nakano, T., Nagamatu, M.: Lagrange neural network for solving csp which includes linear inequality constraints. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 943–948. Springer, Heidelberg (2005)Google Scholar
  20. 20.
    Ortiz-Bayliss, J.C., Özcan, E., Parkes, A.J., Terashima-Marín, H.: Mapping the performance of heuristics for constraint satisfaction. In: IEEE Congress on Evolutionary Computation (CEC 2010), pp. 1–8 (July 2010)Google Scholar
  21. 21.
    Özcan, E., Bilgin, B., Korkmaz, E.E.: A comprehensive analysis of hyper-heuristics. Intelligence Data Analysis 12(1), 3–23 (2008)Google Scholar
  22. 22.
    Prosser, P.: An empirical study of phase transitions in binary constraint satisfaction problems. Tech. Rep. Report AISL-49-94, University of Strathclyde (1994)Google Scholar
  23. 23.
    Purdom, P.W.: Search rearrangement backtracking and polynomial average time. Artificial Intelligence 21, 117–133 (1983)CrossRefGoogle Scholar
  24. 24.
    Ross, P., Marfn-Blazquez, J.: Constructive hyper-heuristics in class timetabling, vol. 2 (September 2005)Google Scholar
  25. 25.
    Rossi, F., Petrie, C., Dhar, V.: On the equivalence of constraint satisfaction problems. In: Proceedings of the 9th European Conference on Artificial Intelligence, pp. 550–556 (1990)Google Scholar
  26. 26.
    Russell, S., Norvig, P.: Artificial Intelligence A Modern Approach. Prentice-Hall, Englewood Cliffs (1995)zbMATHGoogle Scholar
  27. 27.
    Smith, B.M.: Locating the phase transition in binary constraint satisfaction problems. Artificial Intelligence 81, 155–181 (1996)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Terashima-Marín, H., Ross, P., Farías-Zárate, C., López-Camacho, E., Valenzuela-Rendón, M.: Generalized hyper-heuristics for solving 2d regular and irregular packing problems. Annals of Operations Research 179, 369–392 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Tsang, E.: Foundations of Constraint Satisfaction. Academic Press Limited, London (1993)Google Scholar
  30. 30.
    Tsang, E.P.K., Wang, C.J.: A generic neural network approach for constraint satisfaction problems. In: Neural Network Applications, pp. 12–22. Springer, Heidelberg (1992)CrossRefGoogle Scholar
  31. 31.
    Williams, C.P., Hogg, T.: Using deep structure to locate hard problems. In: Proc. of AAAI 1992, San Jose, CA, pp. 472–477 (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • José Carlos Ortiz-Bayliss
    • 1
  • Hugo Terashima-Marín
    • 1
  • Santiago Enrique Conant-Pablos
    • 1
  1. 1.Tecnológico de MonterreyMonterreyMexico

Personalised recommendations