Heavy-tailed distributions in combinatorial search

  • Carla P. Gomes
  • Bart Selman
  • Nuno Crato
Session 2b
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1330)


Combinatorial search methods often exhibit a large variability in performance. We study the cost profiles of combinatorial search procedures. Our study reveals some intriguing properties of such cost profiles. The distributions are often characterized by very long tails or “heavy tails”. We will show that these distributions are best characterized by a general class of distributions that have no moments (i.e., an infinite mean, variance, etc.). Such non-standard distributions have recently been observed in areas as diverse as economics, statistical physics, and geophysics. They are closely related to fractal phenomena, whose study was introduced by Mandelbrot. We believe this is the first finding of these distributions in a purely computational setting. We also show how random restarts can effectively eliminate heavy-tailed behavior, thereby dramatically improving the overall performance of a search procedure.


Heavy Tail Stable Distribution Cost Distribution Steiner Triple System Combinatorial Search 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Anderson, L. (1985). Completing Partial Latin Squares. Mathematisk Fysiske Meddelelser, 41, 1985, 23–69.Google Scholar
  2. Brelaz, D. (1979). New methods to color the vertices of a graph. Comm. of the ACM (1979) 251–256.Google Scholar
  3. Chambers, John M., Mallows, C.L., and Stuck, B.W. (1976) A method for simulating stable random variables. Journal of the American Statistical Association 71, 340–344.Google Scholar
  4. Cheeseman, Peter and Kanefsky, Bob and Taylor, William M. (1991). Where the Really Hard Problems Are. Proceedings IJCAI-91, 1991, 163–169.Google Scholar
  5. Colbourn, C. (1983). Embedding Partial Steiner Triple Systems is NP-Complete. J. Combin. Theory (A) 35 (1983), 100–105.Google Scholar
  6. Dechter, R. (1991) Constraint networks. Encyclopedia of Artificial Intelligence John Wiley, New York (1991) 276–285.Google Scholar
  7. de Lima, Pedro J.F. (1997). On the robustness of nonlinearity tests to moment condition failure. Journal of Econometrics 76, 251–280.Google Scholar
  8. Denes, J. and Keedwell, A. (1974) Latin Squares and their Applications. Akademiai Kiado, Budapest, and English Universities Press, London, 1974.Google Scholar
  9. Frost, Daniel, Rish, Irina, and Vila, Lluís (1997) Summarizing CSP hardness with continuous probability distributions. Proc. AAAI-97.Google Scholar
  10. Fujita, M., Slaney, J., and Bennett, F. (1993). Automatic Generation of Some Results in Finite Algebra Proc. IJCAI, 1993.Google Scholar
  11. Freuder, E. and Mackworth, A. (Eds.). Constraint-basedreasoning. MIT Press, Cambridge, MA, USA, 1994.Google Scholar
  12. Gent, I. P. and Walsh, T.. (1993). Easy Problems are Sometimes Hard, the DIMACS Challenge on Satisfiability Testing. Piscataway, NJ, Oct. 1993. Full version AIJ (Hogg et al. 1996).Google Scholar
  13. Gent, I. and Walsh, T. (1996) The Satisfiability Constraint Gap. Artificial Intelligence, 81, 1996.Google Scholar
  14. Gomes, C.P. and Selman, B. (1997a) Problem Structure in the Presence of Perturbations Proc. AAAI-97, Providence, RI, 1997.Google Scholar
  15. Gomes, C.P. and Selman, B. (1997b) Algorithm Portfolio Design: Theory vs. Practice, Proc. UAI-97, Providence, RI, 1997.Google Scholar
  16. Hall, Peter (1982) On some simple estimates of an exponent of regular variation. Journal of the Royal Statistical Society, B 44, 37–42.Google Scholar
  17. Huberman, B.A., Lukose, R.M., and Hogg, T. (1997). An economics approach to hard computational problems. Science, 265, 51–54.Google Scholar
  18. Hogg, T., Huberman, B.A., and Williams, C.P. (Eds.) (1996). Phase Transitions and Complexity. Artificial Intelligence, 81 (Spec. Issue; 1996)Google Scholar
  19. Kirkpatrick, S. and Selman, B. (1994) Critical Behavior in the Satisfiability of Random Boolean Expressions. Science, 264 (May 1994) 1297–1301.Google Scholar
  20. Kwan, Alvin C. M. (1995) Validity of normality assumption in CSP research, PRICAI'96: Topics in Artificial Intelligence. Proceedings of the 4th Pacific Rim International Conference on Artificial Intelligence, 459–465.Google Scholar
  21. Lam, C., Thiel, L., and Swiercz, S. (1989) The Non-existence of Finite Projective Planes of Order 10. Can. J. Math., Vol. XLI, 6, 1989, 1117–1123.Google Scholar
  22. Mandelbrot, Benoit B. (1960) The Pareto-Lévy law and the distribution of income. International Economic Review 1, 79–106.Google Scholar
  23. Mandelbrot, Benoit B. (1963) The variation of certain speculative prices. Journal of Business 36, 394–419.Google Scholar
  24. Mandelbrot, B. (1983) The fractal geometry of nature. Freeman: New York. 1983.Google Scholar
  25. Mitchell, D., Selman, B., and Levesque, H.J. (1989) Hard and easy distributions of SAT problems. Proc. AAAI-92, San Jose, CA (1992) 459–465.Google Scholar
  26. Puget, J.-F. (1994) A C++ Implementation of CLP. Technical Report 94-01 ILOG S.A., Gentilly, France, (1994).Google Scholar
  27. Russell, S and Norvig P. (1995) Artificial Intelligence a Modern Approach. Prentice Hall, Englewood Cliffs, NJ. (1995).Google Scholar
  28. Samorodnitsky, Gennady and Taqqu, Murad S. (1994) Stable Non-Gaussian Random Processes: Stochastic Models with Infinite Variance, Chapman and Hall, New York.Google Scholar
  29. Selman, B. and Kirkpatrick, S. (1996) Finite-Size Scaling of the Computational Cost of Systematic Search. Artificial Intelligence, Vol. 81, 1996, 273–295.Google Scholar
  30. Smith, B. and Dyer, M. Locating the Phase Transition in Binary Constraint Satisfaction Problems. Artificial Intelligence, 81, 1996.Google Scholar
  31. Smith, B. and Grant S.A., Sparse Constraint Graphs and Exceptionally Hard Problems. IJCAI-95, 646–651, 1995. Full version in AIJ (Hogg et al. 1996).Google Scholar
  32. van Hentenryck, P., Deville, Y., and Teng Choh-Man (1992) A generic arc consistency algorithm and its specializations. Artificial Intelligence, 57, 1992.Google Scholar
  33. Williams, C.P. and Hogg, T. (1992) Using deep structure to locate hard problems. Proc. AAAI-92, San Jose, CA, July 1992, 472–277.Google Scholar
  34. Zhang, W. and Korf, R. A Study of Complexity Transitions on the Asymmetric Travelling Salesman Problem. Artificial Intelligence, 81, 1996.Google Scholar
  35. Zolotarev, V.M. (1986) One-dimensional Stable Distributions. Vol. 65 of “Translations of mathematical monographs”, American Mathematical Society. Translation from the original 1983 Russian Ed.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1997

Authors and Affiliations

  • Carla P. Gomes
    • 1
  • Bart Selman
    • 2
  • Nuno Crato
    • 3
  1. 1.Rome Laboratory, Rome LabUSA
  2. 2.Computer Science DepartmentCornell UniversityIthaca
  3. 3.Dept. of MathematicsNew Jersey Institute of TechnologyNewarkUSA

Personalised recommendations