Improving Effectiveness of Neighborhood-Based Algorithms for Optimization of Costly Pseudo-Boolean Black-Box Functions

  • Oleg ZaikinEmail author
  • Stepan Kochemazov
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12095)


Optimization of costly black-box functions is hard. Not only we know next to nothing about their nature, we need to calculate their values in as small number of points as possible. The problem is even more pronounced for pseudo-Boolean black-box functions since it is harder to approximate them. For such functions the local search methods where a neighborhood of a point must be traversed are in a particular disadvantage compared to evolutionary strategies. In the paper we propose two heuristics that make use of the search history to prioritize the more promising points from a neighborhood to be processed first. In the experiments involving minimization of an extremely costly pseudo-Boolean black-box function we show that the proposed heuristics significantly improve the performance of a hill climbing algorithm, making it outperform (1+1)-EA with an additional benefit of being more stable.


Pseudo-Boolean optimization Black-box optimization Local search Costly function Boolean satisfiability problem 



The research was partially supported by Russian Foundation for Basic Research (grant no. 19-07-00746-a). Stepan Kochemazov is additionally supported by the Council for Grants of the President of Russia (stipend SP-2017.2019.5).


  1. 1.
    Audet, C., Hare, W.: Derivative-Free and Blackbox Optimization. Springer Series in Operations Research and Financial Engineering. Springer, Heidelberg (2017). Scholar
  2. 2.
    Balyo, T., Biere, A., Iser, M., Sinz, C.: SAT race 2015. Artif. Intell. 241, 45–65 (2016)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bard, G.V.: Algebraic Cryptanalysis, 1st edn. Springer, Heidelberg (2009). Scholar
  4. 4.
    Biere, A., Heule, M., van Maaren, H., Walsh, T. (eds.): Handbook of Satisfiability. Frontiers in Artificial Intelligence and Applications, vol. 185. IOS Press, Amsterdam (2009)zbMATHGoogle Scholar
  5. 5.
    Brimberg, J., Hansen, P., Mladenovic, N., Taillard, É.D.: Improvement and comparison of heuristics for solving the uncapacitated multisource weber problem. Oper. Res. 48(3), 444–460 (2000). Scholar
  6. 6.
    De Cannière, C., Preneel, B.: Trivium. In: Robshaw, M., Billet, O. (eds.) New Stream Cipher Designs. LNCS, vol. 4986, pp. 244–266. Springer, Heidelberg (2008). Scholar
  7. 7.
    Irkutsk supercomputer center of SB RAS.
  8. 8.
    Costa, A., Nannicini, G.: RBFOpt: an open-source library for black-box optimization with costly function evaluations. Math. Program. Comput. 10(4), 597–629 (2018). Scholar
  9. 9.
    Droste, S., Jansen, T., Wegener, I.: On the analysis of the (1+1) evolutionary algorithm. Theor. Comput. Sci. 276(1–2), 51–81 (2002). Scholar
  10. 10.
    Günther, C.G.: Alternating step generators controlled by de Bruijn sequences. In: Chaum, D., Price, W.L. (eds.) EUROCRYPT 1987. LNCS, vol. 304, pp. 5–14. Springer, Heidelberg (1988). Scholar
  11. 11.
    Gutmann, H.M.: A radial basis function method for global optimization. J. Glob. Opt. 19(3), 201–227 (2001)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Hell, M., Johansson, T., Maximov, A., Meier, W.: The grain family of stream ciphers. In: Robshaw, M., Billet, O. (eds.) New Stream Cipher Designs. LNCS, vol. 4986, pp. 179–190. Springer, Heidelberg (2008). Scholar
  13. 13.
    Heule, M.J.H., Kullmann, O., Biere, A.: Cube-and-conquer for satisfiability. Handbook of Parallel Constraint Reasoning, pp. 31–59. Springer, Cham (2018). Scholar
  14. 14.
    Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Glob. Opt. 13(4), 455–492 (1998). Scholar
  15. 15.
    Kempe, D., Kleinberg, J., Tardos, E.: Maximizing the spread of influence through a social network. In: KDD 2003, pp. 137–146. Association for Computing Machinery, New York (2003).
  16. 16.
    Kochemazov, S., Zaikin, O.: ALIAS: a modular tool for finding backdoors for SAT. In: Beyersdorff, O., Wintersteiger, C.M. (eds.) SAT 2018. LNCS, vol. 10929, pp. 419–427. Springer, Cham (2018). Scholar
  17. 17.
    Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45(3), 385–482 (2003)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Menezes, A.J., Vanstone, S.A., Oorschot, P.C.V.: Handbook of Applied Cryptography, 1st edn. CRC Press Inc., Boca Raton (1996)zbMATHGoogle Scholar
  19. 19.
    Metropolis, N., Ulam, S.: The Monte Carlo method. J. Am. Stat. Assoc. 44(247), 335–341 (1949)CrossRefGoogle Scholar
  20. 20.
    Pavlenko, A., Buzdalov, M., Ulyantsev, V.: Fitness comparison by statistical testing in construction of SAT-based guess-and-determine cryptographic attacks. In: GECCO 2019, pp. 312–320 (2019).
  21. 21.
    Pavlenko, A., Semenov, A., Ulyantsev, V.: Evolutionary computation techniques for constructing SAT-based attacks in algebraic cryptanalysis. In: Kaufmann, P., Castillo, P.A. (eds.) EvoApplications 2019. LNCS, vol. 11454, pp. 237–253. Springer, Cham (2019). Scholar
  22. 22.
    Rios, L., Sahinidis, N.: Derivative-free optimization: a review of algorithms and comparison of software implementations. J. Glob. Opt. 56, 1247–1293 (2013). Scholar
  23. 23.
    Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach, 3rd edn. Prentice Hall, Upper Saddle River (2009)zbMATHGoogle Scholar
  24. 24.
    Semenov, A., Otpuschennikov, I., Gribanova, I., Zaikin, O., Kochemazov, S.: Translation of algorithmic descriptions of discrete functions to SAT with applications to cryptanalysis problems. Log. Methods Comput. Sci. 16, 29:1–29:42 (2020)zbMATHGoogle Scholar
  25. 25.
    Semenov, A., Zaikin, O.: Algorithm for finding partitionings of hard variants of Boolean satisfiability problem with application to inversion of some cryptographic functions. SpringerPlus 5(1), 1–16 (2016)CrossRefGoogle Scholar
  26. 26.
    Semenov, A., Zaikin, O., Otpuschennikov, I., Kochemazov, S., Ignatiev, A.: On cryptographic attacks using backdoors for SAT. In: AAAI 2018, pp. 6641–6648 (2018)Google Scholar
  27. 27.
    Verel, S., Derbel, B., Liefooghe, A., Aguirre, H., Tanaka, K.: A surrogate model based on walsh decomposition for pseudo-Boolean functions. In: Auger, A., Fonseca, C.M., Lourenço, N., Machado, P., Paquete, L., Whitley, D. (eds.) PPSN 2018. LNCS, vol. 11102, pp. 181–193. Springer, Cham (2018). Scholar
  28. 28.
    Wolfram, S.: Random sequence generation by cellular automata. Adv. Appl. Math. 7(2), 123–169 (1986)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Yasumoto, T., Okuwaga, T.: Rokk 1.0.1. In: SAT Competition 2014: Solver and Benchmark Descriptions. Series of Publications B, vol. B-2017-1, p. 70. Department of Computer Science, University of Helsinki (2014)Google Scholar
  30. 30.
    Zaikin, O., Kochemazov, S.: An improved SAT-based guess-and-determine attack on the alternating step generator. In: Nguyen, P., Zhou, J. (eds.) ISC 2017. LNCS, vol. 10599, pp. 21–38. Springer, Cham (2017). Scholar
  31. 31.
    Zaikin, O., Kochemazov, S.: Black-box optimization in an extended search space for SAT solving. In: Khachay, M., Kochetov, Y., Pardalos, P. (eds.) MOTOR 2019. LNCS, vol. 11548, pp. 402–417. Springer, Cham (2019). Scholar
  32. 32.
    Zaikin, O., Kochemazov, S.: On black-box optimization in divide-and-conquer SAT solving. Opt. Methods Softw., 1–25 (2019).

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.ISDCT SB RASIrkutskRussia

Personalised recommendations