Skip to main content
SpringerLink
Log in

DICE: A New Family of Bivariate Estimation of Distribution Algorithms Based on Dichotomised Multivariate Gaussian Distributions

  • Conference paper
  • First Online:
Applications of Evolutionary Computation (EvoApplications 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10199))

Included in the following conference series:

Abstract

A new family of Estimation of Distribution Algorithms (EDAs) for discrete search spaces is presented. The proposed algorithms, which we label DICE (Discrete Correlated Estimation of distribution algorithms) are based, like previous bivariate EDAs such as MIMIC and BMDA, on bivariate marginal distribution models. However, bivariate models previously used in similar discrete EDAs were only able to exploit an O(d) subset of all the \(O(d^{2})\) bivariate variable dependencies between d variables. We introduce, and utilize in DICE, a model based on dichotomised multivariate Gaussian distributions. These models are able to capture and make use of all \(O(d^{2})\) bivariate variable interactions in binary and multary search spaces. This paper tests the performances of these new EDA models and algorithms on a suite of challenging combinatorial optimization problems, and compares their performances to previously used discrete-space bivariate EDA models. EDAs utilizing these new dichotomised Gaussian (DG) models exhibit significantly superior optimization performances, with the performance gap becoming more marked with increasing dimensionality.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Notes

  1. 1.

    Strictly speaking, CMA-ES does not quite fall into the canonical EDA framework as given in Algorithm 1. However, it shares almost all of the core features of a typical EDA.

  2. 2.

    Downloadable from: http://www.math.nus.edu.sg/~matsundf/.

References

  1. Aizerman, A., Braverman, E., Rozoner, L.: Theoretical foundations of the potential function method in pattern recognition learning. Autom. Remote Control 25, 821–837 (1964)

    Google Scholar 

  2. Altenberg, L.: NK fitness landscapes. In: Back, T., Fogel, D., Michalewicz, Z. (eds.) Handbook of Evolutionary Computation, pp. B2.7:5–B2.7:10. Oxford University Press, New York (1997)

    Google Scholar 

  3. Baluja, S., Caruana, R.: Removing the genetics from the standard genetic algorithm. In: Machine Learning: Proceedings of the Twelfth International Conference, pp. 38–46 (1995)

    Google Scholar 

  4. Baluja, S., Davies, S.: Using optimal dependency-trees for combinational optimization. In: Proceedings of the Fourteenth International Conference on Machine Learning, pp. 30–38. Morgan Kaufmann Publishers (1997)

    Google Scholar 

  5. Boros, E., Hammer, P., Tavares, G.: Local search heuristics for quadratic unconstrained binary optimization (QUBO). J. Heuristics 13(2), 99–132 (2007)

    Article  Google Scholar 

  6. Caprara, A., Furini, F., Lodi, A., Mangia, M., Rovatti, R., Setti, G.: Generation of antipodal random vectors with prescribed non-stationary 2-nd order statistics. IEEE Trans. Sig. Process. 62(6), 1603–1612 (2014)

    Article  MathSciNet  Google Scholar 

  7. Chang, Y.W., Hsieh, C.J., Chang, K.W., Ringgaard, M., Lin, C.J.: Training and testing low-degree polynomial data mappings via linear SVM. J. Mach. Learn. Res. 11(Apr), 1471–1490 (2010)

    MathSciNet  MATH  Google Scholar 

  8. Chow, C., Liu, C.: Approximating discrete probability distributions with dependence trees. IEEE Trans. Inform. Theory 14(3), 462–467 (1968)

    Article  MATH  Google Scholar 

  9. Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge University Press, New York (2000)

    Book  MATH  Google Scholar 

  10. De Bonet, J., Isbell, C., Viola, P., et al.: MIMIC: finding optima by estimating probability densities. In: Advances in Neural Information Processing Systems, pp. 424–430 (1997)

    Google Scholar 

  11. Emrich, L., Piedmonte, M.: A method for generating high-dimensional multivariate binary variates. Am. Stat. 45(4), 302–304 (1991)

    Google Scholar 

  12. Etxeberria, R., Larranaga, P.: Global optimization using Bayesian networks. In: Second Symposium on Artificial Intelligence (CIMAF-99), Habana, Cuba, pp. 332–339 (1999)

    Google Scholar 

  13. Gange, S.: Generating multivariate categorical variates using the iterative proportional fitting algorithm. Am. Stat. 49(2), 134–138 (1995)

    Google Scholar 

  14. Genz, A., Bretz, F.: Computation of Multivariate Normal and t Probabilities, vol. 195. Springer Science & Business Media, New York (2009)

    MATH  Google Scholar 

  15. Glover, F., Hao, J.K., Kochenberger, G.: Polynomial unconstrained binary optimisation - part 2. Int. J. Metaheuristics 1(4), 317–354 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  16. Goldberg, Y., Elhadad, M.: SplitSVM: fast, space-efficient, non-heuristic, polynomial kernel computation for NLP applications. In: Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies: Short Papers, pp. 237–240. Association for Computational Linguistics (2008)

    Google Scholar 

  17. González-Fernández, Y., Soto, M.: A survey of estimation of distribution algorithms based on copulas. Technical report

    Google Scholar 

  18. Hansen, N., Kern, S.: Evaluating the CMA evolution strategy on multimodal test functions. In: International Conference on Parallel Problem Solving from Nature, pp. 282–291. Springer (2004)

    Google Scholar 

  19. Harik, G., Lobo, F., Goldberg, D.: The compact genetic algorithm. IEEE Trans. Evol. Comput. 3(4), 287–297 (1999)

    Article  Google Scholar 

  20. Harik, G., Lobo, F., Sastry, K.: Linkage learning via probabilistic modeling in the extended compact genetic algorithm(ECGA). In: Pelikan, M., Sastry, K., CantúPaz, E. (eds.) Scalable Optimization via Probabilistic Modeling, pp. 39–61. Springer, New York (2006)

    Chapter  Google Scholar 

  21. Heras, F., Larrosa, J., Oliveras, A.: MiniMaxSAT: an efficient weighted Max-SAT solver. J. Artif. Intell. Res. (JAIR) 31, 1–32 (2008)

    MathSciNet  MATH  Google Scholar 

  22. Higham, N.: Computing the nearest correlation matrix: a problem from finance. IMA J. Numer. Anal. 22(3), 329–343 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  23. Hoos, H., Stützle, T.: Stochastic Local Search: Foundations & Applications. Elsevier, Amsterdam (2004)

    MATH  Google Scholar 

  24. Hyrš, M., Schwarz, J.: Multivariate Gaussian copula in estimation of distribution algorithm with model migration. In: 2014 IEEE Symposium on Foundations of Computational Intelligence (FOCI), pp. 114–119. IEEE (2014)

    Google Scholar 

  25. Jin, R., Wang, S., Yan, F., Zhu, J.: Generating spatial correlated binary data through a copulas method. Sci. Res. 3(4), 206–212 (2015)

    Article  Google Scholar 

  26. Lane, F., Azad, R., Ryan, C.: Principled evolutionary algorithm design and the kernel trick. In: Proceedings of the 2016 on Genetic and Evolutionary Computation Conference Companion, pp. 149–150. ACM (2016)

    Google Scholar 

  27. Lane, F., Azad, R., Ryan, C.: Principled evolutionary algorithm search operator design and the kernel trick. In: 2016 IEEE Symposium on Model Based Evolutionary Algorithms (IEEE MBEA 2016), part of the IEEE Symposium Series on Computational Intelligence 2016, pp. 1–9 (2016)

    Google Scholar 

  28. Larrañaga, P., Etxeberria, R., Lozano, J., Peña, J.: Combinatorial optimization by learning and simulation of Bayesian networks. In: Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence, pp. 343–352. Morgan Kaufmann Publishers Inc. (2000)

    Google Scholar 

  29. Larranaga, P., Lozano, J., Bengoetxea, E.: Estimation of distribution algorithms based on multivariate normal and Gaussian networks. Technical report, EHU-KZAA-IK-1 (2001)

    Google Scholar 

  30. Lee, A.: Generating random binary deviates having fixed marginal distributions and specified degrees of association. Am. Stat. 47(3), 209–215 (1993)

    Google Scholar 

  31. Macke, J., Berens, P., Ecker, A., Tolias, A., Bethge, M.: Generating spike trains with specified correlation coefficients. Neural Comput. 21(2), 397–423 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  32. Macke, J., Murray, I., Latham, P.: How biased are maximum entropy models? In: Advances in Neural Information Processing Systems, pp. 2034–2042 (2011)

    Google Scholar 

  33. Mühlenbein, H.: The equation for response to selection and its use for prediction. Evol. Comput. 5(3), 303–346 (1997)

    Article  Google Scholar 

  34. Pelikan, M., Goldberg, D., Cantú-Paz, E.: BOA: the Bayesian optimization algorithm. In: Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation, vol. 1, pp. 525–532. Morgan Kaufmann Publishers (1999)

    Google Scholar 

  35. Pelikan, M., Mühlenbein, H.: The bivariate marginal distribution algorithm. In: Roy, R., Furuhashi, T., Chawdhry, P.K. (eds.) Advances in Soft Computing, pp. 521–535. Springer, New York (1999)

    Chapter  Google Scholar 

  36. Propp, J., Wilson, D.: Exact sampling with coupled Markov chains and applications to statistical mechanics. Random Struct. Algorithms 9(1–2), 223–252 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  37. Qi, H., Sun, D.: A quadratically convergent Newton method for computing the nearest correlation matrix. SIAM J. Matrix Anal. Appl. 28(2), 360–385 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  38. Qi, H., Sun, D.: An augmented Lagrangian dual approach for the H-weighted nearest correlation matrix problem. IMA J. Numer. Anal. 31(2), 491–511 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  39. Rasmussen, C., Williams, C.: Gaussian Processes for Machine Learning. The MIT Press, Cambridge (2006)

    MATH  Google Scholar 

  40. Salinas-Gutiérrez, R., Hernández-Aguirre, A., Villa-Diharce, E.R.: Using copulas in estimation of distribution algorithms. In: Aguirre, A.H., Borja, R.M., Garciá, C.A.R. (eds.) MICAI 2009. LNCS (LNAI), vol. 5845, pp. 658–668. Springer, Heidelberg (2009). doi:10.1007/978-3-642-05258-3_58

    Chapter  Google Scholar 

  41. Zhang, Q., Sun, J., Tsang, E., Ford, J.: Estimation of distribution algorithm based on mixture. Technical report

    Google Scholar 

Download references

Acknowledgements

This work was supported, in part, by Science Foundation Ireland grant 10/CE/I1855 to Lero - the Irish Software Engineering Research Centre (www.lero.ie).

Author information

Authors and Affiliations

  1. CSIS Department, University of Limerick, Limerick, Ireland

    Fergal Lane & Conor Ryan

  2. School of Computing and Digital Technology, Birmingham City University, Birmingham, UK

    R. Muhammad Atif Azad

Authors
  1. Fergal Lane
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. Muhammad Atif Azad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Conor Ryan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fergal Lane .

Editor information

Editors and Affiliations

  1. Politecnico di Torino, Turin, Italy

    Giovanni Squillero

  2. Edinburgh Napier University, Edinburgh, United Kingdom

    Kevin Sim

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Lane, F., Azad, R.M.A., Ryan, C. (2017). DICE: A New Family of Bivariate Estimation of Distribution Algorithms Based on Dichotomised Multivariate Gaussian Distributions. In: Squillero, G., Sim, K. (eds) Applications of Evolutionary Computation. EvoApplications 2017. Lecture Notes in Computer Science(), vol 10199. Springer, Cham. https://doi.org/10.1007/978-3-319-55849-3_43

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-55849-3_43

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-55848-6

  • Online ISBN: 978-3-319-55849-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation