Advertisement

Journal of Global Optimization

, Volume 66, Issue 4, pp 729–768 | Cite as

MONEDA: scalable multi-objective optimization with a neural network-based estimation of distribution algorithm

  • Luis Martí
  • Jesús García
  • Antonio Berlanga
  • José M. Molina
Article

Abstract

The extension of estimation of distribution algorithms (EDAs) to the multi-objective domain has led to multi-objective optimization EDAs (MOEDAs). Most MOEDAs have limited themselves to porting single-objective EDAs to the multi-objective domain. Although MOEDAs have proved to be a valid approach, the last point is an obstacle to the achievement of a significant improvement regarding “standard” multi-objective optimization evolutionary algorithms. Adapting the model-building algorithm is one way to achieve a substantial advance. Most model-building schemes used so far by EDAs employ off-the-shelf machine learning methods. However, the model-building problem has particular requirements that those methods do not meet and even evade. The focus of this paper is on the model-building issue and how it has not been properly understood and addressed by most MOEDAs. We delve down into the roots of this matter and hypothesize about its causes. To gain a deeper understanding of the subject we propose a novel algorithm intended to overcome the drawbacks of current MOEDAs. This new algorithm is the multi-objective neural estimation of distribution algorithm (MONEDA). MONEDA uses a modified growing neural gas network for model-building (MB-GNG). MB-GNG is a custom-made clustering algorithm that meets the above demands. Thanks to its custom-made model-building algorithm, the preservation of elite individuals and its individual replacement scheme, MONEDA is capable of scalably solving continuous multi-objective optimization problems. It performs better than similar algorithms in terms of a set of quality indicators and computational resource requirements.

Keywords

Multi-objective optimization problems Estimation of distribution algorithms Model-building problem Neural networks Growing neural gas 

Notes

Acknowledgments

This work has been funded in part by projects CNPq BJT 407851/2012-7, FAPERJ APQ1 211.451/2015, MINECO TEC2014-57022-C2-2-R and TEC2012-37832-C02-01.

References

  1. 1.
    Ahn, C.W.: Advances in Evolutionary Algorithms. Theory, Design and Practice. Springer, ISBN 3-540-31758-9 (2006)Google Scholar
  2. 2.
    Ahn, C.W., Ramakrishna, R.S.: Multiobjective real-coded Bayesian optimization algorithm revisited: Diversity preservation. In: GECCO’07. Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, ACM Press, New York, NY, USA, pp. 593–600, (2007). doi: 10.1145/1276958.1277079
  3. 3.
    Ahn, C.W., Goldberg, D.E., Ramakrishna, R.S.: Real-coded Bayesian optimization algorithm: bringing the strength of BOA into the continuous world. In: 2004 Genetic and Evolutionary Computation (GECCO 2004), Lecture Notes in Computer Science, vol. 3102, Springer, pp. 840–851 (2004)Google Scholar
  4. 4.
    Bäck, T.: Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms. Oxford University Press, New York (1996)MATHGoogle Scholar
  5. 5.
    Bäck, T., Fogel, D.B., Michalewicz, Z. (eds.): Handbook of Evolutionary Computation. Institute of Physics Publishing and Oxford University Press (1997)Google Scholar
  6. 6.
    Bader, J.: Hypervolume-Based Search for Multiobjective Optimization: Theory and Methods. PhD thesis, ETH Zurich, Switzerland (2010)Google Scholar
  7. 7.
    Bader, J., Zitzler, E.: HypE: An Algorithm for Fast Hypervolume-Based Many-Objective Optimization. TIK Report 286, Computer Engineering and Networks Laboratory (TIK), ETH Zurich (2008)Google Scholar
  8. 8.
    Bader, J., Zitzler, E.: HypE: an algorithm for fast hypervolume-based many-objective optimization. Evolut. Comput. 19(1), 45–76 (2011). doi: 10.1162/EVCO_a_00009 CrossRefGoogle Scholar
  9. 9.
    Bader, J., Deb, K., Zitzler, E.: Faster hypervolume-based search using Monte Carlo sampling. In: Beckmann, M., Künzi, H.P., Fandel, G., Trockel, W., Basile, A., Drexl, A., Dawid, H., Inderfurth, K., Kürsten, W., Schittko, U., Ehrgott, M., Naujoks, B., Stewart, T.J., Wallenius, J. (eds.) Multiple Criteria Decision Making for Sustainable Energy and Transportation Systems, Springer, Berlin/Heidelberg, Lecture Notes in Economics and Mathematical Systems, vol. 634, pp. 313–326, (2010). doi: 10.1007/978-3-642-04045-0_27
  10. 10.
    Bandyopadhyay, S., Pal, S.K., Aruna, B.: Multiobjective GAs, quantitative indices, and pattern classification. IEEE Trans. Syst. Man Cybern. Part B Cybern. 34(5), 2088–2099 (2004)CrossRefGoogle Scholar
  11. 11.
    Basseur, M., Zitzler, E.: Handling uncertainty in indicator-based multiobjective optimization. Int. J. Comput. Intell. Res. 2(3), 255–272 (2006)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Basseur, M., Zitzler, E.: A preliminary study on handling uncertainty in indicator-based multiobjective optimization. In: Rothlauf, F., Branke, J., Cagnoni, S., Costa, E., Cotta, C., Drechsler, R., Lutton, E., Machado, P., Moore, J.H., Romero, J., Smith, G.D., Squillero, G., Takagi, H. (eds.) Applications of Evolutionary Computing. EvoWorkshops 2006: EvoBIO, EvoCOMNET, EvoHOT, EvoIASP, EvoINTERACTION, EvoMUSART, and EvoSTOC, Springer, Budapest, Hungary, Lecture Notes in Computer Science, vol. 3907, pp. 727–739 (2006)Google Scholar
  13. 13.
    Bellman, R.E.: Adaptive Control Processes. Princeton University Press, Princeton (1961)MATHCrossRefGoogle Scholar
  14. 14.
    Benjamini, Y.: Opening the box of a boxplot. Am. Stat. 42(4), 257–262 (1988). doi: 10.2307/2685133 Google Scholar
  15. 15.
    Berkhin, P.: Survey of Clustering Data Mining Techniques. Tech. rep., Accrue Software, San Jose, CA. http://citeseer.ist.psu.edu/berkhin02survey.html. (2002)
  16. 16.
    Beume, N.: S-metric calculation by considering dominated hypervolume as Klee’s measure problem. Evolut. Comput. 17(4), 477–492 (2009). doi: 10.1162/evco.2009.17.4.17402 CrossRefGoogle Scholar
  17. 17.
    Beume, N., Rudolph, G.: Faster S-metric calculation by considering dominated hypervolume as Klee’s measure problem. In: Kovalerchuk, B. (ed.) Proceedings of the Second IASTED International Conference on Computational Intelligence. IASTED/ACTA Press, pp. 233–238 (2006)Google Scholar
  18. 18.
    Bleuler, S., Laumanns, M., Thiele, L., Zitzler, E.: PISA—a platform and programming language independent interface for search algorithms. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L. (eds.) Evolutionary Multi-Criterion Optimization. Second International Conference, EMO 2003, Springer. Lecture Notes in Computer Science, vol. 2632, Faro, Portugal, pp. 494–508 (2003)Google Scholar
  19. 19.
    Bosman, P.A., Thierens, D.: Adaptive variance scaling in continuous multi-objective estimation-of-distribution algorithms. In: Proceedings of the 9th annual conference on Genetic and evolutionary computation—GECCO’07. ACM Press, New York, New York, USA, p 500 (2007). doi: 10.1145/1276958.1277067. http://portal.acm.org/citation.cfm?doid=1276958.1277067
  20. 20.
    Bosman, P.A.N.: Design and Application of Iterated Density-Estimation Evolutionary Algorithms. PhD thesis, Institute of Information and Computing Sciences, Universiteit Utrecht, Utrecht, The Netherlands (2003)Google Scholar
  21. 21.
    Bosman, P.A.N.: The anticipated mean shift and cluster registration in mixture-based EDAs for multi-objective optimization. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation—GECCO’10. ACM Press, New York, New York, USA, p. 351 (2010). doi: 10.1145/1830483.1830549. http://portal.acm.org/citation.cfm?doid=1830483.1830549
  22. 22.
    Bosman, P.A.N., Thierens, D.: Multi-objective optimization with diversity preserving mixture-based iterated density estimation evolutionary algorithms. Int. J. Approx. Reason. 31(3), 259–289 (2002)MathSciNetMATHCrossRefGoogle Scholar
  23. 23.
    Bosman, P.A.N., Thierens, D.: The balance between proximity and diversity in multiobjective evolutionary algorithms. IEEE Trans. Evolut. Comput. 7(2), 174–188 (2003)CrossRefGoogle Scholar
  24. 24.
    Bosman, P.A.N, Thierens, D.: The naïve MIDEA: a baseline multi-objective EA. In: Coello, C.A.C., Hernández A.A., Zitzler, E, (eds.) Evolutionary Multi-Criterion Optimization. Third International Conference, EMO 2005, Springer. Lecture Notes in Computer Science, vol. 3410, Guanajuato, México, pp. 428–442 (2005)Google Scholar
  25. 25.
    Bosman, P.A.N., Grahl, J., Thierens, D.: Benchmarking parameter-free amalgam on functions with and without noise. Evolut. Comput. 21(3), 445–469 (2013). doi: 10.1162/EVCO_a_00094 CrossRefGoogle Scholar
  26. 26.
    Box, G.E.P., Muller, M.E.: A note on the generation of random normal deviates. Ann. Math. Stat. 29, 610–611 (1958)MATHCrossRefGoogle Scholar
  27. 27.
    Branke, J., Miettinen, K., Deb, K., Słowiǹski, R. (eds.) Multiobjective Optimization, Lecture Notes in Computer Science, vol. 5252. Springer, Berlin/Heidelberg (2008)Google Scholar
  28. 28.
    Brockhoff, D., Zitzler, E.: Dimensionality reduction in multiobjective optimization: the minimum objective subset problem. In: Waldmann, K.H., Stocker, U.M. (eds.) Operations Research Proceedings 2006. Springer, pp. 423–429 (2007)Google Scholar
  29. 29.
    Brockhoff, D., Zitzler, E.: Improving hypervolume-based multiobjective evolutionary algorithms by using objective reduction methods. In: IEEE Congress on Evolutionary Computation (CEC 2007). IEEE Press, pp. 2086–2093 (2007)Google Scholar
  30. 30.
    Brockhoff, D., Saxena, D.K., Deb, K., Zitzler, E.: On handling a large number of objectives a posteriori and during optimization. In: Knowles, J., Corne, D., Deb, K. (eds.) Multi-Objective Problem Solving from Nature: From Concepts to Applications, Natural Computing Series. Springer, pp. 377–403 (2008). doi: 10.1007/978-3-540-72964-8
  31. 31.
    Chambers, J., Cleveland, W., Kleiner, B., Tukey, P.: Graphical Methods for Data Analysis. Wadsworth, Belmont (1983)Google Scholar
  32. 32.
    Coello C.A.C.: 20 years of evolutionary multi-objective optimization: what has been done and what remains to be done. In: Yen, G.Y., Fogel, D.B. (eds.) Computational Intelligence: Principles and Practice, IEEE Computational Intelligence Society, Vancouver, Canada, chap 4, pp. 73–88 (2006)Google Scholar
  33. 33.
    Coello, C.A.C.: Evolutionary multiobjective optimization: a historical view of the field. IEEE Comput. Intell. Mag. 1(1), 28–36 (2006)Google Scholar
  34. 34.
    Coello C.A.C., Lamont, G.B., Van Veldhuizen D.A.: Evolutionary Algorithms for Solving Multi-Objective Problems, 2nd edn. Genetic and Evolutionary Computation, Springer, New York. http://www.springer.com/west/home/computer/foundations?SGWID=4-156-22-173660344-0 (2007)
  35. 35.
    Conover, W.J.: Practical Nonparametric Statistics, 3rd edn. Wiley, New York (1999)Google Scholar
  36. 36.
    Cooper, G., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9(4), 309–347 (1992)MATHGoogle Scholar
  37. 37.
    Corne, D.W., Knowles, J.D.: No free lunch and free leftovers theorems for multiobjective optimisation problems. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L. (eds.) Evolutionary Multi-Criterion Optimization. Second International Conference, EMO 2003, Springer, Faro, Portugal, Lecture Notes in Computer Science, vol. 2632, pp. 327–341 (2003)Google Scholar
  38. 38.
    Corne, D.W., Knowles, J.D., Oates. M.J.: The Pareto envelope-based selection algorithm for multiobjective optimization. In: Schoenauer, M., Deb, K., Rudolph, G., Yao, X., Lutton, E., Merelo, J.J., Schwefel, H.P. (eds.) Proceedings of the Parallel Problem Solving from Nature VI Conference, Springer. Lecture Notes in Computer Science No. 1917, Paris, France, pp. 839–848 (2000)Google Scholar
  39. 39.
    Corne, D.W., Jerram, N.R., Knowles, J.D., Oates, M.J.: PESA-II: region-based selection in evolutionary multiobjective optimization. In: Spector, L., Goodman, E.D., Wu, A., Langdon, W., Voigt, H.M., Gen. M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M.H., Burke, E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’2001), Morgan Kaufmann Publishers, San Francisco, California, pp. 283–290 (2001)Google Scholar
  40. 40.
    Costa, M., Minisci, E.: MOPED: a multi-objective Parzen-based estimation of distribution algorithm for continuous problems. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L. (eds.) Evolutionary Multi-Criterion Optimization. Second International Conference, EMO 2003, Springer. Lecture Notes in Computer Science, vol. 2632, Faro, Portugal, pp. 282–294 (2003)Google Scholar
  41. 41.
    Costa, M., Minisci, E., Pasero, E.: An hybrid neural/genetic approach to continuous multi-objective optimization problems. In: Apolloni, B., Marinaro, M., Tagliaferri, R. (eds.) Italian Workshop on Neural Neural Nets (WIRN), Springer, Lecture Notes in Computer Science, vol. 2859, pp. 61–69 (2003)Google Scholar
  42. 42.
    Darwin, C.: On the Origin of Species by Means of Natural Selection, or The Preservation of Favoured Races in the Struggle for Life. John Murray, London (1859)Google Scholar
  43. 43.
    De Jong, K.A.: Evolutionary Computation: A Unified Approach. MIT Press, Cambridge (2006)MATHGoogle Scholar
  44. 44.
    Deb, K.: Multi-Objective Optimization using Evolutionary Algorithms. Wiley, Chichester, UK. ISBN 0-471-87339-X (2001)Google Scholar
  45. 45.
    Deb, K., Saxena, D.K.: On finding Pareto-optimal solutions through dimensionality reduction for certain large-dimensional multi-objective optimization problems. Tech. Rep. 2005011, KanGAL (2005)Google Scholar
  46. 46.
    Deb, K., Saxena, D.K.: Searching for Pareto-optimal solutions through dimensionality reduction for certain large-dimensional multi-objective optimization problems. In: 2006 IEEE Conference on Evolutionary Computation (CEC’2006), IEEE Press, Piscataway, New Jersey, pp. 3352–3360 (2006)Google Scholar
  47. 47.
    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evolut. Comput. 6(2), 182–197 (2002)CrossRefGoogle Scholar
  48. 48.
    Deb, K., Thiele, L., Laumanns, M., Zitzler, E.: Scalable test problems for evolutionary multiobjective optimization. In: Abraham,A., Jain, L., Goldberg, R. (eds.) Evolutionary Multiobjective Optimization: Theoretical Advances and Applications, Advanced Information and Knowledge Processing, Springer Verlag, pp. 105–145 (2004)Google Scholar
  49. 49.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. B 39, 1–38 (1977)MathSciNetMATHGoogle Scholar
  50. 50.
    Ehrgott, M.: Multicriteria Optimization, Lecture Notes in Economics and Mathematical Systems, vol. 491. Springer (2005)Google Scholar
  51. 51.
    Etxeberria, R., Larrañaga, P.: Global optimization using Bayesian networks. In: Ochoa, A., Soto, M.R., Santana, R. (eds.) Proceedings of the Second Symposium on Artificial Intelligence (CIMAF-99), Habana, Cuba, pp. 151–173 (1999)Google Scholar
  52. 52.
    Fleischer, M.: The measure of Pareto optima. Applications to multi-objective metaheuristics. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L. (eds.) Evolutionary Multi-Criterion Optimization. Second International Conference, EMO 2003, Springer. Lecture Notes in Computer Science, vol. 2632, Faro, Portugal, pp. 519–533 (2003)Google Scholar
  53. 53.
    Flentge, F.: Locally weighted interpolating growing neural gas. IEEE Trans. Neural Netw. 17(6), 1382–1393 (2006)CrossRefGoogle Scholar
  54. 54.
    Fonseca, C., Fleming, P.: Multiobjective Genetic Algorithms. In: IEE Colloquium on Genetic Algorithms for Control Systems Engineering, IEE, pp. 6/1-6/5 (1993)Google Scholar
  55. 55.
    Fonseca, C.M., Fleming, P.J.: Genetic algorithms for multiobjective optimization: formulation, discussion and generalization. In: Forrest, S. (ed.) Proceedings of the FifthInternational Conference on Genetic Algorithms, University of Illinois at Urbana-Champaign, Morgan Kauffman Publishers, San Mateo, California, pp. 416–423 (1993)Google Scholar
  56. 56.
    Fonseca, C.M., Paquete, L., López-Ibánez, M.: An improved dimension-sweep algorithm for the hypervolume indicator. In: 2006 IEEE Congress on Evolutionary Computation (CEC’2006), pp. 1157–1163 (2006)Google Scholar
  57. 57.
    Fritzke, B.: Fast learning with incremental RBF networks. Neural Process. Lett. 1, 2–5 (1994)CrossRefGoogle Scholar
  58. 58.
    Fritzke, B.: A growing neural gas network learns topologies. In: Tesauro, G., Touretzky, D.S., Leen, T.K. (eds.) Advances in Neural Information Processing Systems, vol. 7, pp. 625–632. MIT Press, Cambridge (1995)Google Scholar
  59. 59.
    Fritzke, B.: Incremental neuro-fuzzy systems. In: Application of Soft Computing, SPIE International Symposium on Optical Science, Engineering and Instrumentation, San Diego, CA (1997)Google Scholar
  60. 60.
    Grünwald, P.D.: The Minimum Description Length Principle (Adaptive Computation and Machine Learning). The MIT Press (2007)Google Scholar
  61. 61.
    Hartigan, J.A.: Clustering Algorithms. Wiley Series in Probability and Mathematical Statistics. Wiley, New York (1975)Google Scholar
  62. 62.
    Horn, J., Nafpliotis, N., Goldberg, D.E.: A Niched Pareto genetic algorithm for multiobjective optimization. In: Proceedings of the First IEEE Conference on Evolutionary Computation, IEEE World Congress on Computational Intelligence, IEEE Service Center, Piscataway, New Jersey, vol. 1, pp. 82–87 (1994)Google Scholar
  63. 63.
    Huband, S., Barone, L., While, L., Hingston, P.: A scalable multi-objective test problem toolkit. In: Coello C.A.C., Hernández Aguirre, A., Zitzler, E. (eds.) Evolutionary Multi-Criterion Optimization. Third International Conference, EMO 2005, Springer. Lecture Notes in Computer Science, vol. 3410, Guanajuato, México, pp. 280–295 (2005)Google Scholar
  64. 64.
    Huband, S., Hingston, P., Barone, L., While, L.: A review of multiobjective test problems and a scalable test problem toolkit. IEEE Trans. Evolut. Comput. 10(5), 477–506 (2006)MATHCrossRefGoogle Scholar
  65. 65.
    Ishibuchi, H., Murata, T.: Multi-objective genetic local search algorithm and its application to flowshop scheduling. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 28(3), 392–403 (1998)CrossRefGoogle Scholar
  66. 66.
    Jain, A.K., Murty, M.N., Flynn, P.J.: Data clustering: a review. ACM Comput. Surv. 31(3), 264–323 (1999). doi: 10.1145/331499.331504 CrossRefGoogle Scholar
  67. 67.
    Kambhatla, N., Leen, T.K.: Dimension reduction by local principal component analysis. Neural Comput. 9(7), 1493–1516 (1997)CrossRefGoogle Scholar
  68. 68.
    Karshenas, H., Santana, R., Bielza, C., Larranaga, P.: Multiobjective estimation of distribution algorithm based on joint modeling of objectives and variables. IEEE Trans. Evolut. Comput. 18(4), 519–542 (2014). doi: 10.1109/TEVC.2013.2281524 CrossRefGoogle Scholar
  69. 69.
    Khan, N.: Bayesian Optimization Algorithms for Multiobjective and Hierarchically Difficult Problems. Master’s thesis, Graduate College of the University of Illinois at Urbana-Champaign, Urbana, Illinois, USA (2003)Google Scholar
  70. 70.
    Khan, N., Goldberg, D.E., Pelikan, M.: Multi-objective Bayesian optimization algorithm. In: Langdon, W., Cantú-Paz, E., Mathias, K., Roy, R., Davis, D., Poli, R., Balakrishnan, K., Honavar, V., Rudolph, G., Wegener, J., Bull, L., Potter, M., Schultz, A., Miller, J., Burke, E., Jonoska, N. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’2002), Morgan Kaufmann Publishers, San Francisco, California, p. 684 (2002)Google Scholar
  71. 71.
    Khare, V., Yao, X., Deb, K.: Performance scaling of multi-objective evolutionary algorithms. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L. (eds.) Evolutionary Multi-Criterion Optimization. Second International Conference, EMO 2003, Springer. Lecture Notes in Computer Science, vol. 2632, Faro, Portugal, pp. 376–390 (2003)Google Scholar
  72. 72.
    Knowles, J., Corne, D.: Quantifying the effects of objective space dimension in evolutionary multiobjective optimization. In: Obayashi, S., Deb, K., Poloni, C., Hiroyasu, T., Murata, T. (eds.) Proceedings 4th International Conference Evolutionary Multi-Criterion Optimization (EMO 2007), Springer, Berlin/Heidelberg, pp. 757–771 (2007). doi: 10.1007/978-3-540-70928-2_57
  73. 73.
    Knowles, J., Thiele, L., Zitzler, E.: A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers. 214, Computer Engineering and Networks Laboratory (TIK), ETH Zurich, Switzerland, revised version (2006)Google Scholar
  74. 74.
    Knowles, J., Thiele, L., Zitzler, E.: A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers. TIK Report 214, Computer Engineering and Networks Laboratory (TIK), ETH Zurich (2006)Google Scholar
  75. 75.
    Knowles, J.D.: Local-Search and Hybrid Evolutionary Algorithms for Pareto Optimization. PhD thesis, The University of Reading, Department of Computer Science, Reading, UK (2002)Google Scholar
  76. 76.
    Knowles, J.D., Corne, D.W.: The Pareto archived evolution strategy: a new baseline algorithm for multiobjective optimisation. In: 1999 Congress on Evolutionary Computation, IEEE Service Center, Washington, D.C., pp. 98–105 (1999)Google Scholar
  77. 77.
    Knowles, J.D., Corne, D.W.: Approximating the nondominated front using the Pareto archived evolution strategy. Evolut. Comput. 8(2), 149–172 (2000)CrossRefGoogle Scholar
  78. 78.
    Kruskal, W.H., Wallis, W.A.: Use of ranks in one-criterion analysis of variance. J. Am. Stat. Assoc. 47, 583–621 (1952)MATHCrossRefGoogle Scholar
  79. 79.
    Larrañaga, P.: A review on estimation of distribution algorithms. In: Larrañaga, P., Lozano, J.A. (eds.) Estimation of Distribution Algorithms. A New Tool forEvolutionary Computation, Kluwer Academic Publishers, Boston/Dordrecht/London, pp. 55–98 (2002)Google Scholar
  80. 80.
    Larrañaga, P., Lozano, J.A. (eds.) Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation. Genetic Algorithms and Evolutionary Computation, Kluwer Academic Publishers, Boston/Dordrecht/London (2002)Google Scholar
  81. 81.
    Laumanns, M., Ocenasek, J.: Bayesian optimization algorithms for multi-objective optimization. In: Merelo Guervós JJ, Adamidis P, Beyer HG, nas JLFV, Schwefel HP (eds.) Parallel Problem Solving from Nature–PPSN VII, Springer-Verlag. Lecture Notes in Computer Science No. 2439, Granada, Spain, pp. 298–307 (2002)Google Scholar
  82. 82.
    Levon, J.: OProfile manual. Victoria University of Manchester. http://oprofile.sourceforge.net/ (2004)
  83. 83.
    Lozano, J.A., Larrañaga, P., Inza, I., Bengoetxea, E.: (eds) Towards a New Evolutionary Computation: Advances on Estimation of Distribution Algorithms. Springer, Berlin (2006)Google Scholar
  84. 84.
    MacQueen, J.: Some methods for classification and analysis ofmultivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical, vol 1, pp. 281–297 (1967)Google Scholar
  85. 85.
    Mann, H.B., Whitney, D.R.: On a test of whether one of two random variables is stochastically larger than the other. Ann. Math. Stat. 18, 50–60 (1947)MathSciNetMATHCrossRefGoogle Scholar
  86. 86.
    Martí, L., García, J., Berlanga, A., Molina, J.M.: A cumulative evidential stopping criterion for multiobjective optimization evolutionary algorithms. In: Thierens, D., Deb, K., Pelikan, M., Beyer, H.G., Doerr, B., Poli, R., Bittari, M. (eds.) GECCO’09: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, ACM Press, New York, p. 911, (2007). doi: 10.1145/1276958.1277141. http://portal.acm.org/citation.cfm?doid=1276958.1277141
  87. 87.
    Martí, L., García, J., Berlanga, A., Molina, J.M.: Introducing MONEDA: scalable multiobjective optimization with a neural estimation of distribution algorithm. In: GECCO’08: 10th Annual Conference on Genetic and Evolutionary Computation, ACM Press, New York, NY, USA, pp. 689–696 (2008). doi: 10.1145/1389095.1389230
  88. 88.
    Martí, L., García, J., Berlanga, A., Molina, J.M.: Model-building algorithms for multiobjective EDAs: directions for improvement. In: 2008 IEEE Conference on Evolutionary Computation (CEC), Part of 2008 IEEE World Congress on Computational Intelligence (WCCI 2008), IEEE Press, Piscataway, New Jersey, pp. 2848–2855 (2008). doi: 10.1109/CEC.2008.4631179. http://ieeexplore.ieee.org/iel5/4625778/4630767/04631179.pdf?tp=&arnumber=4631179&isnumber=4630767
  89. 89.
    Martí, L., García, J., Berlanga, A., Molina, J.M.: An approach to stopping criteria for multi-objective optimization evolutionary algorithms: The MGBM criterion. In: 2009 IEEE Conference on Evolutionary Computation (CEC 2009), IEEE Press, Piscataway, New Jersey, pp. 1263–1270 (2009). doi: 10.1109/CEC.2009.4983090
  90. 90.
    Martí, L., García, J., Berlanga, A., Molina, J.M.: Solving complex high-dimensional problems with the multi-objective neural estimation of distribution algorithm. In: Raidl, G., Alba, E., Bacardit, J., Bates Congdon, C., Beyer, H.G., Birattari, M., Blum, C., Bosman, P.A.N., Corne, D., Cotta, C., Di Penta, M., Doerr, B., Drechsler, R., Ebner, M., Grahl, J., Jansen, T., Knowles, J., Lenaerts, T., Middendorf, M., Miller, J.F., O’Neill, M., Poli, R., Squillero, G., Stanley, K., Stützle, T., van Hemert, J. (eds.) GECCO 2009: 11th Annual Conference on Genetic and Evolutionary Computation, ACM Press, New York, NY, USA, pp. 619–626 (2009). doi: 10.1145/1569901.1569987. http://portal.acm.org/citation.cfm?id=1569901.1569987
  91. 91.
    Martí, L., García, J., Berlanga, A., Coello, C.A.C., Molina, J.M.: On Current Model-building Methods for Multi-objective Estimation of Distribution Algorithms: Shortcommings and Directions for Improvement. Tech. Rep. GIAA2010E001, Grupo de Inteligencia Artificial Aplicada, Universidad Carlos III de Madrid, Colmenarejo, Spain. http://www.giaa.inf.uc3m.es/miembros/lmarti/model-building (2010)
  92. 92.
    Martinetz, T.M.: Competitive Hebbian learning rule forms perfectly topology preserving maps. In: International Conference on Artificial Neural Networks (ICANN’93), Springer-Verlag, Amsterdam, pp. 427–434 (1993)Google Scholar
  93. 93.
    Martinetz, T.M., Berkovich, S.G., Shulten, K.J.: Neural-gas network for vector quantization and its application to time-series prediction. IEEE Trans. Neural Netw. 4, 558–560 (1993)CrossRefGoogle Scholar
  94. 94.
    Massey, F.J.: The Kolmogorov–Smirnov test for goodness of fit. J. Am. Stat. Assoc. 46(253), 68–78 (1951)MATHCrossRefGoogle Scholar
  95. 95.
    Miettinen, K.: Nonlinear Multiobjective Optimization, International Series in Operations Research & Management Science, vol. 12. Kluwer, Norwell, MA (1999)Google Scholar
  96. 96.
    Mühlenbein, H., Mahnig, T.: FDA—a scalable evolutionary algorithm for the optimization of additively decomposed functions. Evolut. Comput. 7(4), 353–376 (1999)CrossRefGoogle Scholar
  97. 97.
    Ocenasek, J.: Parallel Estimation of Distribution Algorithms. PhD thesis, Faculty of Information Technology, Brno University of Technology, Brno, Czech Republic (2002)Google Scholar
  98. 98.
    Ocenasek, J., Schwarz, J.: Estimation of distribution algorithm for mixed continuous-discrete optimization problems. In: 2nd Euro-International Symposium on Computational Intelligence, pp. 227–232 (2002)Google Scholar
  99. 99.
    Pareto, V.: Cours D’Économie Politique. F. Rouge, Lausanne(1896)Google Scholar
  100. 100.
    Parzen, E.: On estimation of a probability density function and mode. Ann. Math. Stat. 33, 1065–1076 (1962)MathSciNetMATHCrossRefGoogle Scholar
  101. 101.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco (1988)MATHGoogle Scholar
  102. 102.
    Pelikan, M.: Hierarchical Bayesian Optimization Algorithm. Toward a New Generation of Evolutionary Algorithms. Studies in Fuzziness and Soft Computing, Springer (2005)Google Scholar
  103. 103.
    Pelikan, M., Goldberg, D.E.: Hierarchical bayesian optimizationalgorithm. In: Pelikan, M., Sastry, K., Cantú-Paz, E. (eds.) Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications, Studies in Computational Intelligence, Springer–Verlag, pp. 63–90 (2006)Google Scholar
  104. 104.
    Pelikan, M., Goldberg, D.E., Cantú-Paz, E.: BOA: The Bayesian optimization algorithm. In: Banzhaf, W., Daida, J., Eiben, A.E., Garzon, M.H., Honavar, V., Jakiela, M., Smith, R.E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference GECCO-1999, Morgan Kaufmann Publishers, San Francisco, CA, Orlando, FL, vol. I, pp. 525–532 (1999)Google Scholar
  105. 105.
    Pelikan, M., Goldberg, D.E., Lobo, F. A survey of optimization by building and using probabilistic models. IlliGAL Report No. 99018, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL (1999)Google Scholar
  106. 106.
    Pelikan, M., Goldberg, D.E., Cantú-Paz, E.: Hierarchical problem solving by the Bayesian optimization algorithm. IlliGAL Report No. 2000002, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL (2000)Google Scholar
  107. 107.
    Pelikan, M., Sastry, K., Goldberg, D.E.: Multiobjective hBOA, clustering, and scalability. In: GECCO’05: Proceedings of the 2005 Conference on Genetic and Evolutionary Computation, ACM Press, New York, NY, USA, pp. 663–670 (2005). doi: 10.1145/1068009.1068122
  108. 108.
    Pelikan, M., Sastry, K., Cantú-Paz, E. (eds.) Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications. Studies in Computational Intelligence, Springer (2006)Google Scholar
  109. 109.
    Pelikan, M., Sastry, K., Goldberg, D.E. Multiobjective estimation ofdistribution algorithms. In: Pelikan, M., Sastry, K., Cantú-Paz, E. (eds.) Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications, Studies in Computational Intelligence, Springer–Verlag, pp. 223–248 (2006)Google Scholar
  110. 110.
    Praditwong, K., Yao, X.: How well do multi-objective evolutionary algorithms scale to large problems. In: 2007 IEEE Congress on Evolutionary Computation (CEC 2007), IEEE Press, Piscataway, New Jersey, pp. 3959–3966 (2007). doi: 10.1109/CEC.2007.4424987
  111. 111.
    Purshouse, R.C.: On the Evolutionary Optimisation of Many Objectives. PhD thesis, Department of Automatic Control and Systems Engineering, The University of Sheffield, Sheffield, UK (2003)Google Scholar
  112. 112.
    Purshouse, R.C., Fleming, P.J.: Evolutionary multi-objective optimisation: an exploratory analysis. In: Proceedings of the 2003 Congress on Evolutionary Computation (CEC’2003), IEEE Press, Canberra, Australia, vol. 3, pp. 2066–2073 (2003)Google Scholar
  113. 113.
    Purshouse, R.C., Fleming, P.J.: On the evolutionary optimization of many conflicting objectives. IEEE Trans. Evolut. Comput. 11(6), 770–784 (2007). doi: 10.1109/TEVC.2007.910138 CrossRefGoogle Scholar
  114. 114.
    Qin, A.K., Suganthan, P.N.: Robust growing neural gas algorithm with application in cluster analysis. Neural Netw. 17(8–9), 1135–1148 (2004). doi: 10.1016/j.neunet.2004.06.013 MATHCrossRefGoogle Scholar
  115. 115.
    Rubinstein, R.Y.: Simulation and the Monte Carlo Method. Wiley, New York (1981)MATHCrossRefGoogle Scholar
  116. 116.
    Schaffer, J.D.: Multiple objective optimization with vector evaluated genetic algorithms. In: Genetic Algorithms and their Applications: Proceedings of the First International Conference on Genetic Algorithms, Lawrence Erlbaum, pp. 93–100 (1985)Google Scholar
  117. 117.
    Schervish, M.J.: Theory of Statistics, 2nd edn. Springer Series in Statistics, Springer, Berlin/Heidelberg (1997)Google Scholar
  118. 118.
    Schütze, O., Mostaghim, S., Dellnitz, M., Teich, J.: Covering Pareto sets by multilevel evolutionary subdivision techniques. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L. (eds.) Evolutionary Multi-Criterion Optimization. Second International Conference, EMO 2003, Springer. Lecture Notes in Computer Science, vol. 2632, Faro, Portugal, pp. 118–132 (2003)Google Scholar
  119. 119.
    Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978)MathSciNetMATHCrossRefGoogle Scholar
  120. 120.
    Shapiro, J.: Diversity loss in general estimation of distribution algorithms. In: Parallel Problem Solving from Nature—PPSN IX, pp. 92–101 (2006). doi: 10.1007/11844297_10
  121. 121.
    Srinivas, N., Deb, K.: Multiobjective optimization using nondominated sorting in genetic algorithms. Evolut. Comput. 2(3), 221–248 (1994)CrossRefGoogle Scholar
  122. 122.
    Stewart, T.J., Bandte, O., Braun, H., Chakraborti, N., Ehrgott, M., Göbelt, M., Jin, Y., Nakayama, H., Poles, S., Di Stefano. D.: Real-world applications of multiobjective optimization. In: Branke, J., Miettinen, K., Deb, K., Słowiǹski, R. (eds.) Multiobjective Optimization, Lecture Notes in Computer Science, vol. 5252, Springer–Verlag, Berlin/Heidelberg, pp. 285–327 (2008)Google Scholar
  123. 123.
    Thierens, D.: Convergence time analysis for the multi-objective counting ones problem. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L. (eds.) Evolutionary Multi-Criterion Optimization. Second International Conference, EMO 2003, Springer. Lecture Notes in Computer Science, vol. 2632, Faro, Portugal, pp. 355–364 (2003)Google Scholar
  124. 124.
    Thierens, D., Bosman, P.A.N.: Multi-objective mixture-based iterated density estimation evolutionary algorithms. In: Spector, L., Goodman, E., Wu, A., Langdon, W., Voigt, H., Gen, M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M., Burke, E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference GECCO-2001, Morgan Kaufmann Publishers, San Francisco, CA, pp. 663–670 (2001)Google Scholar
  125. 125.
    Thierens, D., Bosman, P.A.N.: Multi-objective optimization with iterated density estimation evolutionary algorithms using mixture models. In: Proceedings of the Third International Symposium on Adaptive Systems-Evolutionary Computation and Probabilistic Graphical Models, Institute of Cybernetics, Mathematics and Physics, Havana, Cuba, pp. 129–136 (2001)Google Scholar
  126. 126.
    Timm, H., Borgelt, C., Doring, C., Kruse, R.: An extension to possibilistic fuzzy cluster analysis. Fuzzy Sets Syst. 147(1), 3–16 (2004)MathSciNetMATHCrossRefGoogle Scholar
  127. 127.
    Vapnik, V.N.: An overview of statistical learning theory. IEEE Trans. Neural Netw. 10(5), 988–999 (1999). doi: 10.1109/72.788640 CrossRefGoogle Scholar
  128. 128.
    Wang, H., Zhang, Q., Jiao, L., Yao, X.: Regularity model for noisy multiobjective optimization. IEEE Trans. Cybern. PP(99), pp. 1–1 (2015). doi: 10.1109/TCYB.2015.2459137
  129. 129.
    While, L., Bradstreet, L., Barone, L., Hingston, P.: Heuristics for optimising the calculation of hypervolume for multi-objective optimization problems. In: 2005 IEEE Congress on Evolutionary Computation (CEC’2005), IEEE Service Center, Edinburgh, Scotland, vol. 3, pp. 2225–2232 (2005)Google Scholar
  130. 130.
    While, L., Hingston, P., Barone, L., Huband, S.: A faster algorithm for calculating hypervolume. IEEE Trans. Evolut. Comput. 10(1), 29–38 (2006). doi: 10.1109/TEVC.2005.851275 CrossRefGoogle Scholar
  131. 131.
    Wilcoxon, F.: Individual comparisons by ranking methods. Biom. Bull. 1, 80–83 (1945)CrossRefGoogle Scholar
  132. 132.
    Xu, R., Wunsch II, D.: Survey of clustering algorithms. IEEE Trans. Neural Netw. 16(3), 645–678 (2005)CrossRefGoogle Scholar
  133. 133.
    Xu, R., Wunsch II, D.: Clustering, Illustrated edn. IEEE Press Series on Computational Intelligence, Wiley, IEEE Press, New York (2008)Google Scholar
  134. 134.
    Yuan, B., Gallagher, M.: On the importance of diversity maintenance in estimation of distribution algorithms. In: GECCO’05: Proceedings of the 2005 Conference on Genetic and Evolutionary Computation, ACM Press, New York, NY, USA, pp. 719–726 (2005). doi: 10.1145/1068009.1068129
  135. 135.
    Zhang, Q., Zhou, A., Jin, Y.: RM-MEDA: a regularity model-based multiobjective estimation of distribution algorithm. IEEE Trans. Evolut. Comput. 12(1), 41–63 (2008). doi: 10.1109/TEVC.2007.894202 CrossRefGoogle Scholar
  136. 136.
    Zhou, A., Zhang, Q., Jin, Y., Tsang, E., Okabe, T.: A model-based evolutionary algorithm for bi-objective optimization. In: 2005 IEEE Congress on Evolutionary Computation (CEC’2005), IEEE Service Center, Edinburgh, Scotland, vol. 3, pp. 2568–2575 (2005)Google Scholar
  137. 137.
    Zitzler, E., Künzli, S.: Indicator-based selection in multiobjective search. In: Yao, X. (ed.) Parallel Problem Solving from Nature—PPSN VIII, Springer-Verlag, Berlin/Heidelberg, Lecture Notes in Computer Science, vol. 3242, pp. 832–842 (2004)Google Scholar
  138. 138.
    Zitzler, E., Thiele, L.: An Evolutionary Algorithm for Multiobjective Optimization: The Strength Pareto Approach. Tech. Rep. 43, Computer Engineering and Communication Networks Lab (TIK), Swiss Federal Institute of Technology (ETH), Zurich, Switzerland (1998)Google Scholar
  139. 139.
    Zitzler, E., Deb, K., Thiele, L.: Comparison of multiobjective evolutionary algorithms on test functions of different difficulty. In: Wu, A.S. (ed.) Proceedings of the 1999 Genetic and Evolutionary Computation Conference. Workshop Program, Orlando, Florida, pp. 121–122 (1999)Google Scholar
  140. 140.
    Zitzler, E., Laumanns, M., Thiele, L.: SPEA2: improving the strength Pareto evolutionary algorithm. In: Giannakoglou, K., Tsahalis, D., Periaux, J., Papailou, P., Fogarty, T. (eds.) EUROGEN 2001. Evolutionary Methods for Design, Optimization and Control with Applications to Industrial Problems, Athens, Greece, pp. 95–100(2002)Google Scholar
  141. 141.
    Zitzler, E., Laumanns, M., Thiele, L., Fonseca, C.M., Grunert da Fonseca, V.: Why quality assessment of multiobjective optimizers is difficult. In: Langdon, W.B., Cantú-Paz, E., Mathias, K., Roy, R., Davis, D., Poli, R., Balakrishnan, K., Honavar, V., Rudolph, G., Wegener, J., Bull, L., Potter, M., Schultz, A., Miller, J., Burke, E., Jonoska, N. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’2002), Morgan Kaufmann Publishers, San Francisco, California, pp. 666–673 (2002)Google Scholar
  142. 142.
    Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C.M., Grunert da Fonseca, V.: Performance assessment of multiobjective optimizers: an analysis and review. IEEE Trans. Evolut. Comput. 7(2), 117–132 (2003)CrossRefGoogle Scholar
  143. 143.
    Zitzler, E., Brockhoff, D., Thiele, L.: The hypervolume indicator revisited: on the design of pareto-compliant indicators via weighted integration. In: Obayashi, S., et al. (eds.) Conference on Evolutionary Multi-Criterion Optimization (EMO 2007), Springer, Berlin, LNCS, vol. 4403, pp. 862–876 (2007)Google Scholar
  144. 144.
    Zitzler, E., Knowles, J., Thiele, L.: Quality assessment of pareto set approximations. In: Branke, J., Miettinen, K., Deb, K., Słowiǹski, R. (eds.) Multiobjective Optimization, Lecture Notes in Computer Science, vol. 5252, Springer–Verlag, Berlin/Heidelberg, pp. 373–404 (2008)Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Luis Martí
    • 1
  • Jesús García
    • 2
  • Antonio Berlanga
    • 2
  • José M. Molina
    • 2
  1. 1.Institute of ComputingUniversidade Federal FlumenseNiteróiBrazil
  2. 2.Group of Applied Artificial Intelligence, Department of InformaticsUniversidad Carlos III de MadridMadridSpain

Personalised recommendations