Principles in Noisy Optimization pp 171-242 | Cite as

# Learning Automata and Niching-Induced Noisy Optimization for Multi-robot Path-Planning

## Abstract

The chapter proposes an extension of the traditional differential evolution (DE) algorithm to handle the effect of noise in the objective surface. The proposed extension offers four merits with respect to its traditional counterpart. The first strategy is concerned with determining the effective fitness estimate of a trial solution on the basis of the distribution of its noisy fitness samples, instead of their conventional averaging. Second, the DE/current-to-best/1 mutation scheme is amended to generate promising donor vectors in the presence of noise in the objective surface. The third strategy, referred to as stochastic learning automata-induced adaptive sampling, is employed to judiciously select the sample size for the periodic fitness evaluation of a trial solution based on the fitness estimate and the fitness variance in its local neighborhood. Finally, a modified probabilistic crowding-induced niching behavior is introduced to promote quality solutions to the next generation, ensuring both the population quality and diversity. Computer simulations performed on a noisy version of a renowned set of 28 benchmark functions disclose that the proposed algorithm surpasses its contestants with respect to function error value, the number of successful runs, and successful performance metrics. The performance of the proposed algorithm has been studied on a real-world multi-robot path-planning problem. The application justifies the importance of the proposed noise handling strategies in practical systems.

## Keywords

Noisy single-objective optimization problem Differential evolution Learning automata Crowding-based niching Multi-robot path-planning## References

- 1.D.V. Arnold, H.G. Beyer, A general noise model and its effects on evolution strategy performance. IEEE Trans. Evol. Comput.
**10**(4), 380–391 (2006)CrossRefGoogle Scholar - 2.H. Kita, Y. Sano, Genetic algorithms for optimization of uncertain functions and their applications, in
*IEEE SICE 2003 Annual Conference*, vol. 3 (2003), pp. 2744–2749Google Scholar - 3.A.N. Aizawa, B.W. Wah, Scheduling of genetic algorithms in a noisy environment. Evol. Comput.
**2**(2), 97–122 (1994)CrossRefGoogle Scholar - 4.A.N. Aizawa, B.W. Wah, Dynamic control of genetic algorithms in a noisy environment, in
*Genetic Algorithms*, pp. 48–55Google Scholar - 5.B.L. Miller, D.E. Goldberg, Genetic algorithms, tournament selection, and the effects of noise. Complex Syst.
**9**(3), 193–212 (1995)MathSciNetGoogle Scholar - 6.Y. Jin, J. Branke, Evolutionary optimization in uncertain environments—a survey. IEEE Trans. Evol. Comput.
**9**(3), 303–317 (2005)CrossRefGoogle Scholar - 7.P. Stagge, Averaging efficiently in the presence of noise, in
*Parallel Problem Solving from Nature—PPSN V*(Springer, Berlin, 1998), pp. 188–197Google Scholar - 8.B.L. Miller,
*Noise, sampling, and efficient genetic algorithms*, IlliGAL Report no. 97001 (1997)Google Scholar - 9.J. Branke, C. Schmidt, Sequential sampling in noisy environments, in
*Parallel Problem Solving from Nature—PPSN VIII*(Springer, Berlin, 2004), pp. 202–211Google Scholar - 10.E.C. Paz, Adaptive sampling for noisy problems, in
*Genetic and Evolutionary Computation–GECCO*(Springer, Berlin, 2004), pp. 947–958Google Scholar - 11.A.D. Pietro, L. While, L. Barone, Applying evolutionary algorithms to problems with noisy, time-consuming fitness functions, in
*IEEE Congress on Evolutionary Computation——CEC*, vol. 2 (2004), pp. 1254–1261Google Scholar - 12.J.M. Fitzpatrick, J.J. Grefenstette, Genetic algorithms in noisy environments. Mach. Learn.
**3**(2–3), 101–120 (1988)Google Scholar - 13.M. Rattray, J. Shapiro, Noisy fitness evaluation in genetic algorithms and the dynamics of learning,
*Foundations of Genetic Algorithms: 4th Workshop: Revised Papers*, vol. 4, ed. by R. Belew, M. Vose (Morgan Kaufmann, San Francisco, 1998), pp. 117–139Google Scholar - 14.B.L. Miller, D.E. Goldberg, Genetic algorithms, selection schemes, and the varying effects of noise. Evol. Comput.
**4**(2), 113–131 (1996)CrossRefGoogle Scholar - 15.D.E. Goldberg, K. Deb, J.H. Clark, Genetic algorithms, noise, and the sizing of populations. Complex systems
**6**, 333–362 (1991)zbMATHGoogle Scholar - 16.J. Branke, C. Schmidt, Selection in the presence of noise, in
*Genetic and Evolutionary Computation—GECCO*(Springer, Berlin, 2003), pp. 766–777CrossRefGoogle Scholar - 17.S. Markon, D.V. Arnold, T. Back, T. Beielstein, H.G. Beyer, Thresholding-a selection operator for noisy ES, in
*IEEE Congress on Evolutionary Computation—CEC*, vol. 1 (2001), pp. 465–472Google Scholar - 18.G. Rudolph, A partial order approach to noisy fitness functions, in
*IEEE Congress on Evolutionary Computation——CEC*, vol. 1 (2001), pp. 318–325Google Scholar - 19.T. Beielstein, S. Markon, Threshold selection, hypothesis tests, and DOE methods, in
*IEEE Congress on Evolutionary Computation—CEC*(2002), pp. 777–782Google Scholar - 20.N. Hansen, A.S.P. Niederberger, L. Guzzella, P. Koumoutsakos, A method for handling uncertainty in evolutionary optimization with an application to feedback control of combustion. IEEE Trans. Evol. Comput.
**13**(1), 180–197 (2009)CrossRefGoogle Scholar - 21.R. Coulom, P. Rolet, N. Sokolovska, O. Teytaud, Handling expensive optimization with large noise, in
*ACM Foundations of Genetic Algorithms*(2011), pp. 61–68Google Scholar - 22.J.W. Kruisselbrink,
*Evolution Strategies for Robust Optimization*(Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Faculty of Science, Leiden, 2012)Google Scholar - 23.H. Jang, R. Choe, K.R. Ryu, Deriving a robust policy for container stacking using a noise-tolerant genetic algorithm, in
*ACM Research in Applied Computation Symposium*(2012), pp. 31–36Google Scholar - 24.E. Mininno, F. Neri, A memetic differential evolution approach in noisy optimization. Memet. Comput.
**2**(2), 111–135 (2010)CrossRefGoogle Scholar - 25.B. Liu, X. Zhang, H. Ma, Hybrid differential evolution for noisy optimization, in
*Evolutionary*In*IEEE Congress on Evolutionary Computation—CEC*(2008), pp. 587–592Google Scholar - 26.H. Pan, L. Wang, B. Liu, Particle swarm optimization for function optimization in noisy environment. Appl. Math. Comput.
**181**(2), 908–919 (2006)MathSciNetzbMATHGoogle Scholar - 27.Z. Zhang, T. Xin, Immune algorithm with adaptive sampling in noisy environments and its application to stochastic optimization problems. IEEE Comput. Intell. Mag.
**2**(4), 29–40 (2007)MathSciNetCrossRefGoogle Scholar - 28.S. Rahnamayan, H.R. Tizhoosh, M. Salama, Opposition-based differential evolution for optimization of noisy problems, in
*IEEE Congress on Evolutionary Computation—CEC*(IEEE, 2006), pp. 1865–1872Google Scholar - 29.S. Das, A. Konar, U.K. Chakraborty, Improved differential evolution algorithms for handling noisy optimization problems, in
*IEEE Congress on Evolutionary Computation—CEC*, vol. 2 (2005), pp. 1691–1698Google Scholar - 30.E. Mendel, R.A. Krohling, M. Campos, Swarm algorithms with chaotic jumps applied to noisy optimization problems. Inf. Sci.
**181**(20), 4494–4514 (2011)MathSciNetCrossRefGoogle Scholar - 31.J.R. Vilela,
*Population statistics for particle swarm optimization on problems subject to noise*, Ph.D. thesis, 2014Google Scholar - 32.P.D. Stroud, Kalman-extended genetic algorithm for search in nonstationary environments with noisy fitness evaluations. IEEE Trans. Evol. Comput.
**5**(1), 66–77 (2001)CrossRefGoogle Scholar - 33.J.E. Fieldsend, R.M. Everson, The rolling tide evolutionary algorithm: a multiobjective optimizer for noisy optimization problems. IEEE Trans. Evol. Comput.
**19**(1), 103–117 (2015)CrossRefGoogle Scholar - 34.E.J. Hughes, Evolutionary multi-objective ranking with uncertainty and noise, in
*Evolutionary Multi-Criterion Optimization*(Springer, Berlin, 2001), pp. 329–343Google Scholar - 35.D. Büche, P. Stoll, R. Dornberger, P. Koumoutsakos, Multiobjective evolutionary algorithm for the optimization of noisy combustion processes. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev.
**32**(4), 460–473 (2002)CrossRefGoogle Scholar - 36.C.K. Goh, K.C. Tan, An investigation on noisy environments in evolutionary multiobjective optimization. IEEE Trans. Evol. Comput.
**11**(3), 354–381 (2007)CrossRefGoogle Scholar - 37.P. Boonma, J. Suzuki, A confidence-based dominance operator in evolutionary algorithms for noisy multiobjective optimization problems, in
*IEEE Tools with Artificial Intelligence*(2009), pp. 387–394Google Scholar - 38.A. Syberfeldt, A. Ng, R.I. John, P. Moore, Evolutionary optimisation of noisy multi-objective problems using confidence-based dynamic resampling. Eur. J. Oper. Res.
**204**(3), 533–544 (2010)MathSciNetCrossRefGoogle Scholar - 39.T. Park, K.R. Ryu, Accumulative sampling for noisy evolutionary multi-objective optimization, in
*ACM Genetic and Evolutionary Computation–GECCO*(2011), pp. 793–800Google Scholar - 40.F. Siegmund, A. H. C. Ng, K. Deb, A comparative study of dynamic resampling strategies for guided evolutionary multi-objective optimization, in
*IEEE Congress on Evolutionary Computation—CEC*(2013), pp. 1826–1835Google Scholar - 41.P. Rakshit, A. Konar, S. Das, L.C. Jain, A.K. Nagar, Uncertainty management in differential evolution induced multiobjective optimization in presence of measurement noise. IEEE Trans. Syst. Man Cybern. Syst.
**44**(7), 922–937 (2014)CrossRefGoogle Scholar - 42.P. Rakshit, A. Konar, Extending multi-objective differential evolution for optimization in presence of noise. Inf. Sci.
**305**, 56–76 (2015)CrossRefGoogle Scholar - 43.P. Rakshit, A. Konar, A.K. Nagar, Artificial Bee Colony induced multi-objective optimization in presence of noise, in
*IEEE Congress on Evolutionary Computation—CEC*(2014), pp. 3176–3183Google Scholar - 44.P. Rakshit, A. Konar, Non-dominated sorting bee colony optimization in the presence of noise. Soft Comput., pp. 1–21 (2015)Google Scholar
- 45.P. Rakshit, A. Konar, P. Bhowmik, I. Goswami, S. Das, L.C. Jain, A.K. Nagar, Realization of an adaptive memetic algorithm using differential evolution and Q-learning: a case study in multirobot path-planning. IEEE Trans. Syst. Man Cybern. Syst.
**43**(4), 814–831 (2013)CrossRefGoogle Scholar - 46.R. Storn, K. Price, Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim.
**11**(4), 341–359 (1997)MathSciNetCrossRefGoogle Scholar - 47.K. Price, R.M. Storn, J.A. Lampinen,
*Differential Evolution: A Practical Approach to Global Optimization*(Springer Science & Business Media, 2006)Google Scholar - 48.U.K. Chakraborty,
*Advances in Differential Evolution*(Springer, Heidelberg, 2008)CrossRefGoogle Scholar - 49.S. Das, P.N. Suganthan, Differential evolution: a survey of the state-of-the-art. IEEE Trans. Evol. Comput.
**15**(1), 4–31 (2011)CrossRefGoogle Scholar - 50.Y. Sano, H. Kita, I. Kamihira, M. Yamaguchi, Online optimization of an engine controller by means of a genetic algorithm using history of search, in
*IEEE Industrial Electronics Society—IECON*, vol. 4 (2000), pp. 2929–2934Google Scholar - 51.Y. Sano, H. Kita, Optimization of noisy fitness functions by means of genetic algorithms using history of search with test of estimation, in
*IEEE Congress on Evolutionary Computation—CEC*, vol. 1 (2002), pp. 360–365Google Scholar - 52.S. Lakshmivarahan, M.A.L. Thathachar, Absolutely expedient learning algorithms for stochastic automata. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum.
**3**, 281–286 (1973)MathSciNetzbMATHGoogle Scholar - 53.K.S. Narendra, M.L.A.A. Thathachar, Learning automata—a survey. IEEE Trans. Syst. Man Cybern.
**4**, 323–334 (1974)MathSciNetCrossRefGoogle Scholar - 54.X. Li, Niching without niching parameters: particle swarm optimization using a ring topology. IEEE Trans. Evol. Comput.
**14**(1), 150–169 (2010)CrossRefGoogle Scholar - 55.S.W. Mahfoud,
*Crowding and preselection revisited*, Urbana, IL 61801, 1992Google Scholar - 56.O.J. Mengshoel, D.E. Goldberg, Probabilistic crowding: deterministic crowding with probabilistic replacement, in
*Genetic and Evolutionary Computation–GECCO*(1999), pp. 409–416Google Scholar - 57.J.J. Liang, B.Y. Qu, P.N. Suganthan, A.G.H. Díaz,
*Problem definitions and evaluation criteria for the CEC 2013 special session on real-parameter optimization*, Computational Intelligence Laboratory, Zhengzhou University, Zhengzhou, China and Nanyang Technological University, Singapore, Technical Report 201212, 2013Google Scholar - 58.J. Chakraborty, A. Konar, A distributed multi-robot path-planning using particle swarm optimization, in
*2nd National Conference on Recent Trends in Information Systems*, pp. 216–221 (2008)Google Scholar - 59.J. Chakraborty, A. Konar, U.K. Chakraborty, L.C. Jain, Distributed co-operative multi robot path-planning using differential evolution, in
*IEEE Congress on Evolutionary Computation*(2009), pp. 718–725Google Scholar - 60.P. Bhattacharjee, P. Rakshit, I. Goswami, A. Konar, A.K. Nagar, Multi-robot path-planning using artificial bee colony optimization algorithm, in
*Proceedings of IEEE 2011 Third World Congress on Nature and Biologically Inspired Computing*(NaBIC) (2011), pp. 219–224Google Scholar - 61.D. Zaharie, Influence of crossover on the behavior of differential evolution algorithms. Appl. Soft Comput.
**9**(3), 1126–1138 (2009)CrossRefGoogle Scholar - 62.P. Rakshit, A. Konar, A.K. Nagar, Learning automata induced artificial bee colony for noisy optimization, in
*Proceedings of IEEE Congress on Evolutionary Computation*(2017), pp. 984–991Google Scholar - 63.J. Derrac, S. García, D. Molina, F. Herrera, A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol. Comput.
**1**(1), 3–18 (2011)CrossRefGoogle Scholar - 64.D. Sheskin,
*Handbook of Parametric and Nonparametric Statistical Procedures*, 4th edn. (Chapman and Hall/CRC, 2007)Google Scholar - 65.J. Zhang, A.C. Sanderson, JADE: adaptive differential evolution with optional external archive. IEEE Trans. Evol. Comput.
**13**(5), 945–958 (2009)CrossRefGoogle Scholar - 66.J. Brest, S. Greiner, B. Bošković, M. Mernik, V. Zumer, Self-adapting control parameters in differential evolution: a comparative study on numerical benchmark problems. IEEE Trans. Evol. Comput.
**10**(6), 646–657 (2006)CrossRefGoogle Scholar - 67.A.K. Qin, P.N. Suganthan, Self-adaptive differential evolution algorithm for numerical optimization, in
*IEEE Congress on Evolutionary Computation—CEC*, vol. 2 (2005), pp. 1785–1791Google Scholar - 68.J. Liu, J. Lampinen, A fuzzy adaptive differential evolution algorithm. Soft. Comput.
**9**(6), 448–462 (2005)CrossRefGoogle Scholar - 69.G.E.P. Box, M.E. Muller, A note on the generation of random normal deviates. Ann. Math. Stat.
**29**, 610–611 (1958)CrossRefGoogle Scholar - 70.D.E. Knuth,
*Seminumerical Algorithms, The Art of Computer Programming*, vol. 2 (1981)Google Scholar - 71.W. Hörmann, J. Leydold, G. Derflinger, General principles in random variate generation, in
*Automatic Nonuniform Random Variate Generation*(Springer, Berlin, 2004), pp. 13–41CrossRefGoogle Scholar - 72.G. Marsaglia, W.W. Tsang, The ziggurat method for generating random variables. J. Stat. Softw.
**5**(8), 1–7 (2000)CrossRefGoogle Scholar - 73.J.H. Ahrens, U. Dieter, Generating gamma variates by a modified rejection technique. Commun. ACM
**25**(1), 47–54 (1982)MathSciNetCrossRefGoogle Scholar - 74.J. Bolte,
*Linear congruential generators*, Wolfram Demonstrations ProjectGoogle Scholar