Learning Automata and Niching-Induced Noisy Optimization for Multi-robot Path-Planning

  • Pratyusha RakshitEmail author
  • Amit Konar
Part of the Cognitive Intelligence and Robotics book series (CIR)


The chapter proposes an extension of the traditional differential evolution (DE) algorithm to handle the effect of noise in the objective surface. The proposed extension offers four merits with respect to its traditional counterpart. The first strategy is concerned with determining the effective fitness estimate of a trial solution on the basis of the distribution of its noisy fitness samples, instead of their conventional averaging. Second, the DE/current-to-best/1 mutation scheme is amended to generate promising donor vectors in the presence of noise in the objective surface. The third strategy, referred to as stochastic learning automata-induced adaptive sampling, is employed to judiciously select the sample size for the periodic fitness evaluation of a trial solution based on the fitness estimate and the fitness variance in its local neighborhood. Finally, a modified probabilistic crowding-induced niching behavior is introduced to promote quality solutions to the next generation, ensuring both the population quality and diversity. Computer simulations performed on a noisy version of a renowned set of 28 benchmark functions disclose that the proposed algorithm surpasses its contestants with respect to function error value, the number of successful runs, and successful performance metrics. The performance of the proposed algorithm has been studied on a real-world multi-robot path-planning problem. The application justifies the importance of the proposed noise handling strategies in practical systems.


Noisy single-objective optimization problem Differential evolution Learning automata Crowding-based niching Multi-robot path-planning 


  1. 1.
    D.V. Arnold, H.G. Beyer, A general noise model and its effects on evolution strategy performance. IEEE Trans. Evol. Comput. 10(4), 380–391 (2006)CrossRefGoogle Scholar
  2. 2.
    H. Kita, Y. Sano, Genetic algorithms for optimization of uncertain functions and their applications, in IEEE SICE 2003 Annual Conference, vol. 3 (2003), pp. 2744–2749Google Scholar
  3. 3.
    A.N. Aizawa, B.W. Wah, Scheduling of genetic algorithms in a noisy environment. Evol. Comput. 2(2), 97–122 (1994)CrossRefGoogle Scholar
  4. 4.
    A.N. Aizawa, B.W. Wah, Dynamic control of genetic algorithms in a noisy environment, in Genetic Algorithms, pp. 48–55Google Scholar
  5. 5.
    B.L. Miller, D.E. Goldberg, Genetic algorithms, tournament selection, and the effects of noise. Complex Syst. 9(3), 193–212 (1995)MathSciNetGoogle Scholar
  6. 6.
    Y. Jin, J. Branke, Evolutionary optimization in uncertain environments—a survey. IEEE Trans. Evol. Comput. 9(3), 303–317 (2005)CrossRefGoogle Scholar
  7. 7.
    P. Stagge, Averaging efficiently in the presence of noise, in Parallel Problem Solving from Nature—PPSN V (Springer, Berlin, 1998), pp. 188–197Google Scholar
  8. 8.
    B.L. Miller, Noise, sampling, and efficient genetic algorithms, IlliGAL Report no. 97001 (1997)Google Scholar
  9. 9.
    J. Branke, C. Schmidt, Sequential sampling in noisy environments, in Parallel Problem Solving from Nature—PPSN VIII (Springer, Berlin, 2004), pp. 202–211Google Scholar
  10. 10.
    E.C. Paz, Adaptive sampling for noisy problems, in Genetic and Evolutionary Computation–GECCO (Springer, Berlin, 2004), pp. 947–958Google Scholar
  11. 11.
    A.D. Pietro, L. While, L. Barone, Applying evolutionary algorithms to problems with noisy, time-consuming fitness functions, in IEEE Congress on Evolutionary Computation——CEC, vol. 2 (2004), pp. 1254–1261Google Scholar
  12. 12.
    J.M. Fitzpatrick, J.J. Grefenstette, Genetic algorithms in noisy environments. Mach. Learn. 3(2–3), 101–120 (1988)Google Scholar
  13. 13.
    M. Rattray, J. Shapiro, Noisy fitness evaluation in genetic algorithms and the dynamics of learning, Foundations of Genetic Algorithms: 4th Workshop: Revised Papers, vol. 4, ed. by R. Belew, M. Vose (Morgan Kaufmann, San Francisco, 1998), pp. 117–139Google Scholar
  14. 14.
    B.L. Miller, D.E. Goldberg, Genetic algorithms, selection schemes, and the varying effects of noise. Evol. Comput. 4(2), 113–131 (1996)CrossRefGoogle Scholar
  15. 15.
    D.E. Goldberg, K. Deb, J.H. Clark, Genetic algorithms, noise, and the sizing of populations. Complex systems 6, 333–362 (1991)zbMATHGoogle Scholar
  16. 16.
    J. Branke, C. Schmidt, Selection in the presence of noise, in Genetic and Evolutionary Computation—GECCO (Springer, Berlin, 2003), pp. 766–777CrossRefGoogle Scholar
  17. 17.
    S. Markon, D.V. Arnold, T. Back, T. Beielstein, H.G. Beyer, Thresholding-a selection operator for noisy ES, in IEEE Congress on Evolutionary Computation—CEC, vol. 1 (2001), pp. 465–472Google Scholar
  18. 18.
    G. Rudolph, A partial order approach to noisy fitness functions, in IEEE Congress on Evolutionary Computation——CEC, vol. 1 (2001), pp. 318–325Google Scholar
  19. 19.
    T. Beielstein, S. Markon, Threshold selection, hypothesis tests, and DOE methods, in IEEE Congress on Evolutionary Computation—CEC (2002), pp. 777–782Google Scholar
  20. 20.
    N. Hansen, A.S.P. Niederberger, L. Guzzella, P. Koumoutsakos, A method for handling uncertainty in evolutionary optimization with an application to feedback control of combustion. IEEE Trans. Evol. Comput. 13(1), 180–197 (2009)CrossRefGoogle Scholar
  21. 21.
    R. Coulom, P. Rolet, N. Sokolovska, O. Teytaud, Handling expensive optimization with large noise, in ACM Foundations of Genetic Algorithms (2011), pp. 61–68Google Scholar
  22. 22.
    J.W. Kruisselbrink, Evolution Strategies for Robust Optimization (Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Faculty of Science, Leiden, 2012)Google Scholar
  23. 23.
    H. Jang, R. Choe, K.R. Ryu, Deriving a robust policy for container stacking using a noise-tolerant genetic algorithm, in ACM Research in Applied Computation Symposium (2012), pp. 31–36Google Scholar
  24. 24.
    E. Mininno, F. Neri, A memetic differential evolution approach in noisy optimization. Memet. Comput. 2(2), 111–135 (2010)CrossRefGoogle Scholar
  25. 25.
    B. Liu, X. Zhang, H. Ma, Hybrid differential evolution for noisy optimization, in Evolutionary In IEEE Congress on Evolutionary Computation—CEC (2008), pp. 587–592Google Scholar
  26. 26.
    H. Pan, L. Wang, B. Liu, Particle swarm optimization for function optimization in noisy environment. Appl. Math. Comput. 181(2), 908–919 (2006)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Z. Zhang, T. Xin, Immune algorithm with adaptive sampling in noisy environments and its application to stochastic optimization problems. IEEE Comput. Intell. Mag. 2(4), 29–40 (2007)MathSciNetCrossRefGoogle Scholar
  28. 28.
    S. Rahnamayan, H.R. Tizhoosh, M. Salama, Opposition-based differential evolution for optimization of noisy problems, in IEEE Congress on Evolutionary Computation—CEC (IEEE, 2006), pp. 1865–1872Google Scholar
  29. 29.
    S. Das, A. Konar, U.K. Chakraborty, Improved differential evolution algorithms for handling noisy optimization problems, in IEEE Congress on Evolutionary Computation—CEC, vol. 2 (2005), pp. 1691–1698Google Scholar
  30. 30.
    E. Mendel, R.A. Krohling, M. Campos, Swarm algorithms with chaotic jumps applied to noisy optimization problems. Inf. Sci. 181(20), 4494–4514 (2011)MathSciNetCrossRefGoogle Scholar
  31. 31.
    J.R. Vilela, Population statistics for particle swarm optimization on problems subject to noise, Ph.D. thesis, 2014Google Scholar
  32. 32.
    P.D. Stroud, Kalman-extended genetic algorithm for search in nonstationary environments with noisy fitness evaluations. IEEE Trans. Evol. Comput. 5(1), 66–77 (2001)CrossRefGoogle Scholar
  33. 33.
    J.E. Fieldsend, R.M. Everson, The rolling tide evolutionary algorithm: a multiobjective optimizer for noisy optimization problems. IEEE Trans. Evol. Comput. 19(1), 103–117 (2015)CrossRefGoogle Scholar
  34. 34.
    E.J. Hughes, Evolutionary multi-objective ranking with uncertainty and noise, in Evolutionary Multi-Criterion Optimization (Springer, Berlin, 2001), pp. 329–343Google Scholar
  35. 35.
    D. Büche, P. Stoll, R. Dornberger, P. Koumoutsakos, Multiobjective evolutionary algorithm for the optimization of noisy combustion processes. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 32(4), 460–473 (2002)CrossRefGoogle Scholar
  36. 36.
    C.K. Goh, K.C. Tan, An investigation on noisy environments in evolutionary multiobjective optimization. IEEE Trans. Evol. Comput. 11(3), 354–381 (2007)CrossRefGoogle Scholar
  37. 37.
    P. Boonma, J. Suzuki, A confidence-based dominance operator in evolutionary algorithms for noisy multiobjective optimization problems, in IEEE Tools with Artificial Intelligence (2009), pp. 387–394Google Scholar
  38. 38.
    A. Syberfeldt, A. Ng, R.I. John, P. Moore, Evolutionary optimisation of noisy multi-objective problems using confidence-based dynamic resampling. Eur. J. Oper. Res. 204(3), 533–544 (2010)MathSciNetCrossRefGoogle Scholar
  39. 39.
    T. Park, K.R. Ryu, Accumulative sampling for noisy evolutionary multi-objective optimization, in ACM Genetic and Evolutionary Computation–GECCO (2011), pp. 793–800Google Scholar
  40. 40.
    F. Siegmund, A. H. C. Ng, K. Deb, A comparative study of dynamic resampling strategies for guided evolutionary multi-objective optimization, in IEEE Congress on Evolutionary Computation—CEC (2013), pp. 1826–1835Google Scholar
  41. 41.
    P. Rakshit, A. Konar, S. Das, L.C. Jain, A.K. Nagar, Uncertainty management in differential evolution induced multiobjective optimization in presence of measurement noise. IEEE Trans. Syst. Man Cybern. Syst. 44(7), 922–937 (2014)CrossRefGoogle Scholar
  42. 42.
    P. Rakshit, A. Konar, Extending multi-objective differential evolution for optimization in presence of noise. Inf. Sci. 305, 56–76 (2015)CrossRefGoogle Scholar
  43. 43.
    P. Rakshit, A. Konar, A.K. Nagar, Artificial Bee Colony induced multi-objective optimization in presence of noise, in IEEE Congress on Evolutionary Computation—CEC (2014), pp. 3176–3183Google Scholar
  44. 44.
    P. Rakshit, A. Konar, Non-dominated sorting bee colony optimization in the presence of noise. Soft Comput., pp. 1–21 (2015)Google Scholar
  45. 45.
    P. Rakshit, A. Konar, P. Bhowmik, I. Goswami, S. Das, L.C. Jain, A.K. Nagar, Realization of an adaptive memetic algorithm using differential evolution and Q-learning: a case study in multirobot path-planning. IEEE Trans. Syst. Man Cybern. Syst. 43(4), 814–831 (2013)CrossRefGoogle Scholar
  46. 46.
    R. Storn, K. Price, Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11(4), 341–359 (1997)MathSciNetCrossRefGoogle Scholar
  47. 47.
    K. Price, R.M. Storn, J.A. Lampinen, Differential Evolution: A Practical Approach to Global Optimization (Springer Science & Business Media, 2006)Google Scholar
  48. 48.
    U.K. Chakraborty, Advances in Differential Evolution (Springer, Heidelberg, 2008)CrossRefGoogle Scholar
  49. 49.
    S. Das, P.N. Suganthan, Differential evolution: a survey of the state-of-the-art. IEEE Trans. Evol. Comput. 15(1), 4–31 (2011)CrossRefGoogle Scholar
  50. 50.
    Y. Sano, H. Kita, I. Kamihira, M. Yamaguchi, Online optimization of an engine controller by means of a genetic algorithm using history of search, in IEEE Industrial Electronics Society—IECON, vol. 4 (2000), pp. 2929–2934Google Scholar
  51. 51.
    Y. Sano, H. Kita, Optimization of noisy fitness functions by means of genetic algorithms using history of search with test of estimation, in IEEE Congress on Evolutionary Computation—CEC, vol. 1 (2002), pp. 360–365Google Scholar
  52. 52.
    S. Lakshmivarahan, M.A.L. Thathachar, Absolutely expedient learning algorithms for stochastic automata. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 3, 281–286 (1973)MathSciNetzbMATHGoogle Scholar
  53. 53.
    K.S. Narendra, M.L.A.A. Thathachar, Learning automata—a survey. IEEE Trans. Syst. Man Cybern. 4, 323–334 (1974)MathSciNetCrossRefGoogle Scholar
  54. 54.
    X. Li, Niching without niching parameters: particle swarm optimization using a ring topology. IEEE Trans. Evol. Comput. 14(1), 150–169 (2010)CrossRefGoogle Scholar
  55. 55.
    S.W. Mahfoud, Crowding and preselection revisited, Urbana, IL 61801, 1992Google Scholar
  56. 56.
    O.J. Mengshoel, D.E. Goldberg, Probabilistic crowding: deterministic crowding with probabilistic replacement, in Genetic and Evolutionary Computation–GECCO (1999), pp. 409–416Google Scholar
  57. 57.
    J.J. Liang, B.Y. Qu, P.N. Suganthan, A.G.H. Díaz, Problem definitions and evaluation criteria for the CEC 2013 special session on real-parameter optimization, Computational Intelligence Laboratory, Zhengzhou University, Zhengzhou, China and Nanyang Technological University, Singapore, Technical Report 201212, 2013Google Scholar
  58. 58.
    J. Chakraborty, A. Konar, A distributed multi-robot path-planning using particle swarm optimization, in 2nd National Conference on Recent Trends in Information Systems, pp. 216–221 (2008)Google Scholar
  59. 59.
    J. Chakraborty, A. Konar, U.K. Chakraborty, L.C. Jain, Distributed co-operative multi robot path-planning using differential evolution, in IEEE Congress on Evolutionary Computation (2009), pp. 718–725Google Scholar
  60. 60.
    P. Bhattacharjee, P. Rakshit, I. Goswami, A. Konar, A.K. Nagar, Multi-robot path-planning using artificial bee colony optimization algorithm, in Proceedings of IEEE 2011 Third World Congress on Nature and Biologically Inspired Computing (NaBIC) (2011), pp. 219–224Google Scholar
  61. 61.
    D. Zaharie, Influence of crossover on the behavior of differential evolution algorithms. Appl. Soft Comput. 9(3), 1126–1138 (2009)CrossRefGoogle Scholar
  62. 62.
    P. Rakshit, A. Konar, A.K. Nagar, Learning automata induced artificial bee colony for noisy optimization, in Proceedings of IEEE Congress on Evolutionary Computation (2017), pp. 984–991Google Scholar
  63. 63.
    J. Derrac, S. García, D. Molina, F. Herrera, A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol. Comput. 1(1), 3–18 (2011)CrossRefGoogle Scholar
  64. 64.
    D. Sheskin, Handbook of Parametric and Nonparametric Statistical Procedures, 4th edn. (Chapman and Hall/CRC, 2007)Google Scholar
  65. 65.
    J. Zhang, A.C. Sanderson, JADE: adaptive differential evolution with optional external archive. IEEE Trans. Evol. Comput. 13(5), 945–958 (2009)CrossRefGoogle Scholar
  66. 66.
    J. Brest, S. Greiner, B. Bošković, M. Mernik, V. Zumer, Self-adapting control parameters in differential evolution: a comparative study on numerical benchmark problems. IEEE Trans. Evol. Comput. 10(6), 646–657 (2006)CrossRefGoogle Scholar
  67. 67.
    A.K. Qin, P.N. Suganthan, Self-adaptive differential evolution algorithm for numerical optimization, in IEEE Congress on Evolutionary Computation—CEC, vol. 2 (2005), pp. 1785–1791Google Scholar
  68. 68.
    J. Liu, J. Lampinen, A fuzzy adaptive differential evolution algorithm. Soft. Comput. 9(6), 448–462 (2005)CrossRefGoogle Scholar
  69. 69.
    G.E.P. Box, M.E. Muller, A note on the generation of random normal deviates. Ann. Math. Stat. 29, 610–611 (1958)CrossRefGoogle Scholar
  70. 70.
    D.E. Knuth, Seminumerical Algorithms, The Art of Computer Programming, vol. 2 (1981)Google Scholar
  71. 71.
    W. Hörmann, J. Leydold, G. Derflinger, General principles in random variate generation, in Automatic Nonuniform Random Variate Generation (Springer, Berlin, 2004), pp. 13–41CrossRefGoogle Scholar
  72. 72.
    G. Marsaglia, W.W. Tsang, The ziggurat method for generating random variables. J. Stat. Softw. 5(8), 1–7 (2000)CrossRefGoogle Scholar
  73. 73.
    J.H. Ahrens, U. Dieter, Generating gamma variates by a modified rejection technique. Commun. ACM 25(1), 47–54 (1982)MathSciNetCrossRefGoogle Scholar
  74. 74.
    J. Bolte, Linear congruential generators, Wolfram Demonstrations ProjectGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.Department of Electronics and Telecommunication EngineeringJadavpur UniversityKolkataIndia
  2. 2.Department of Electronics and Telecommunication EngineeringJadavpur UniversityKolkataIndia

Personalised recommendations