Simulation-only experiments to mimic the effects of the reality gap in the automatic design of robot swarms

  • Antoine LigotEmail author
  • Mauro BirattariEmail author


The reality gap—the discrepancy between reality and simulation—is a critical issue in the off-line automatic design of control software for robot swarms, as well as for single robots. It is understood that the reality gap manifests itself as a drop in performance: when control software generated in simulation is ported to physical robots, the performance observed is often disappointing compared with the one obtained in simulation. In this paper, we investigate whether, to observe the effects of the reality gap, it is necessary to assume that the control software is designed in a context that is simpler than the one in which it is evaluated. In the first experiment, we show that a performance drop may be observed also in an artificial, simulation-only reality gap: control software is generated on the basis of a simulation model and assessed on a second one. We will call this second model a pseudo-reality. We selected the simulation model to be used as a pseudo-reality by trial and error, so as to qualitatively replicate previously published observations made in experiments with physical robots. The results show that a performance drop occurs even if we can exclude that pseudo-reality is more complex than the simulation model used for the design. In the second experiment, we eliminate the trial-and-error selection of the first experiment by evaluating control software across multiple pseudo-realities, which are sampled around the original simulation model used for the design. The results of the second experiment confirm those of the first one and show that they do not depend on the specific pseudo-reality we previously selected by trial and error. Moreover, they suggest that one could use multiple pseudo-realities to evaluate automatic design methods and, from this simulation-only evaluation, infer their robustness to the reality gap.


Swarm robotics Automatic design Reality gap 



The project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (Grant Agreement No 681872). Mauro Birattari acknowledges support from the Belgian Fonds de la Recherche Scientifique—FNRS.


  1. Andrychowicz, M., Baker, B., Chociej, M., Jozefowicz, R., McGrew, B., Pachocki, J., Petron, A., Plappert, M., Powell, G., Ray, A., Schneider, J., Sidor, S., Tobin, J., Welinder, P., Weng, L., & Zaremba, W. (2018). Learning dexterous in-hand manipulation. eprint arXiv:1808.00177.
  2. Beni, G. (2004). From swarm intelligence to swarm robotics. In E. Şahin & W. M. Spears (Eds.), Swarm robotics, SAB (Vol. 3342, pp. 1–9). Berlin Heidelberg: Springer.Google Scholar
  3. Berman, S., Kumar, V., & Nagpal, R. (2011). Design of control policies for spatially inhomogeneous robot swarms with application to commercial pollination. In L. Zexiang (Ed.), IEEE international conference robotics and automation, ICRA (pp. 378–385). Piscataway: IEEE Press.Google Scholar
  4. Birattari, M. (2009). Tuning metaheuristics: A machine learning perspective. Berlin Heidelberg: Springer.zbMATHGoogle Scholar
  5. Birattari, M., Delhaisse, B., Francesca, G., & Kerdoncuff, Y. (2016). Observing the effects of overdesign in the automatic design of control software for robot swarms. In M. Dorigo, et al. (Eds.), Swarm intelligence, 10th international conference, ANTS (Vol. 9882, pp. 45–57). Cham: Springer, LNCS.Google Scholar
  6. Birattari, M., Ligot, A., Bozhinoski, D., Brambilla, M., Francesca, G., Garattoni, L., et al. (2019). Automatic off-line design of robot swarms: A manifesto. Frontiers in Robotics and AI, 6(59), 1–6.Google Scholar
  7. Birattari, M., Stützle, T., Paquete, L., & Varrentrapp, K. (2002). A racing algorithm for configuring metaheuristics. In W. Langdon, et al. (Eds.), Proceedings of the genetic and evolutionary computation conference, GECCO (pp. 11–18). San Francisco, CA: Morgan Kaufmann.Google Scholar
  8. Boeing, A., & Braunl, T. (2012). Leveraging multiple simulators for crossing the reality gap. In International Conference on control automation: Robotics and vision, ICARCV (pp. 1113–1119). Piscataway, NJ: IEEE Press.Google Scholar
  9. Bongard, J., & Lipson, H. (2004). Once more unto the breach: co-evolving a robot and its simulator. In J. Pollack, et al. (Eds.), Artificial life IX: Proceedings of the conference on the simulation and synthesis of living systems (pp. 57–62).Google Scholar
  10. Brambilla, M., Brutschy, A., Dorigo, M., & Birattari, M. (2015). Property-driven design for swarm robotics: A design method based on prescriptive modeling and model checking. ACM Transactions on Autonomous and Adaptive Systems, 9(4), 17.1–28.Google Scholar
  11. Brambilla, M., Ferrante, E., Birattari, M., & Dorigo, M. (2013). Swarm robotics: A review from the swarm engineering perspective. Swarm Intelligence, 7(1), 1–41.Google Scholar
  12. Bredeche, N., Montanier, J. M., Liu, W., & Winfield, A. F. (2012). Environment-driven distributed evolutionary adaptation in a population of autonomous robotic agents. Mathematical and Computer Modelling of Dynamical Systems, 18(1), 101–129.zbMATHGoogle Scholar
  13. Brooks, R. (1992). Artificial life and real robots. In F. J. Varela & P. Bourgine (Eds.), Towards a practice of autonomous systems. Proceedings of the first european conference on artificial life (pp. 3–10). Cambridge, MA: MIT Press.Google Scholar
  14. Caruana, R., Lawrence, S., & Lee Giles, C. (2001). Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping. In T. Leen, T. Dietterich, & V. Tresp (Eds.), Advances in neural information processing systems 13, NIPS 2000 (pp. 402–408). MIT Press.Google Scholar
  15. Dorigo, M., & Birattari, M. (2007). Swarm intelligence. Scholarpedia, 2(9), 1462.Google Scholar
  16. Floreano, D., Husbands, P., & Nolfi, S. (2008). Evolutionary robotics. In Springer Handbook of robotics (pp. 1423–1451). Springer, Berlin, Germany.Google Scholar
  17. Floreano, D., & Mondada, F. (1996). Evolution of plastic neurocontrollers for situated agents. In: P. Maes, et al. (Eds.), From animals to animats 4: Proceedings of the international conference on simulation of adaptive behavior. Zurich: ETH Zurich.Google Scholar
  18. Floreano, D., & Urzelai, J. (2001). Evolution of plastic control networks. Autonomous Robots, 11(3), 311–317.zbMATHGoogle Scholar
  19. Francesca, G., & Birattari, M. (2016). Automatic design of robot swarms: Achievements and challenges. Frontiers in Robotics and AI, 3(29), 1–9.Google Scholar
  20. Francesca, G., Brambilla, M., Brutschy, A., Garattoni, L., Miletitch, R., Podevijn, G., et al. (2015). AutoMoDe-Chocolate: Automatic design of control software for robot swarms. Swarm Intelligence, 9(2/3), 125–152.Google Scholar
  21. Francesca, G., Brambilla, M., Brutschy, A., Trianni, V., & Birattari, M. (2014). AutoMoDe: A novel approach to the automatic design of control software for robot swarms. Swarm Intelligence, 8(2), 89–112.Google Scholar
  22. Garattoni, L., Francesca, G., Brutschy, A., Pinciroli, C., & Birattari, M. (2015). Software infrastructure for e-puck (and TAM). Tech. Rep. TR/IRIDIA/2015-004, IRIDIA, Université libre de Bruxelles, Belgium.Google Scholar
  23. Geman, S., Bienenstock, E., & Doursat, R. (1992). Neural networks and the bias/variance dilemma. Neural Computation, 4(1), 1–58.Google Scholar
  24. Gutiérrez, Á., Campo, A., Dorigo, M., Donate, J., Monasterio-Huelin, F., & Magdalena, L. (2009). Open e-puck range and bearing miniaturized board for local communication in swarm robotics. In K. Kosuge (Ed.), IEEE international conference on robotics and automation, ICRA (pp. 3111–3116). Piscataway, NJ: IEEE Press.Google Scholar
  25. Haasdijk, E., Bredeche, N., & Eiben, A. (2014). Combining environment-driven adaptation and task-driven optimisation in evolutionary robotics. PLoS ONE, 9(6), e98466.Google Scholar
  26. Hamann, H. (2018). Swarm robotics: A formal approach. Berlin: Springer.Google Scholar
  27. Hamann, H., & Wörn, H. (2008). A framework of space–time continuous models for algorithm design in swarm robotics. Swarm Intelligence, 2(2–4), 209–239.Google Scholar
  28. Hasselmann, K., Ligot, A., Francesca, G., & Birattari, M. (2018a). Reference models for AutoMoDe. Tech. Rep. TR/IRIDIA/2018-002, IRIDIA, Université libre de Bruxelles, Belgium.Google Scholar
  29. Hasselmann, K., Robert, F., & Birattari, M. (2018b). Automatic design of communication-based behaviors for robot swarms. In M. Dorigo, et al. (Eds.), Swarm intelligence, ANTS, LNCS (Vol. 11172, pp. 16–29). Springer: Cham.Google Scholar
  30. Jakobi, N. (1997). Evolutionary robotics and the radical envelope-of-noise hypothesis. Adaptive Behavior, 6(2), 325–368.Google Scholar
  31. Jakobi, N. (1998). Minimal simulations for evolutionary robotics. PhD thesis, University of Sussex, Falmer, UKGoogle Scholar
  32. Jakobi, N., Husbands, P., Harvey, I. (1995). Noise and the reality gap: the use of simulation in evolutionary robotics. In F. Morán, et al. (Eds.), Advances in artificial life (Vol. 929, pp. 704–720). London: Springer, LNCS.Google Scholar
  33. König, L., & Mostaghim, S. (2009). Decentralized evolution of robotic behavior using finite state machines. International Journal of Intelligent Computing and Cybernetics, 2(4), 695–723.MathSciNetzbMATHGoogle Scholar
  34. Koos, S., Mouret, J. B., & Doncieux, S. (2013). The transferability approach: Crossing the reality gap in evolutionary robotics. IEEE Transactions on Evolutionary Computation, 17(1), 122–145.Google Scholar
  35. Kuckling, J., Ligot, A., Bozhinoski, D., & Birattari, M. (2018). Behavior trees as a control architecture in the automatic modular design of robot swarms. In M. Dorigo, et al. (Eds.), Swarm intelligence, ANTS, LNCS (Vol. 11172, pp. 30–43). Springer: Cham.Google Scholar
  36. Lee, J. B., & Arkin, R. C. (2003). Adaptive multi-robot behavior via learning momentum. In C. S. George Lee (Ed.), IEEE/RSJ international conference on intelligent robots and systems, IROS (pp. 2029–2036). Piscataway, NJ: IEEE Press.Google Scholar
  37. Ligot, A., & Birattari, M. (2019). Simulation-only experiments to mimic the effects of the reality gap in the automatic design of robot swarms. Supplementary material
  38. López-Ibáñez, M., Dubois-Lacoste, J., Pérez Cáceres, L., Birattari, M., & Stützle, T. (2016). The irace package: Iterated racing for automatic algorithm configuration. Operations Research Perspectives, 3, 43–58.MathSciNetGoogle Scholar
  39. Miglino, O., Lund, H., & Nolfi, S. (1995). Evolving mobile robots in simulated and real environments. Artificial Life, 2(4), 417–434.Google Scholar
  40. Mondada, F., Bonani, M., Raemy, X., Pugh, J., Cianci, C., Klaptocz, A., Magnenat, S., Zufferey, JC., Floreano, D., & Martinoli, A. (2009). The e-puck, a robot designed for education in engineering. In P. Gonçalves, P. Torres & C. Alves (Eds.), Proceedings of the 9th conference on autonomous robot systems and competitions (pp. 59–65). Instituto Politécnico de Castelo Branco, Portugal.Google Scholar
  41. Mondada, F., Franzi, E., & Ienne, P. (1994). Mobile robot miniaturisation: A tool for investigation in control algorithms. In T. Yoshikawa & F. Miyazaki (Eds.), Experimental robotics III (pp. 501–513). Berlin, Heidelberg: Springer.Google Scholar
  42. Morgan, N., & Bourlard, H. (1990). Generalization and parameter estimation in feedforward nets: Some experiments. In D. S. Touretzky (Ed.), Advances in neural information processing systems 2, NIPS 1990 (pp. 630–637). San Francisco: Morgan Kaufmann.Google Scholar
  43. Nolfi, S., Floreano, D., Miglino, G., & Mondada, F. (1994). How to evolve autonomous robots: Different approaches in evolutionary robotics. In R. A. Brooks & P. Maes (Eds.), Artificial Life IV: Proceedings of the workshop on the synthesis and simulation of living systems (pp. 190–197). Cambridge, MA: MIT Press.Google Scholar
  44. Peng, X. B., Andrychowicz, M., Zaremba, W., & Abbeel, P. (2018). Sim-to-real transfer of robotic control with dynamics randomization. In 2018 IEEE international conference on robotics and automation (ICRA) (pp. 1–8).Google Scholar
  45. Pinciroli, C., Trianni, V., O’Grady, R., Pini, G., Brutschy, A., Brambilla, M., et al. (2012). ARGoS: A modular, parallel, multi-engine simulator for multi-robot systems. Swarm Intelligence, 6(4), 271–295.Google Scholar
  46. Reina, A., Valentini, G., Fernández-Oto, C., Dorigo, M., & Trianni, V. (2015). A design pattern for decentralised decision making. PLoS ONE, 10(10), e0140950.Google Scholar
  47. Şahin, E. (2004). Swarm robotics: From sources of inspiration to domains of application. In E. Şahin & W. M. Spears (Eds.), Swarm robotics, SAB (Vol. 3342, pp. 10–20). Berlin Heidelberg: Springer, LNCSGoogle Scholar
  48. Silva, F., Duarte, M., Correia, L., Oliveira, S., & Christensen, A. (2016). Open issues in evolutionary robotics. Evolutionary Computation, 24(2), 205–236.Google Scholar
  49. Silva, F., Urbano, P., Correia, L., & Christensen, A. L. (2015). odNEAT: An algorithm for decentralised online evolution of robotic controllers. Evolutionary Computation, 23(3), 421–449.Google Scholar
  50. Urzelai, J., & Floreano, D. (2000). Evolutionary robotics: Coping with environmental change. In: L. D. Whitney, et al (Eds.), Proceedings of conference on the genetic and evolutionary computation conference, GECCO (pp. 941–948). San Francisco, CA: Morgan Kaufmann.Google Scholar
  51. Watson, R., Ficici, S., & Pollack, J. (2002). Embodied evolution: Distributing an evolutionary algorithm in a population of robots. Robotics and Autonomous Systems, 39(1), 1–18.Google Scholar
  52. Zagal, J. C., & Ruiz-Del-Solar, J. (2007). Combining simulation and reality in evolutionary robotics. Journal of Intelligent and Robotic Systems, 50(1), 19–39.Google Scholar
  53. Zagal, J. C., Ruiz-Del-Solar, J., & Vallejos, P. (2004). Back to reality: Crossing the reality gap in evolutionary robotics. IFAC/EURON Symposium on Intelligent Autonomous Vehicles, IAV, 37, 834–839.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.IRIDIAUniversité libre de BruxellesBrusselsBelgium

Personalised recommendations