Race from Pixels: Evolving Neural Network Controller for Vision-Based Car Driving

  • Borys Tymchenko
  • Svitlana Antoshchuk
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 836)


Modern robotics uses many advanced precise algorithms to control autonomous agents. Now arises tendency to apply machine learning in niches, where precise algorithms are hard to design or implement. With machine learning, for continuous control tasks, evolution strategies are used. We propose an enhancement to crossover operator, which diminishes probability of degraded offsprings compared to conventional crossover operators. Our experiments in TORCS environment show, that presented algorithm can evolve robust neural networks for non-trivial continuous control tasks such as driving a racing car in various tracks.


Neuroevolution Genetic algorithms Neural networks Deep learning Continuous control Crossover Autonomous vehicles TORCS 


  1. 1.
    Kingma, D.P., Ba, J.: Adam: A Method for Stochastic Optimization. CoRR 1412.6980 (2014).
  2. 2.
    Bojarski, M., Testa, D.D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L.D., Monfort, M., Muller, U., Zhang, J., Zhang, X., Zhao, J., Zieba, K.: End to End Learning for Self-Driving Cars. CoRR 1604.07316 (2016).
  3. 3.
    Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.A.: Playing atari with deep reinforcement learning. CoRR 1312.5602 (2013).
  4. 4.
    Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., Abbeel, P.: Overcoming exploration in reinforcement learning with demonstrations. CoRR 1709.10089 (2017).
  5. 5.
    Bruce, J., Sünderhauf, N., Mirowski, P., Hadsell, R., Milford, M.: One-shot reinforcement learning for robot navigation with interactive replay. CoRR 1711.10137 (2017).
  6. 6.
    Salimans, T., Ho, J., Chen, X., Sutskever, I.: Evolution strategies as a scalable alternative to reinforcement learning. CoRR 1703.03864 (2017).
  7. 7.
    Lehman, J., Chen, J., Clune, J., Stanley, K.O.: Safe mutations for deep and recurrent neural networks through output gradients. CoRR 1712.06563 (2017).
  8. 8.
    Gomez, F.J., Miikkulainen, R.: Solving non-Markovian control tasks with neuroevolution. In: Proceedings of the 16th International Joint Conference on Artificial Intelligence, IJCAI 1999, San Francisco, CA, USA, vol. 2, pp. 1356–1361. Morgan Kaufmann Publishers Inc. (1999).
  9. 9.
    Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002). Scholar
  10. 10.
    Stanley, K.O., D’Ambrosio, D.B., Gauci, J.: A hypercube-based encoding for evolving large-scale neural networks. Artif. Life 15(2), 185–212 (2009). Scholar
  11. 11.
    Zhang, X., Clune, J., Stanley, K.O.: On the relationship between the openai evolution strategy and stochastic gradient descent. CoRR 1712.06564 (2017).
  12. 12.
    Such, F.P., Madhavan, V., Conti, E., Lehman, J., Stanley, K.O., Clune, J.: Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning. CoRR 1712.06567 (2017).
  13. 13.
    He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Computing Research Repository, 1502.01852 (2015).
  14. 14.
    Loiacono, D., Cardamone, L., Lanzi, P.L.: Simulated car racing championship: competition software manual. CoRR 1304.1672 (2013).
  15. 15.
    Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D.: Continuous control with deep reinforcement learning. CoRR 1509.02971 (2015).
  16. 16.
    Blickle, T., Thiele, L.: A comparison of selection schemes used in evolutionary algorithms. Evol. Comput. 4(4), 361–394 (1996). Scholar
  17. 17.
    Koehn, P.: Combining genetic algorithms and neural networks: the encoding problem. The University of Tennessee, Knoxville (1994).
  18. 18.
    Gwiazda, T.: Genetic Algorithms Reference Volume 2 Mutation Operator for Numerical Optimization Problems. In: Genetic Algorithms Reference. Simon and Schuster (2007).
  19. 19.
    Lau, B.: Using keras and deep deterministic policy gradient to play TORCS (2016).
  20. 20.
    Ding, W.G.: Python script for illustrating convolutional neural network (convnet) (2018).
  21. 21.
    Sastry, K., Goldberg, D., Kendall, G.: Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques. Springer, Boston (2005).
  22. 22.
    Lehman, J., Chen, J., Clune, J., Stanley, K.O.: ES is more than just a traditional finite-difference approximator. CoRR 1712.06568 (2017).

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Odessa National Polytechnical UniversityOdesaUkraine

Personalised recommendations