Advertisement

Race from Pixels: Evolving Neural Network Controller for Vision-Based Car Driving

  • Borys Tymchenko
  • Svitlana Antoshchuk
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 836)

Abstract

Modern robotics uses many advanced precise algorithms to control autonomous agents. Now arises tendency to apply machine learning in niches, where precise algorithms are hard to design or implement. With machine learning, for continuous control tasks, evolution strategies are used. We propose an enhancement to crossover operator, which diminishes probability of degraded offsprings compared to conventional crossover operators. Our experiments in TORCS environment show, that presented algorithm can evolve robust neural networks for non-trivial continuous control tasks such as driving a racing car in various tracks.

Keywords

Neuroevolution Genetic algorithms Neural networks Deep learning Continuous control Crossover Autonomous vehicles TORCS 

References

  1. 1.
    Kingma, D.P., Ba, J.: Adam: A Method for Stochastic Optimization. CoRR 1412.6980 (2014). http://arxiv.org/abs/1412.6980
  2. 2.
    Bojarski, M., Testa, D.D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L.D., Monfort, M., Muller, U., Zhang, J., Zhang, X., Zhao, J., Zieba, K.: End to End Learning for Self-Driving Cars. CoRR 1604.07316 (2016). http://arxiv.org/abs/1604.07316
  3. 3.
    Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.A.: Playing atari with deep reinforcement learning. CoRR 1312.5602 (2013). http://arxiv.org/abs/1312.5602
  4. 4.
    Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., Abbeel, P.: Overcoming exploration in reinforcement learning with demonstrations. CoRR 1709.10089 (2017). http://arxiv.org/abs/1709.10089
  5. 5.
    Bruce, J., Sünderhauf, N., Mirowski, P., Hadsell, R., Milford, M.: One-shot reinforcement learning for robot navigation with interactive replay. CoRR 1711.10137 (2017). http://arxiv.org/abs/1711.10137
  6. 6.
    Salimans, T., Ho, J., Chen, X., Sutskever, I.: Evolution strategies as a scalable alternative to reinforcement learning. CoRR 1703.03864 (2017). http://arxiv.org/abs/1703.03864
  7. 7.
    Lehman, J., Chen, J., Clune, J., Stanley, K.O.: Safe mutations for deep and recurrent neural networks through output gradients. CoRR 1712.06563 (2017). http://arxiv.org/abs/1712.06563
  8. 8.
    Gomez, F.J., Miikkulainen, R.: Solving non-Markovian control tasks with neuroevolution. In: Proceedings of the 16th International Joint Conference on Artificial Intelligence, IJCAI 1999, San Francisco, CA, USA, vol. 2, pp. 1356–1361. Morgan Kaufmann Publishers Inc. (1999). http://dl.acm.org/citation.cfm?id=1624312.1624411
  9. 9.
    Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002).  https://doi.org/10.1162/106365602320169811CrossRefGoogle Scholar
  10. 10.
    Stanley, K.O., D’Ambrosio, D.B., Gauci, J.: A hypercube-based encoding for evolving large-scale neural networks. Artif. Life 15(2), 185–212 (2009).  https://doi.org/10.1162/artl.2009.15.2.15202CrossRefGoogle Scholar
  11. 11.
    Zhang, X., Clune, J., Stanley, K.O.: On the relationship between the openai evolution strategy and stochastic gradient descent. CoRR 1712.06564 (2017). http://arxiv.org/abs/1712.06564
  12. 12.
    Such, F.P., Madhavan, V., Conti, E., Lehman, J., Stanley, K.O., Clune, J.: Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning. CoRR 1712.06567 (2017). http://arxiv.org/abs/1712.06567
  13. 13.
    He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Computing Research Repository, 1502.01852 (2015). http://arxiv.org/abs/1502.01852
  14. 14.
    Loiacono, D., Cardamone, L., Lanzi, P.L.: Simulated car racing championship: competition software manual. CoRR 1304.1672 (2013). http://arxiv.org/abs/1304.1672
  15. 15.
    Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D.: Continuous control with deep reinforcement learning. CoRR 1509.02971 (2015). http://arxiv.org/abs/1509.02971
  16. 16.
    Blickle, T., Thiele, L.: A comparison of selection schemes used in evolutionary algorithms. Evol. Comput. 4(4), 361–394 (1996).  https://doi.org/10.1162/evco.1996.4.4.361CrossRefGoogle Scholar
  17. 17.
    Koehn, P.: Combining genetic algorithms and neural networks: the encoding problem. The University of Tennessee, Knoxville (1994). http://homepages.inf.ed.ac.uk/pkoehn/publications/gann94.pdf
  18. 18.
    Gwiazda, T.: Genetic Algorithms Reference Volume 2 Mutation Operator for Numerical Optimization Problems. In: Genetic Algorithms Reference. Simon and Schuster (2007). https://books.google.com.ua/books?id=O1FVGQAACAAJ
  19. 19.
    Lau, B.: Using keras and deep deterministic policy gradient to play TORCS (2016). https://yanpanlau.github.io/2016/10/11/Torcs-Keras.html
  20. 20.
    Ding, W.G.: Python script for illustrating convolutional neural network (convnet) (2018). https://github.com/gwding/drawconvnet
  21. 21.
    Sastry, K., Goldberg, D., Kendall, G.: Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques. Springer, Boston (2005).  https://doi.org/10.1007/0-387-28356-0_4
  22. 22.
    Lehman, J., Chen, J., Clune, J., Stanley, K.O.: ES is more than just a traditional finite-difference approximator. CoRR 1712.06568 (2017). http://arxiv.org/abs/1712.06568

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Odessa National Polytechnical UniversityOdesaUkraine

Personalised recommendations