Abstract
Modern robotics uses many advanced precise algorithms to control autonomous agents. Now arises tendency to apply machine learning in niches, where precise algorithms are hard to design or implement. With machine learning, for continuous control tasks, evolution strategies are used. We propose an enhancement to crossover operator, which diminishes probability of degraded offsprings compared to conventional crossover operators. Our experiments in TORCS environment show, that presented algorithm can evolve robust neural networks for non-trivial continuous control tasks such as driving a racing car in various tracks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kingma, D.P., Ba, J.: Adam: A Method for Stochastic Optimization. CoRR 1412.6980 (2014). http://arxiv.org/abs/1412.6980
Bojarski, M., Testa, D.D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L.D., Monfort, M., Muller, U., Zhang, J., Zhang, X., Zhao, J., Zieba, K.: End to End Learning for Self-Driving Cars. CoRR 1604.07316 (2016). http://arxiv.org/abs/1604.07316
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.A.: Playing atari with deep reinforcement learning. CoRR 1312.5602 (2013). http://arxiv.org/abs/1312.5602
Nair, A., McGrew, B., Andrychowicz, M., Zaremba, W., Abbeel, P.: Overcoming exploration in reinforcement learning with demonstrations. CoRR 1709.10089 (2017). http://arxiv.org/abs/1709.10089
Bruce, J., Sünderhauf, N., Mirowski, P., Hadsell, R., Milford, M.: One-shot reinforcement learning for robot navigation with interactive replay. CoRR 1711.10137 (2017). http://arxiv.org/abs/1711.10137
Salimans, T., Ho, J., Chen, X., Sutskever, I.: Evolution strategies as a scalable alternative to reinforcement learning. CoRR 1703.03864 (2017). http://arxiv.org/abs/1703.03864
Lehman, J., Chen, J., Clune, J., Stanley, K.O.: Safe mutations for deep and recurrent neural networks through output gradients. CoRR 1712.06563 (2017). http://arxiv.org/abs/1712.06563
Gomez, F.J., Miikkulainen, R.: Solving non-Markovian control tasks with neuroevolution. In: Proceedings of the 16th International Joint Conference on Artificial Intelligence, IJCAI 1999, San Francisco, CA, USA, vol. 2, pp. 1356–1361. Morgan Kaufmann Publishers Inc. (1999). http://dl.acm.org/citation.cfm?id=1624312.1624411
Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002). https://doi.org/10.1162/106365602320169811
Stanley, K.O., D’Ambrosio, D.B., Gauci, J.: A hypercube-based encoding for evolving large-scale neural networks. Artif. Life 15(2), 185–212 (2009). https://doi.org/10.1162/artl.2009.15.2.15202
Zhang, X., Clune, J., Stanley, K.O.: On the relationship between the openai evolution strategy and stochastic gradient descent. CoRR 1712.06564 (2017). http://arxiv.org/abs/1712.06564
Such, F.P., Madhavan, V., Conti, E., Lehman, J., Stanley, K.O., Clune, J.: Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning. CoRR 1712.06567 (2017). http://arxiv.org/abs/1712.06567
He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Computing Research Repository, 1502.01852 (2015). http://arxiv.org/abs/1502.01852
Loiacono, D., Cardamone, L., Lanzi, P.L.: Simulated car racing championship: competition software manual. CoRR 1304.1672 (2013). http://arxiv.org/abs/1304.1672
Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D.: Continuous control with deep reinforcement learning. CoRR 1509.02971 (2015). http://arxiv.org/abs/1509.02971
Blickle, T., Thiele, L.: A comparison of selection schemes used in evolutionary algorithms. Evol. Comput. 4(4), 361–394 (1996). https://doi.org/10.1162/evco.1996.4.4.361
Koehn, P.: Combining genetic algorithms and neural networks: the encoding problem. The University of Tennessee, Knoxville (1994). http://homepages.inf.ed.ac.uk/pkoehn/publications/gann94.pdf
Gwiazda, T.: Genetic Algorithms Reference Volume 2 Mutation Operator for Numerical Optimization Problems. In: Genetic Algorithms Reference. Simon and Schuster (2007). https://books.google.com.ua/books?id=O1FVGQAACAAJ
Lau, B.: Using keras and deep deterministic policy gradient to play TORCS (2016). https://yanpanlau.github.io/2016/10/11/Torcs-Keras.html
Ding, W.G.: Python script for illustrating convolutional neural network (convnet) (2018). https://github.com/gwding/drawconvnet
Sastry, K., Goldberg, D., Kendall, G.: Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques. Springer, Boston (2005). https://doi.org/10.1007/0-387-28356-0_4
Lehman, J., Chen, J., Clune, J., Stanley, K.O.: ES is more than just a traditional finite-difference approximator. CoRR 1712.06568 (2017). http://arxiv.org/abs/1712.06568
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix A Track Outlines
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Tymchenko, B., Antoshchuk, S. (2019). Race from Pixels: Evolving Neural Network Controller for Vision-Based Car Driving. In: Chertov, O., Mylovanov, T., Kondratenko, Y., Kacprzyk, J., Kreinovich, V., Stefanuk, V. (eds) Recent Developments in Data Science and Intelligent Analysis of Information. ICDSIAI 2018. Advances in Intelligent Systems and Computing, vol 836. Springer, Cham. https://doi.org/10.1007/978-3-319-97885-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-97885-7_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97884-0
Online ISBN: 978-3-319-97885-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)