Skip to main content
SpringerLink
Log in

From Reinforcement Learning Towards Artificial General Intelligence

  • Conference paper
  • First Online:
Trends and Innovations in Information Systems and Technologies (WorldCIST 2020)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1160))

Included in the following conference series:

Abstract

The present work surveys research that integrates successfully a number of complementary fields in Artificial Intelligence. Starting from integrations in Reinforcement Learning: Deep Reinforcement Learning and Relational Reinforcement Learning, we then present Neural-Symbolic Learning and Reasoning since it is applied to Deep Reinforcement Learning. Finally, we present integrations in Deep Reinforcement Learning, such as, Relational Deep Reinforcement Learning. We propose that this road is breaking through barriers in Reinforcement Learning and making us closer to Artificial General Intelligence, and we share views about the current challenges to get us further towards this goal.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Russell, S.J., Norvig, P.: Artificial Intelligence - A Modern Approach. Pearson Education, London (2010). Third International Edition

    MATH  Google Scholar 

  2. LeCun, Y., Bengio, Y., Hinton, G.E.: Deep learning. Nature 521, 436–444 (2015)

    Article  Google Scholar 

  3. Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction. IEEE Trans. Neural Netw. 16, 285–286 (1988)

    MATH  Google Scholar 

  4. Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: a survey. ArXiv, cs.AI/9605103 (1996)

    Google Scholar 

  5. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M.A., Fidjeland, A., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)

    Article  Google Scholar 

  6. Irpan, A.: Deep reinforcement learning doesn’t work yet (2018). https://www.alexirpan.com/2018/02/14/rl-hard.html

  7. Domingos, P.: The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World. Basic Books, New York (2015)

    Google Scholar 

  8. Bishop, C.M.: Pattern Recognition and Machine Learning. Information Science and Statistics, 5th edn. Springer, Heidelberg (2007)

    MATH  Google Scholar 

  9. Domingos, P.M., Lowd, D.: Unifying logical and statistical AI with markov logic. Commun. ACM 62(7), 74–83 (2019)

    Article  Google Scholar 

  10. Garnelo, M., Arulkumaran, K., Shanahan, M.: Towards deep symbolic reinforcement learning. ArXiv, abs/1609.05518 (2016)

    Google Scholar 

  11. Santoro, A., Raposo, D., Barrett, D.G.T., Malinowski, M., Pascanu, R., Battaglia, P.W., Lillicrap, T.P.: A simple neural network module for relational reasoning. In: NIPS (2017)

    Google Scholar 

  12. Zambaldi, V.F., Raposo, D., Santoro, A., Bapst, V., Li, Y., Babuschkin, I., Tuyls, K., Reichert, D.P., Lillicrap, T.P., Lockhart, E., Shanahan, M., Langston, V., Pascanu, R., Botvinick, M.M., Vinyals, O., Battaglia, P.W.: Relational deep reinforcement learning. ArXiv, abs/1806.01830 (2018)

    Google Scholar 

  13. Paes, A., Zaverucha, G., Costa, V.S.: On the use of stochastic local search techniques to revise first-order logic theories from examples. Mach. Learn. 106(2), 197–241 (2017)

    Article  MathSciNet  Google Scholar 

  14. Fitting, M.: First-Order Logic and Automated Theorem Proving. Graduate Texts in Computer Science, 2nd edn. Springer, Heidelberg (1996)

    Book  Google Scholar 

  15. Christopher JCH Watkins and Peter Dayan: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)

    MATH  Google Scholar 

  16. Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T.P., Leach, M., Kavukcuoglu, K., Graepel, T., Hassabis, D.: Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489 (2016)

    Article  Google Scholar 

  17. Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T.P., Simonyan, K., Hassabis, D.: A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 362, 1140–1144 (2018)

    Article  MathSciNet  Google Scholar 

  18. Ghazanfari, B., Afghah, F., Taylor, M.E.: Autonomous extraction of a hierarchical structure of tasks in reinforcement learning, a sequential associate rule mining approach. ArXiv, abs/1811.08275 (2018)

    Google Scholar 

  19. Ghazanfari, B., Taylor, M.E.: Autonomous extracting a hierarchical structure of tasks in reinforcement learning and multi-task reinforcement learning. ArXiv, abs/1709.04579 (2017)

    Google Scholar 

  20. El Bsat, S., Bou-Ammar, H., Taylor, M.E.: Scalable multitask policy gradient reinforcement learning. In: AAAI (2017)

    Google Scholar 

  21. Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Abbeel, P., Zaremba, W.: Hindsight experience replay. In: Advances in Neural Information Processing Systems, pp. 5048–5058 (2017)

    Google Scholar 

  22. Sutton, R.S.: Dyna, an integrated architecture for learning, planning, and reacting. SIGART Bull. 2, 160–163 (1990)

    Article  Google Scholar 

  23. Finn, C., Yu, T., Zhang, T., Abbeel, P., Levine, S.: One-shot visual imitation learning via meta-learning. ArXiv, abs/1709.04905 (2017)

    Google Scholar 

  24. Andreas, J., Klein, D., Levine, S.: Modular multitask reinforcement learning with policy sketches. ArXiv, abs/1611.01796 (2016)

    Google Scholar 

  25. Hussein, A., Gaber, M.M., Elyan, E., Jayne, C.: Imitation learning: a survey of learning methods. ACM Comput. Surv. 50, 21:1–21:35 (2017)

    Article  Google Scholar 

  26. Gunning, D.: Explainable artificial intelligence (XAI). Defense Advanced Research Projects Agency (DARPA), nd Web, 2 (2017)

    Google Scholar 

  27. Ribeiro, M.T., Singh, S., Guestrin, C.: Why should i trust you?: explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135–1144. ACM (2016)

    Google Scholar 

  28. Osband, I., Doron, Y., Hessel, M., Aslanides, J., Sezener, E., Saraiva, A., McKinney, K., Lattimore, T., Szepezvári, C., Singh, S., Van Roy, B., Sutton, R.S., Silver, D., van Hasselt, H.: Behaviour suite for reinforcement learning. ArXiv, abs/1908.03568 (2019)

    Google Scholar 

  29. Džeroski, S., De Raedt, L., Driessens, K.: Relational reinforcement learning. Machine learning 43(1–2), 7–52 (2001)

    Article  Google Scholar 

  30. Tadepalli, P., Givan, R., Driessens, K.: Relational reinforcement learning: an overview. In: Proceedings of the ICML-2004 Workshop on Relational Reinforcement Learning, pp. 1–9 (2004)

    Google Scholar 

  31. Van Otterlo, M.: Relational representations in reinforcement learning: review and open problems. In: Proceedings of the ICML, vol. 2 (2002)

    Google Scholar 

  32. Quinlan, J.R.: Learning logical definitions from relations. Mach. Learn. 5, 239–266 (1990)

    Google Scholar 

  33. Morales, E.F.: Scaling up reinforcement learning with a relational representation. In: Proceedings of the Workshop on Adaptability in Multi-agent Systems, pp. 15–26 (2003)

    Google Scholar 

  34. Neural-symbolic integration. http://www.neural-symbolic.org

  35. Evans, R., Grefenstette, E.: Learning explanatory rules from noisy data. J. Artif. Intell. Res. 61, 1–64 (2017)

    Article  MathSciNet  Google Scholar 

  36. Lloyd, J.W.: Foundations of logic programming. In: Symbolic Computation (1984)

    Google Scholar 

  37. Dong, H., Mao, J., Lin, T., Wang, C., Li, L., Zhou, D.: Neural logic machines. ArXiv, abs/1904.11694 (2019)

    Google Scholar 

  38. Trask, A., Hill, F., Reed, S.E., Rae, J.W., Dyer, C., Blunsom, P.: Neural arithmetic logic units. In: NeurIPS (2018)

    Google Scholar 

  39. Faris, W.G.: The number sense: how the mind creates mathematics by stanislas dehaene. Complexity 4(1), 46–48 (1998)

    Article  Google Scholar 

  40. Gallistel, C.R.: Finding numbers in the brain. Philos. Trans. Roy. Soc. London Ser. B Biol. Sci. 373(1740) (2017)

    Google Scholar 

  41. Fodor, J.A., Pylyshyn, Z.W.: Connectionism and cognitive architecture: a critical analysis. Cognition 28, 3–71 (1988)

    Article  Google Scholar 

  42. Marcus, G.F.: Integrating connectionism and cognitive science, The algebraic mind (2001)

    Google Scholar 

  43. Battaglia, P.W., Hamrick, J.B., Bapst, V., Sanchez-Gonzalez, A., Zambaldi, V.F., Malinowski, M., Tacchetti, A., Raposo, D., Santoro, A., Faulkner, R., Gülçehre, Ç., Francis Song, H., Ballard, A.J., Gilmer, J., Dahl, G.E., Vaswani, A., Allen, K.R., Nash, C., Langston, V., Dyer, C., Heess, N.M.O., Wierstra, D., Kohli, P., Botvinick, M.M., Vinyals, O., Li, Y., Pascanu, R.: Relational inductive biases, deep learning, and graph networks. ArXiv, abs/1806.01261 (2018)

    Google Scholar 

  44. Battaglia, P.W., Pascanu, R., Lai, M., Rezende, D.J., Kavukcuoglu, K.: Interaction networks for learning about objects, relations and physics. In: NIPS (2016)

    Google Scholar 

  45. Zambaldi, V.F., Raposo, D.C., Santoro, A., Bapst, V., Li, Y., Babuschkin, I., Tuyls, K., Reichert, D.P., Lillicrap, T.P., Lockhart, E., Shanahan, M., Langston, V., Pascanu, R., Botvinick, M.M., Vinyals, O., Battaglia, P.W.: Deep reinforcement learning with relational inductive biases. In: ICLR (2019)

    Google Scholar 

  46. Jiang, Z., Luo, S.: Neural logic reinforcement learning. ArXiv, abs/1904.10729 (2019)

    Google Scholar 

  47. Sutton, R.S., McAllester, D.A., Singh, S.P., Mansour, Y.: Policy gradient methods for reinforcement learning with function approximation. In: NIPS (1999)

    Google Scholar 

  48. Botvinick, M.M., Barrett, D.G.T., Battaglia, P.W., de Freitas, N., Kumaran, D., Leibo, J.Z., Lillicrap, T., Modayil, J., Mohamed, S., Rabinowitz, N.C., Rezende, D.J., Santoro, A., Schaul, T., Summerfield, C., Wayne, G., Weber, T., Wierstra, D., Legg, S., Hassabis, D.: Building machines that learn and think for themselves: commentary on lake et al., behavioral and brain sciences, 2017. Behavioral Brain Sci. 40, e255 (2017)

    Article  Google Scholar 

  49. Lake, B.M., Ullman, T.D., Tenenbaum, J.B., Gershman, S.J.: Building machines that learn and think like people. Behav. Brain Sci. 40, e253 (2017)

    Article  Google Scholar 

Download references

Acknowledgements

We thank the institutions CRACS/INESCTEC\(^4\) and LIACC/UP\(^3\), for the support and contribution of its members, that was very valuable, for the research behind this paper and its presentation.

This work is financed by National Funds through the Portuguese funding agency, FCT - Fundação para a Ciência e a Tecnologia within projects: UID/EEA/50014 and UID/EEA/00027.

Author information

Authors and Affiliations

  1. FCUP – Faculdade de Ciências da Universidade do Porto, Porto, Portugal

    Filipe Marinho Rocha & Vítor Santos Costa

  2. FEUP – Faculdade de Engenharia da Universidade do Porto, Porto, Portugal

    Luís Paulo Reis

  3. LIACC – Laboratório de Inteligência Artificial e Ciência de Computadores, Universidade do Porto, Porto, Portugal

    Filipe Marinho Rocha & Luís Paulo Reis

  4. CRACS – Centre of Advanced Computing Systems, INESCTEC, Porto, Portugal

    Filipe Marinho Rocha & Vítor Santos Costa

Authors
  1. Filipe Marinho Rocha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vítor Santos Costa
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luís Paulo Reis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Filipe Marinho Rocha .

Editor information

Editors and Affiliations

  1. Departamento de Engenharia Informática, Universidade de Coimbra, Coimbra, Portugal

    Álvaro Rocha

  2. College of Engineering, The Ohio State University, Columbus, OH, USA

    Hojjat Adeli

  3. FEUP, Universidade do Porto, Porto, Portugal

    Luís Paulo Reis

  4. DIMES, Università della Calabria, Arcavacata di Rende, Italy

    Sandra Costanzo

  5. Faculty of Electrical Engineering, University of Montenegro, Podgorica, Montenegro

    Irena Orovic

  6. Universidade Portucalense, Porto, Portugal

    Fernando Moreira

Rights and permissions

Reprints and permissions

Copyright information

© 2020 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rocha, F.M., Costa, V.S., Reis, L.P. (2020). From Reinforcement Learning Towards Artificial General Intelligence. In: Rocha, Á., Adeli, H., Reis, L., Costanzo, S., Orovic, I., Moreira, F. (eds) Trends and Innovations in Information Systems and Technologies. WorldCIST 2020. Advances in Intelligent Systems and Computing, vol 1160. Springer, Cham. https://doi.org/10.1007/978-3-030-45691-7_37

Download citation

Publish with us

Policies and ethics

Search

Navigation