“Re:ROS”: Prototyping of Reinforcement Learning Environment for Asynchronous Cognitive Architecture

  • Sei UenoEmail author
  • Masahiko Osawa
  • Michita Imai
  • Tsuneo Kato
  • Hiroshi Yamakawa
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 636)


Reinforcement learning (RL), which is a field of machine learning, is effective for behavior acquisition in robots. Asynchronous cognitive architecture, which is a method to model human intelligence, is also effective for behavior acquisition. Accordingly, the combination of RL and asynchronous cognitive architecture is expected to be effective. However, early work on the RL toolkit cannot apply asynchronous cognitive architecture because it cannot solve the difference between the asynchrony, which the asynchronous cognitive architecture has, and the synchrony, which RL modules have. In this study, we propose an RL environment for robots that can apply the asynchronous cognitive architecture by applying asynchronous systems to RL modules. We prototyped the RL environment named “Re:ROS.”


Reinforcement learning Cognitive architecture Reinforcement learning environment Robotics 


  1. 1.
    Anderson, J.R.: ACT: a simple theory of complex cognition. Am. Psychol. 51(4), 355 (1996)CrossRefGoogle Scholar
  2. 2.
    Brooks, R.: A robust layered control system for a mobile robot. IEEE J. Robot. Autom. 2(1), 14–23 (1986)CrossRefGoogle Scholar
  3. 3.
    Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: OpenAI Gym. arXiv preprint arXiv:1606.01540 (2016)
  4. 4.
    Zamora, I., Lopez, N.G., Vilches, V.M., Cordero, A.H.: Extending the OpenAI Gym for robotics: a toolkit for reinforcement learning using ROS and Gazebo. arXiv preprint arXiv:1608.05742 (2016)
  5. 5.
    Kimura, H., Yamashita, T., Kobayashi, S.: Reinforcement learning of walking behavior for a four-legged robot. IEEJ Trans. Electron. Inform. Syst. 122(3), 330–337 (2002)Google Scholar
  6. 6.
    Koenig, N., Howard, A.: Gazebo-3D multiple robot simulator with dynamics (2006)Google Scholar
  7. 7.
    Just, M.A., Varma, S.: The organization of thinking: what functional brain imaging reveals about the neuroarchitecture of complex cognition. Cogn. Affect. Behav. Neurosci. 7(3), 153–191 (2007)CrossRefGoogle Scholar
  8. 8.
    Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., Ng, A.Y.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software, vol. 3(3.2), p. 5 (2009)Google Scholar
  9. 9.
    Osawa, M., Ashihara, Y., Shimada, D., Kurihara, S., Imai, M.: Arbitration of multiple learner and application of cognitive architecture using accumulator utilizing prefrontal area. In: 4th SIG-AGI (2016)Google Scholar
  10. 10.
    Richard, S., Sutton, G., Barto, A.: Reinforcement Learning: An Introduction, 1(1). MIT Press, Cambridge (1998)Google Scholar
  11. 11.
    Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing Atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)
  12. 12.
    Yamada, K., Ohkura, K., Ueda, K.: Cooperative behavior acquisition of autonomous arm robots through reinforcement learning. Trans. Soc. Instr. Control Eng. 39(3), 266–275 (2003)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Sei Ueno
    • 1
    • 2
    Email author
  • Masahiko Osawa
    • 3
    • 4
  • Michita Imai
    • 4
  • Tsuneo Kato
    • 1
  • Hiroshi Yamakawa
    • 5
    • 6
  1. 1.Faculty of Science and EngineeringDoshisha UniversityKyotoJapan
  2. 2.Kyoto University of InformaticsKyotoJapan
  3. 3.Japan Society for Promotion of ScienceTokyoJapan
  4. 4.Graduate School of Science and TechnologyKeio UniversityTokyoJapan
  5. 5.Dwango Artificial Intelligence LaboratoryDwango Ltd.TokyoJapan
  6. 6.The Whole Brain Architecture Initiative (A Specified Non-Profit Organization)TokyoJapan

Personalised recommendations