Deep Q-Learning for Navigation of Robotic Arm for Tokamak Inspection

  • Swati JainEmail author
  • Priyanka SharmaEmail author
  • Jaina BhoiwalaEmail author
  • Sarthak Gupta
  • Pramit Dutta
  • Krishan Kumar Gotewal
  • Naveen Rastogi
  • Daniel Raju
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11337)


Computerized human-machine interfaces are used to control the manipulators and robots for inspection and maintenance activities in Tokamak. The activities embrace routine and critical activities such as tile inspection, dust cleaning, equipment handling and replacement tasks. Camera(s) is deployed on the robotic arm which moves inside the chamber to accomplish the inspection task. For navigating the robotic arm to the desired position, an inverse kinematic solution is required. Such closed-form inverse kinematic solutions become complex in the case of dexterous hyper-redundant robotic arms that have high degrees of freedom and can be used for inspections in narrow gaps. To develop real-time inverse kinematic solver for robots, a technique called Reinforcement Learning is used. There are various strategies to solve Reinforcement problem in polynomial time, one of them is Q-Learning. It can handle problems with stochastic transitions and rewards, without requiring adaption or probabilities of actions to be taken at a certain point. It is observed that Deep Q-Network successfully learned optimal policies from high dimension sensory inputs using Reinforcement Learning.



This work is conducted at Nirma University, Ahmedabad underfunded research project by the Board of Research in Nuclear Sciences under Department of Atomic Energy.


  1. 1.
    Wang, H., Chen, W., Lai, Y., He, T.: Trajectory planning of tokamak flexible in-vessel inspection robot. Fusion Eng. Des. 98–99, 1678–1682 (2015)CrossRefGoogle Scholar
  2. 2.
    Vijayakumari, D., Dhivya, K.: Conceptual framework of robot with nanowire sensor in nuclear reactor. Int. J. Inf. Futur. Res. 1(11), 146–151 (2014)Google Scholar
  3. 3.
    Hyper-Redundant Robotics Research. Accessed 15 02 2018
  4. 4.
    Dutta, P., Gotewal, K.K., Rastogi, N., Tiwari, R.: A hyper-redundant robot development for tokamak inspection. In: AIR 2017, p. 6 (2017)Google Scholar
  5. 5.
    Wang, H., Xu, L., Chen, W.: Design and implementation of visual inspection system handed in tokamak flexible in-vessel robot. Fusion Eng. Des. 106, 21–28 (2016)CrossRefGoogle Scholar
  6. 6.
    Andrew, G., Gryniewski, M., Campbell, T.: AARM: a robot arm for internal operations in nuclear reactors. In: 2010 1st International Conference on Applied Robotics for the Power Industry, CARPI, pp. 1–5 (2010)Google Scholar
  7. 7.
    Peng, X., Yuan, J., Zhang, W., Yang, Y., Song, Y.: Kinematic and dynamic analysis of a serial-link robot for inspection process in EAST vacuum vessel. Fusion Eng. Des. 87(5), 905–909 (2012)CrossRefGoogle Scholar
  8. 8.
    Liu, J., Wang, Y., Li, B., Ma, S.: Neural network based kinematic control of the hyper-redundant snake-like manipulator. In: Advances in Neural Networks – ISNN 2007, vol. 4491, pp. 339–348, April 2015Google Scholar
  9. 9.
    Liu, J., Wang, Y., Ma, S., Li, B.: RBF neural network based shape control of hyper-redundant manipulator with constrained end-effector. In: Wang, J., Yi, Z., Zurada, Jacek M., Lu, B.-L., Yin, H. (eds.) ISNN 2006. LNCS, vol. 3972, pp. 1146–1152. Springer, Heidelberg (2006). Scholar
  10. 10.
    James, S., Johns, E.: 3D Simulation for Robot Arm Control with Deep Q-Learning, p. 6 (2016)Google Scholar
  11. 11.
    Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Department of Computer Engineering, Institute of TechnologyNirma UniversityAhmedabadIndia
  2. 2.Institute of Plasma ResearchBhat, GandhinagarIndia
  3. 3.Homi Bhabha National InstituteMumbaiIndia

Personalised recommendations