Control of Nameplate Pasting Robot for Sand Mold Based on Deep Reinforcement Learning

  • Guiben Tuo
  • Te LiEmail author
  • Haibo Qin
  • Bin Huang
  • Kuo Liu
  • Yongqing Wang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11745)


In order to solve the problem of low-efficiency in the manual operation process of nameplate pasting for sand mold, an intelligent simulation system based on visual sensing and industrial robot is designed to paste nameplate on sand molds, and a deep reinforcement learning control method is proposed. The simulation system including the robot, visual sensor and sand mold is established in ROS combined with the physical simulation engine Gazebo. Then the task of nameplate pasting for sand molds is expressed as a markov process and the robot is trained by DQN method to learn a strategy to complete the task of pasting the nameplate of sand mold. A multi-level reward function algorithm based on multi-distances and collision information is proposed to improve the train success rate. Finally, the method is verified in the simulation system. The results show that the nameplate can be quickly attached to the sand mold cavity by the industrial robot.


Sand mold Nameplate pasting Industrial robot DQN 



This work is partially supported by the National Science Foundation for Young Scientists of China (Grant No. 51805071), the Fundamental Research Funds for the Central Universities (Grant No. DUT18RC(3)073) and Changjiang Scholar Program of Chinese Ministry of Education (No. T2017030).


  1. 1.
    Dame, A., Marchand, E.: Mutual information-based visual servoing. IEEE Trans. Robot. 27(5), 958–969 (2011)CrossRefGoogle Scholar
  2. 2.
    Silveira, G., Malis, E.: Direct visual servoing: vision-based estimation and control using only nonmetric information. IEEE Trans. Robot. 28(4), 974–980 (2012)CrossRefGoogle Scholar
  3. 3.
    Bo, T., Zeyu, G., Han, D.: Survey on uncalibrated robot visual servoing control. Chin. J. Theor. Appl. Mech. 48(4), 767–783 (2016)Google Scholar
  4. 4.
    Jia, B., Liu, S., Zhang, K., Chen, J.: Survey on robot visual servo control: vision system and control strategies. AAS 41(5), 861–873 (2015)Google Scholar
  5. 5.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)zbMATHGoogle Scholar
  6. 6.
    Malis, E., Rives, P.: Robustness of image-based visual servoing with respect to depth distribution errors. In: Proceedings of the 2003 IEEE International Conference on Robotics and Automation, Taipei, China, pp. 1056–1061. IEEE (2003)Google Scholar
  7. 7.
    Mnih, V., Kavukcuoglu, K., Silver, D.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)CrossRefGoogle Scholar
  8. 8.
    Mnih, V., et al.: Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)
  9. 9.
    Zhang, F., Leitner, J., Milford, M., et al.: Towards vision-based deep reinforcement learning for robotic motion control. Comput. Sci. (2015)Google Scholar
  10. 10.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. 2nd edition in progress. London, England (2017)Google Scholar
  11. 11.
    Xian, G.: The study of robotic arm control policy based on DQN. Beijing Jiaotong University, pp. 41–45 (2018)Google Scholar
  12. 12.
    James, S., Johns, E.: 3D simulation for robot arm control with deep Q-learning. arXiv preprint (2016)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Guiben Tuo
    • 1
  • Te Li
    • 1
    Email author
  • Haibo Qin
    • 1
  • Bin Huang
    • 1
  • Kuo Liu
    • 1
  • Yongqing Wang
    • 1
  1. 1.Key Laboratory for Precision and Non-traditional Machining of Ministry of EducationDalian University of TechnologyDalianChina

Personalised recommendations