An intelligent weld control strategy based on reinforcement learning approach

  • Zeshi Jin
  • Haichao LiEmail author
  • Hongming Gao


Welding process control is an important part to realize intelligent welding. The actual welding process is a complex and nonlinear system influenced by multiple factors, such as welding current, arc voltage, welding speed, and so on. In addition, welding process is always interfered by working conditions, which make the reliability of general control model reduce greatly. So an intelligent weld control strategy that based on actor-critic reinforcement learning (ACRL) approach is selected to control the width of weld pool. And the gas tungsten arc welding (GTAW) and gas metal arc welding (GMAW) models are used to conduct simulation experiments of welding process control to verify the feasibility of the controller preliminarily. Finally, the opened-loop control experiment and the closed-loop control experiment are done, and the results are compared to verify the reliability of the controller.


Actor-critic reinforcement learning GMAW Pool width Visual sensor technology Image processing Linear regression modeling 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


Funding information

This work was supported by National Natural Science Foundation of China, No. 51475102.


  1. 1.
    Zhang YM (1996) Adaptive control of full penetration gas tungsten arc welding. IEEE T Contr Syst T 4(4):394–403CrossRefGoogle Scholar
  2. 2.
    Charalabos D (2002) Multivariable adaptive control of the bead profile geometry in gas metal arc welding with thermal scanning. Int J Pres Ves Pip 79:251–262CrossRefGoogle Scholar
  3. 3.
    Reeves RE (1988) Expert system technology—an avenue to an intelligent weld process control system. Weld J 67(6):33–41Google Scholar
  4. 4.
    Barborak DM (1991) PC-based expert system and their application to welding. Weld J 70(1):29–38Google Scholar
  5. 5.
    Hu PF, Huang JS, Zeng M (2017) Application of fuzzy control method in gas metal arc welding. Int J Adv Manuf Technol 92:1769–1775CrossRefGoogle Scholar
  6. 6.
    Lv N, Xu YL, Li SC, Yu XW, Chen SB (2017) Automated control of welding penetration based on audio sensing technology. J Mater Process Technol 250(12):81–98CrossRefGoogle Scholar
  7. 7.
    Liu YK, Zhang WJ, Zhang YM (2014) Neuro-fuzzy based human intelligence modeling and robust control in gas tungsten arc welding process. IEEE T Autom Sci Eng 12(1):324–335MathSciNetCrossRefGoogle Scholar
  8. 8.
    Aviles-Vinas JF, Lopez-Juarez I, Rios-Cabrera R (2015) Acquisition of welding skills in industrial robots. Ind Robot 42(2):156–166CrossRefGoogle Scholar
  9. 9.
    Aviles-Vinas JF, Rios-Cabrera R, Lopez-Juarez I (2016) On-line learning of welding bead geometry in industrial robots. Int J Adv Manuf Technol 83:217–231CrossRefGoogle Scholar
  10. 10.
    Cruz JG, Torres EM, Alfaro SCA (2015) A methodology for modeling and control of weld bead width in the GMAW process. J Braz Soc Mech Sci Eng 37:1529–1541CrossRefGoogle Scholar
  11. 11.
    Chu WH, Tung PC (2005) Development of an automatic arc welding system using a sliding mode control. Int J Mach Tool Manu 45:933–939CrossRefGoogle Scholar
  12. 12.
    Du QY (2006) Extraction and intelligent control of 3D dynamic weld pool shape information of pulsed GTAW with wire filler. PhD thesis. Shanghai Jiao Tong UniversityGoogle Scholar
  13. 13.
    Silva GJ, Datta A, Bhattacharyya SP (2002) New results on the synthesis of PID controllers. IEEE T Automat Contr 47(2):241–252MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Lou YJ (1998) Intelligent control for pulsed GTAW dynamic process based on image sensing of weld pool. PhD thesis. Harbin institute of technologyGoogle Scholar
  15. 15.
    Zhang GJ, Chen SB, Wu L (2003) Neuron self-learning PSD control for backside width of weld Pool in pulsed GTAW with wire filler. China Weld 12(1):87–91Google Scholar
  16. 16.
    Günther J, Pilarski PM, Helfrich G, Shen H, Diepold K (2016) Intelligent laser welding through representation, prediction, and control learning: an architecture with deep neural networks and reinforcement learning. Mechatronics 34:1–11CrossRefGoogle Scholar
  17. 17.
    Wang XS, Cheng YH, Sun W (2007) A proposal of adaptive PID controller based on reinforcement learning. J China Univ Min Technol 17(1):40–44CrossRefGoogle Scholar
  18. 18.
    Marr D, Hildreth E (1980) Theory of edge detection. Proc R Soc Lond B 207:187–217CrossRefGoogle Scholar
  19. 19.
    Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press Cambridge, Massachusetts; London, EnglandGoogle Scholar
  20. 20.
    Jin ZS, Li HC, Jia GQ, Gao HM (2016) Dynamic nonlinear modeling of 3D weld pool surface in GTAW. Robot Comput Integr Manuf 39:1–8CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2018

Authors and Affiliations

  1. 1.State Key Laboratory of Advanced Welding and JoiningHarbin Institute of TechnologyHarbinPeople’s Republic of China

Personalised recommendations