Abstract
We develop and implement a Q-learning based Reinforcement Learning (RL) algorithm for Welding Sequence Optimization (WSO) where structural deformation is used to compute reward function. We utilize a thermomechanical Finite Element Analysis (FEA) method to predict deformation. We run welding simulation experiment using well-known Simufact® software on a typical widely used mounting bracket which contains eight welding beads. RL based welding optimization technique allows the reduction of structural deformation up to ~66%. RL based approach substantially speeds up the computational time over exhaustive search.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Goldak JA, Akhlaghi M (2005) Computational welding mechanics. http://www.worldcat.org/isbn/9780387232874. Accessed 21 May 2016
Masubuchi K (1980) Analysis of welded structures. Pergamon Press Ltd., Oxford
Islam M, Buijk A, Rais-Rohani M et al (2014) Simulation-based numerical optimization of arc welding process for reduced distortion in welded structures. Finite Elem Anal Des 84:54–64
Kumar DA, Biswas P, Mandal NR et al (2011) A study on the effect of welding sequence in fabrication of large stiffened plate panels. J Mar Sci Appl 10:429–436
Jackson K, Darlington R (2011) Advanced engineering methods for assessing welding distortion in aero-engine assemblies. IOP Conf Ser Mater Sci Eng 26:12018
Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT press, Cambridge
Romero-Hdz J et al (2016) An elitism based genetic algorithm for welding sequence optimization to reduce deformation. Res Comput Sci 121:17–36
Romero-Hdz J et al (2016) Deformation and residual stress based multi-objective Genetic Algorithm (GA) for welding sequence optimization. In: Proceedings of the Mexican international conference on artificial intelligence, Cancun, Mexico, vol 1, pp 233–243
Derlukiewicz D, Przybyłek G (2008) Chosen aspects of FEM strength analysis of telescopic jib mounted on mobile platform. Autom Constr 17:278–283
Subbiah S, Singh OP, Mohan SK et al (2011) Effect of muffler mounting bracket designs on durability. Eng Fail Anal 18:1094–1107
Romeo ŞI et al (2015) Study of the dynamic behavior of a car body for mounting the rear axle. Proc Eur Automot Congr 25:782
Mohammed MB, Sun W, Hyde TH (2012) Welding sequence optimization of plasma arc for welded thin structures. WIT Trans Built Environ 125:231–242
Kadivar MH, Jafarpur K, Baradaran GH (2000) Optimizing welding sequence with genetic algorithm. Comput Mech 26:514–519
Damsbo M, Ruboff PT (1998) An evolutionary algorithm for welding task sequence ordering. Proc Artif Intell Symbolic Comput 1476:120–131
Islam M, Buijk A, Rais-Rohani M, Motoyama K (2014) Simulation-based numerical optimization of arc welding process for reduced distortion in welded structures. Finite Elem Anal Des 84:54–64
Warmefjord K, Soderberg R, Lindkvist L (2010) Strategies for optimization of spot welding sequence with respect to geometrical variation in sheet metal assemblies. Design and Manufacturing 3: 569–57
Kim HJ, Kim YD, Lee DH (2004) Scheduling for an arc-welding robot considering heat-caused distortion. J Oper Res Soc 56(1):39–50
Romero-Hdz J et al (2016) Welding sequence optimization using artificial intelligence techniques: an overview. Int J Comput Sci Eng 3(11):90–95
Okumoto Y (2008) Optimization of welding route by automatic machine using reinforcement learning method. J Ship Prod 24:135–138
Tesauro G (1994) TD-gammon, a self-teaching backgammon program, achieves master-level play. Neural Comput 6:215–219
Schultz W, Dayan P, Montague P (1997) A neural substrate of prediction and reward. Science 16:1936–1947
Sutton R (1990) Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In: Proceedings of 7th international conference on machine learning, vol 1, pp 216–224
Choi J, Laibson D, Madrian B et al (2007) Reinforcement learning and savings behavior. Yale Technical Report ICF Working Paper, 09–01
Singh S, Bertsekas D (1997) Reinforcement learning for dynamic channel allocation in cellular telephone systems. Adv Neural Inf Process Syst 9:974–982
Ernst D, Glavic M, Geurts P et al (2005) Approximate value iteration in the reinforcement learning context-application to electrical power system control. Int J Emerg Electr Power Syst 3(1):1066
Abbeel P, Coates A, Quigley M et al (2007) An application of reinforcement learning to aerobatic helicopter flight. Adv Neural Inf Process Syst 19:1–8
Acknowledgements
The authors gratefully acknowledge the support provided by CONACYT (The National Council of Science and Technology) and CIDESI (Center for Engineering and Industrial Development) as well as their personnel that helped to realize this work and the Basic Science Project (254801) supported by CONACYT, Mexico.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Romero-Hdz, J., Saha, B., Toledo-Ramirez, G., Lopez-Juarez, I. (2018). A Reinforcement Learning Based Approach for Welding Sequence Optimization. In: Chen, S., Zhang, Y., Feng, Z. (eds) Transactions on Intelligent Welding Manufacturing. Transactions on Intelligent Welding Manufacturing. Springer, Singapore. https://doi.org/10.1007/978-981-10-7043-3_2
Download citation
DOI: https://doi.org/10.1007/978-981-10-7043-3_2
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-7042-6
Online ISBN: 978-981-10-7043-3
eBook Packages: EngineeringEngineering (R0)