Abstract
This paper presents a novel approach for self-optimization and learning as well as plug-and-play control of highly flexible, modular manufacturing units. The approach is inspired by recent encouraging results of game theory (GT) based learning in computer and control science. However, instead of representing the entire control behavior as a strategic game which might results in long training times and huge data set requirements, we restrict the learning process to the supervisor level by defining appropriate parameters from the basic control level (BCL) to be learned by learning agents. To this end, we define a set of interface parameters to the BCL programmed by IEC 61131 compatible code, which will be used for learning. Typical control parameters include switching thresholds, timing parameters and transition conditions. These parameters will then be considered as players in a multi-player game resulting in a distributed optimization approach. We apply the approach to a laboratory testbed consisting of different production modules which underlines the efficiency improvements for manufacturing units. In addition, plug-and-produce control is enabled by the approach as different configuration of production modules can efficiently be put in operation by re-learning the parameter sets.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
J. Pfrommer, D. Stogl, K. Aleksandrov, S.E Navarro, B. Hein, J. Beyerer, Plug & produce by modelling skills and service-oriented orchestration of reconfigurable manufacturing systems. at-Automatisierungstechnik 63(10), pp. 790–800 (2015)
M. Schleipen, A. Lüder, O. Sauer, H. Flatt, J. Jasperneite, Requirements and concept for plug-and-work-adaptivity in the context of industry 4.0. at-Automatisierungstechnik 63(10), 801–820 (2015)
R.W. Brennan, P. Vrba, P. Tichy, A. Zoitl, C. Sünder, T. Strasser, V. Marik, Developments in dynamic and intelligent reconfiguration of industrial automation. Comput. Ind. 59(6), 533–547 (2008)
P. Leitao, A. Walter Colombo, S. Karnouskos, Industrial automation based on cyber-physical systems technologies: prototype implementations and challenges, Comput. Ind. 81, 11–25 (2016)
J. Wang, Y. Ma, L. Zhang, R.X. Gao, D. Wu, Deep learning for smart manufacturing: methods and applications. J. Manufact. Syst. (in Press) (2018)
T. Wuest, D. Weimer, C. Irgens, K.-D. Thoben, Machine learning in manufacturing: advantages, challenges, and applications. Prod. Manufact. Res. 4(1), 23–45 (2016)
D. Bauso, Game Theory with Engineering Applications (Society for Industrial and Applied Mathematics, Philadelphia, 2016)
S. Zazo, S. Valcarcel Macua, M. Sánchez-Fernández, J. Zazo, Dynamic potential games with constraints: fundamentals and applications in communications. IEEE Trans. Ind. Electron. Mag. 3(4), 49–55 (2015)
S. Buzzi, G. Colavolpe, D. Saturnino, A. Zappone, Potential games for energy-efficient power control and subcarrier allocation in uplink multicell OFDMA systems. IEEE J. Sel. Top. Sig. Process. 6(2), 89–103 (2012)
K. Yamamoto, A comprehensive survey of potential game approaches to wireless networks. IEICE Trans. Commun. E98 B(9), 1804–1823 (2015)
J.R. Marden, S.D. Ruben, L.Y. Pao, A model-free approach to wind farm control using game theoretic methods. IEEE Trans. Control Syst. Technol. 21(4), 1207–1214 (2013)
Q. Zhu, T. Basar, Game-theoretic methods for robustness, security, and resilience of cyberphysical control systems. IEEE Control Syst. Mag. 35(1), 46–65 (2015)
Y. Liang, F. Liu, S. Mei, Distributed real-time economic dispatch in smart grids: a state-based potential game approach. IEEE Trans. Smart Grids 21(4), 1207–1214 (2018)
Y. Zhao, S. Wang, T.C.E. Cheng, X. Yang, Z. Huang, Coordination of supply chains by option contracts: a cooperative game theory approach. Eur. J. Oper. Res. 207(2), 668–675 (2010)
N. Li, D.W. Oyler, M. Zhang, Y. Yildiz, I. Kolmanovsky, A.R. Girard, Game theoretic modeling of driver and vehicle interactions for verification and validation of autonomous vehicle control systems. IEEE Trans. Control Syst. Technol. 21(4), 1207–1214 (2019)
G. Zhou, P. Jiang, G.Q. Huang, A game-theory approach for job scheduling in networked manufacturing. Int. J. Adv. Manufact. Technol. 41(9–10), 972–985 (2009)
A. Schwung, A. Elbel, D. Schwung, System reconfiguration of modular production units using a SOA-based control structure, in Proceedings of the 15th International Conference on Industrial Informatics (INDIN 2017), Emden, Germany (2017)
D. Schwung, J.N. Reimann, A. Schwung, S.X. Ding, Self learning in flexible manufacturing units: a reinforcement learning approach, in Proceedings of the 9th International Conference on Intelligent Systems, Funchal, Portugal (2018)
R.S. Sutton, A.G. Barto, Reinforcement Learning: An Introduction (MIT Press, Cambridge, 1998)
V. Mnih, A.P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, K. Kavukcuoglu, Asynchronous methods for deep reinforcement learning, in Proceedings of The 33rd International Conference on Machine Learning, PMLR, 48 (2016), pp. 1928–1937
J.R. Marden, State based potential games. Automatica 48, 3075–3088 (2012)
J.R. Marden, J.S. Shamma, Revisiting log-linear learning: asynchrony, completeness and payoff-based implementation. Games Econ. Behav. 75, 788–808 (2012)
C. Lemke, M. Budka, B. Gabrys, Metalearning: a survey of trends and technologies. Artif. Intell. Rev. 44(1), 117–130 (2015)
Programmable controllers part 3: programming languages. International Standard IEC 61131-3, 2nd edn. (2003)
Function blocks, International Standard IEC 61499, 1st edn. (2005)
T. Cucinotta, A. Mancina, G.F. Anastasi, G. Lipari, L. Mangeruca, R. Checcozzo, F. Rusina, A real-time service-oriented architecture for industrial automation. IEEE Trans. Ind. Inform. 5(3), 267–277 (2009)
W. Dai, V. Vyatkin, J.H. Christensen, V.N. Dubinin, Bridging service-oriented architecture and IEC 61499 for flexibility and interoperability. IEEE Trans. Ind. Inf. 11(3), 771–781 (2015)
A. Zoitl, T. Strasser, C. Sünder, T. Baier, Is IEC 61499 in harmony with IEC 61131–3? IEEE Ind. Electron. Mag. 3(4), 49–55 (2009)
D. Schwung, T. Kempe, A. Schwung, Self-optimization of energy consumption in complex bulk good processes using reinforcement learning, in Proceedings of the 15th International Conference on Industrial Informatics (INDIN 2017), Emden, Germany (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Schwung, D., Reimann, J.N., Schwung, A., Ding, S.X. (2020). Smart Manufacturing Systems: A Game Theory based Approach. In: Jardim-Goncalves, R., Sgurev, V., Jotsov, V., Kacprzyk, J. (eds) Intelligent Systems: Theory, Research and Innovation in Applications. Studies in Computational Intelligence, vol 864. Springer, Cham. https://doi.org/10.1007/978-3-030-38704-4_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-38704-4_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-38703-7
Online ISBN: 978-3-030-38704-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)