Abstract
M&S of systems, their dynamic structures and particularly the behaviour of internal component objects, should be performed at the level of detail which is adequate to the problem and modelling purpose defined. In the scenarios related to the complex world strictly one level is insufficient - there is necessary to build a multi-resolution model that represents structures and actions at different levels of detail. This is the main and direct reason for the application of the Multi-resolution Agent Model (MrAM) approach in the simulation with functions for a state’s transformation (aggregation/disaggregation). It is common practice to implement methods of resolution adaptation in such a way that they are completely closed in the compiled program code. Meanwhile, the multiplicity of different possible scenarios regarding group and individual behaviours indicates that there are necessary software constructions enabling the end user to create both new, open models of behaviours and algorithms for aggregation/disaggregation of the state.
Moreover, the environment surrounding agents influences target states differently at different moments in time. The article proposes an approach to determining the consensus state of agents with the use of machine learning methods.
The consensus between agents according to the appropriate approach, depending on the conditions and state, will be a generalized method adaptable to the environment of agents. We propose the reinforcement learning model as a multiagent game in order to achieve HRE state and thus complete disaggregation.
To meet the requirements, the original Java-based framework for hybrid simulation (discrete, event-based and continuous) with the ability to model the object as an agent at multiple levels, automatic triggering of updates on all modelled levels, and Groovy-based scripts. The scripting technology is integrated with the standalone Java software and enables the implementation of behaviors and state transformations that are really open in scripts.
The article presents the proposed framework solution on the example of an autonomous system model composed of many cooperating objects. We share our experiences related to the extension of the “SymSG Border Tactics” simulation environment dedicated to CAX exercises in The Poland Border Guard.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In this paper return and reward will be used interchangeably.
References
Chmielewski, M., Kukiełka, M., Frąszczak, D., Bugajewski, D.: The architectural software concepts implemented in distributed high resolution constructive simulation environment SymSG Border Tactics, supporting Polish Border Guard computer assisted exercises. In: 2017 Fourth International Conference on Mathematics and Computers in Sciences and in Industry (MCSI) (2017). https://doi.org/10.1109/MCSI.2017.50. ISBN 978-1-5386-2820-1
Natrajan, A., Reynolds, P.F., Srinivasan, S.: Guidelines for the Design of Multi-resolution Simulations. US DoD DMSO (1997)
Wei, S., Cai, B.G., Li, S.H., Liu, Z.G., Wang, J.: Multi-resolution simulation strategy and its simulation implementation of Train Control System. IEEE (2011). 978-1-4577-0574-8/11
Skrzypecki, S., Pierzchała, D., Tarapata, Z.: Distributed simulation environment of unmanned aerial systems for a search problem. In: Mazal, J. (ed.) MESAS 2018. LNCS, vol. 11472, pp. 65–81. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-14984-0_6
Pierzchała, D.: Application of ontology and rough set theory to information sharing in multi-resolution combat M&S. In: Sobecki, J., Boonjing, V., Chittayasothorn, S. (eds.) Advanced Approaches to Intelligent Information and Database Systems. SCI, vol. 551, pp. 193–203. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-05503-9_19
Pierzchała, D., Czuba, P.: Method of agents’ state estimation in multiresolution multiagent simulation. Comput. Sci. Math. Model. 8, 29–39 (2018). ISSN 2450-0054
Lin, Z., Francis, B., Maggiore, M.: Feasibility for formation stabilization of multiple unicycles. In: 2004 43rd IEEE Conference on Decision and Control (2004). https://doi.org/10.1109/CDC.2004.1428776. ISBN 0-7803-8682-5
Hüttenrauch, M., Šošic, A., Neumann, G.: Deep reinforcement learning for swarm systems. J. Mach. Learn. Res. 20(54), 1–31 (2019)
François-Lavet, V., Henderson, P., Islam, R., Bellemare, M.G., Pineau, J.: An introduction to deep reinforcement learning. Found. Trends Mach. Learn. 11(3–4), 219–354 (2018)
Bellman, R.E., Dreyfus, S.E.: Applied Dynamic Programming. Princeton University Press, Princeton (1962). ISBN 10: 0691079137
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Lin, L.-J.: Self-improving reactive agents based on reinforcement learning, planning and teaching. Mach. Learn. 8(3–4), 293–321 (1992)
Nguyen, T.T., Nguyen, N.D., Nahavandi, S.: Deep reinforcement learning for multi-agent systems: a review of challenges, solutions and applications (2018)
Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., Mordatch, I.: Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments (2017)
Pierzchała, D., Najgebauer, A.: Simulator for analysis cyber threats to RFID based system. In: MATEC Web Conference, vol. 210 (2018). https://doi.org/10.1051/matecconf/201821004022. Published online 05 October 2018
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Pierzchała, D., Czuba, P. (2020). Machine Learning-Based Open Framework for Multiresolution Multiagent Simulation. In: Mazal, J., Fagiolini, A., Vasik, P. (eds) Modelling and Simulation for Autonomous Systems. MESAS 2019. Lecture Notes in Computer Science(), vol 11995. Springer, Cham. https://doi.org/10.1007/978-3-030-43890-6_17
Download citation
DOI: https://doi.org/10.1007/978-3-030-43890-6_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-43889-0
Online ISBN: 978-3-030-43890-6
eBook Packages: Computer ScienceComputer Science (R0)