Applying Complexity Science with Machine Learning, Agent-Based Models, and Game Engines: Towards Embodied Complex Systems Engineering

  • Michael D. NormanEmail author
  • Matthew T. K. Koehler
  • Jason F. Kutarnia
  • Paul E. Silvey
  • Andreas Tolk
  • Brittany A. Tracy
Conference paper
Part of the Springer Proceedings in Complexity book series (SPCOM)


The application of Complexity Science, an undertaking referred to here as Complex Systems Engineering, often presents challenges in the form of agent-based policy development for bottom-up complex adaptive system design and simulation. Determining the policies that agents must follow in order to participate in an emergent property or function that is not pathological in nature is often an intensive, manual process. Here we will examine a novel path to agent policy development in which we do not manually craft the policies, but allow them to emerge through the application of machine learning within a game engine environment. The utilization of a game engine as an agent-based modeling platform provides a novel mechanism to develop and study intelligent agent-based systems that can be experienced and interacted with from multiple perspectives by a learning agent. In this paper we present results from an example use-case and discuss next steps for research in this area.


Artificial intelligence Complexity Emergence Reinforcement learning 


Acknowledgements and Disclaimer

The work presented in this paper was partly supported by the MITRE Innovation Program. The authors wish to specifically acknowledge Sham Chakravorty and Muhammad Sungkar for their contributions to this effort. The views, opinions, and/or findings contained in this paper are those of The MITRE Corporation and should not be construed as an official government position, policy, or decision, unless designated by other documentation. It is approved for Public Release; Distribution Unlimited. Case Number 17-3081-17.


  1. 1.
  2. 2.
    Diallo, S., Mittal, S., Tolk, A.: Research agenda for next-generation complex systems engineering. In: Emergent Behavior in Complex Systems Engineering: A Modeling and Simulation Approach, pp. 379–397 (2018)Google Scholar
  3. 3.
  4. 4.
    Holland, J.H., Miller, J.H.: Artificial adaptive agents in economic theory. Am. Econ. Rev. 81(2), 365–370 (1991)Google Scholar
  5. 5.
  6. 6.
    Kelso, J.S.: Dynamic Patterns: The Self-organization of Brain and Behavior. MIT Press, Cambridge (1997)Google Scholar
  7. 7.
    Liebovitch, L.S., Peluso, P.R., Norman, M.D., Su, J., Gottman, J.M.: Mathematical model of the dynamics of psychotherapy. Cogn. Neurodyn. 5(3), 265–275 (2011)CrossRefGoogle Scholar
  8. 8.
    Norman, M.D.: Complex systems engineering in a federal it environment: lessons learned from traditional enterprise-scale system design and change. In: 2015 9th Annual IEEE International Systems Conference (SysCon), pp. 33–36. IEEE (2015)Google Scholar
  9. 9.
    Norman, M.D., Koehler, M.T., Pitsko, R.: Applied complexity science: enabling emergence through heuristics and simulations. In: Emergent Behavior in Complex Systems Engineering: A Modeling and Simulation Approach, pp. 201–226 (2018)Google Scholar
  10. 10.
  11. 11.
    Reynolds, C.W.: Flocks, herds and schools: a distributed behavioral model. In: ACM SIGGRAPH Computer Graphics, vol. 21, pp. 25–34. ACM (1987)Google Scholar
  12. 12.
    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal Policy Optimization Algorithms. ArXiv e-prints (2017)Google Scholar
  13. 13.
    Sheard, S., Cook, S., Honour, E., Hybertson, D., Krupa, J., McEver, J., McKinney, D., Ondrus, P., Ryan, A., Scheurer, R., et al.: A complexity primer for systems engineers. INCOSE Complex Systems Working Group White Paper (2015)Google Scholar
  14. 14.
    Silvey, P.E., Norman, M.D.: Embodied cognition and multi-agent behavioral emergence. In: Proceedings of the Ninth International Conference on Complex Systems (ICCS 2018) (2018, in press)Google Scholar
  15. 15.
    Strogatz, S.: Sync: The Emerging Science of Spontaneous Order. Penguin, London (2004)Google Scholar
  16. 16.
    Taleb, N.N.: Antifragile: Things That Gain from Disorder, vol. 3. Random House Incorporated (2012)Google Scholar
  17. 17.
    Tampuu, A., Matiisen, T., Kodelja, D., Kuzovkin, I., Korjus, K., Aru, J., Aru, J., Vicente, R.: Multiagent cooperation and competition with deep reinforcement learning. PLoS ONE 12(4), e0172395 (2017)CrossRefGoogle Scholar
  18. 18.
  19. 19.
    Tolk, A.: The next generation of modeling & simulation: integrating big data and deep learning. In: Proceedings of the Conference on Summer Computer Simulation, pp. 1–8. Society for Computer Simulation International (2015)Google Scholar
  20. 20.
    Tolk, A., Diallo, S., Mittal, S.: Complex systems engineering and the challenge of emergence. In: Emergent Behavior in Complex Systems Engineering: A Modeling and Simulation Approach, pp. 79–97 (2018)Google Scholar
  21. 21.
    Tolk, A., Rainey, L.B.: Toward a research agenda for m&s support of system of systems engineering. In: Modeling and Simulation Support for System of Systems Engineering Applications, pp. 581–592 (2015)Google Scholar
  22. 22.
  23. 23.
  24. 24.
    Xu, J.: Model calibration. In: Advances in Modeling and Simulation: Seminal research from 50 Years of Winter Simulation Conferences, pp. 27–46 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Michael D. Norman
    • 1
    Email author
  • Matthew T. K. Koehler
    • 1
  • Jason F. Kutarnia
    • 1
  • Paul E. Silvey
    • 1
  • Andreas Tolk
    • 1
  • Brittany A. Tracy
    • 1
  1. 1.The MITRE CorporationBedfordUSA

Personalised recommendations