Applying Model-Based Situational Awareness and Augmented Reality to Next-Generation Physical Security Systems

  • Elaine M. RaybournEmail author
  • Ray Trechter


Mixed, augmented, and virtual reality holds promise for many security-related applications including physical security systems. When combined with models of a site, an augmented reality (AR) approach can be designed to enhance knowledge and understanding of the status of the facility. The present chapter describes how improved modeling and simulation will increase situational awareness by blurring the lines among the use of tools for analysis, rehearsal, and training—especially when coupled with immersive interaction experiences offered by augmented reality. We demonstrate how the notion of a digital twin can blur these lines. We conclude with challenges that must be overcome when applying digital twins, advanced modeling, and augmented reality to the design and development of next-generation physical security systems.


Augmented reality Digital twin Modeling Simulation Physical security systems Situational awareness Next generation 



Sandia National Laboratories is a multimission laboratory managed and operated by the National Technology and Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International, Inc., for the US Department of Energy’s National Nuclear Security Administration under contract DE-NA-0003525.


  1. 1.
    R. Trechter, Physical security simulation and analysis tools. SAND 2014-3718P (Sandia National Laboratories, Albuquerque, NM, 2014). Accessed 17 Apr 2018
  2. 2.
    D. Callow, SAND2016–12214 physical security system of the future: vision and roadmap (Sandia National Labs, 2016)Google Scholar
  3. 3.
    M. W. Grieves, Digital twin: Manufacturing excellence through virtual factory replication. Digital Twin whitepaper. Michael W. Grieves, LLC (2014). Accessed 17 Apr
  4. 4.
    General Electric Research, Predix technology brief: Digital Twin (2017). Accessed 17 Apr 2018
  5. 5.
    E.M. Raybourn, A new paradigm for serious games: Transmedia learning for more effective training & education. J. Comput. Sci. 5(3), 471–481 (2014)CrossRefGoogle Scholar
  6. 6.
    E.M. Raybourn, Applying simulation experience design methods to creating serious game-based adaptive training systems. Interact. Comput. 19, 207–214 (2007)CrossRefGoogle Scholar
  7. 7.
    J. Clem, W. Atkins, V. Urias, Investigation of cyber-enabled physical attack scenarios. SAND2015-4202C (Sandia National Laboratories, Albuquerque, NM, 2015). Accessed 6 Mar 2018
  8. 8.
    E.M. Raybourn, in A metaphor for immersive environments: Learning experience design challenges and opportunities. Proceedings of MODSIM. (NTSA, Arlington, VA, 2016)Google Scholar
  9. 9.
    M.L. Garcia, The Design and Evaluation of Physical Protection Systems, 2nd edn. (Butterworth-Heinemann, Newton, MA, 2007)Google Scholar
  10. 10.
    K. Klinger, D. Small, E. Gottlieb, J. Whetzel, H. Gillis, J. Wharton, Final report for advanced high security command and control interface LDRD (AHSC2I). SAND2013–8249 (Sandia National Laboratories, Albuquerque, NM, 2013)Google Scholar
  11. 11.
    L. Dignan, GE aims to replicate digital twin success with security-focused digital ghost. Between the lines, (2017). Accessed 17 Apr 2018
  12. 12.
    J. Russel, J. Andersen., C. Sterns, Video motion detector fused radar: the first volumetric ultra-low NAR sensor for exterior environments. SAND2016–0083 (Sandia National Laboratories, Albuquerque, NM, 2016)Google Scholar
  13. 13.
    A. Greenberg, How an entire nation became Russia’s test lab for cyberwar. Wired (2017, June). Accessed 6 Mar 2018
  14. 14.
    ICS-CERT, Cyber attack against Ukrainian critical infrastructure (2016). Accessed 6 Mar 2018
  15. 15.
    R. Lipovski, A. Cherepanov, BlackEnergy trojan strikes again: attacks Ukrainian electric power industry. WeLiveSecurity (2016, Jan). Accessed 6 Mar 2018
  16. 16.
    B. Gregory-Brown, D. Wylie, Securing industrial control systems – 2017: A SANS survey (SANS Institute InfoSec Reading Room, 2017) Accessed 6 Mar 2018
  17. 17.
    J. Stamp, J. Dillinger, W. Young, J. Depoy, Common vulnerabilities in critical infrastructure control systems. SAND2003-1772C (Sandia National Laboratories, Albuquerque, NM, 2003)Google Scholar
  18. 18.
    K. J. Higgins, Latest Ukraine blackout tied to 2015 cyberattackers. Dark Reading (2017, Jan). Accessed 6 Mar 2018
  19. 19.
    Y. Yang, K. McLaughlin, S. Sezer, T. Littler, E.G. Im, B. Pranggono, H.F. Wang, Multiattribute SCADA-specific intrusion detection system for power networks. IEEE Trans. Power Deliv. 29(3), 1092–1102 (2014)CrossRefGoogle Scholar
  20. 20.
    E. M. Raybourn, N. Fabian, W. Davis, R. C. Parks, J. McClain, D. Trumbo, D. Regan., P. Durlach, Data privacy and security considerations for personal assistants for learning (PAL). Proceedings of the 20th International Conference on Intelligent User Interfaces Companion (2015), pp. 69–72Google Scholar
  21. 21.
    R. Walton, National academies report finds grid vulnerable to cyber physical attacks. UtilityDive (2017, July 24). Accessed 6 Mar 2018
  22. 22.
    V. Machi, The future of training and simulation: preparing warfighters for tomorrow’s battlefields. National Defense Magazine (2017, Nov). Accessed 29 Apr 2018
  23. 24.
    E. Hutchins, Cognition in the Wild (The MIT Press, Cambridge, MA, 1995)Google Scholar
  24. 25.
    Paul, Second Ukraine power outage linked to Russian hackers. The Security Ledger (2017, Jan). Accessed 6 Mar 2018

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Sandia National LaboratoriesAlbuquerqueUSA

Personalised recommendations