A Framework for the Evaluation of Trainee Performance in Cyber Range Exercises

  • Mauro AndreoliniEmail author
  • Vincenzo Giuseppe Colacino
  • Michele Colajanni
  • Mirco Marchetti


This paper proposes a novel approach for the evaluation of the performance achieved by trainees involved in cyber security exercises implemented in modern cyber ranges. Our main contributions include: the definition of a distributed monitoring architecture for gathering relevant information about trainees activities; an algorithm for modeling the trainee activities using directed graphs; novel scoring algorithms, based on graph operations, that evaluate different aspects (speed, precision) of a trainee during an exercise. With respect to previous work, our proposal allows to measure exactly how fast a user is progressing towards an objective and where he does wrong. We highlight that this is currently not possible in the most popular cyber ranges.


Cyber range Cyber exercise Graph algorithms Monitoring framework 



  1. 1.
    Bagnato A, Kordy B, Meland PH, Schweitzer P (2012) Attribute decoration of attack–defense trees. Int J Secur Softw Eng (IJSSE) 3(2):1–35CrossRefGoogle Scholar
  2. 2.
    Bowen BM, Devarajan R, Stolfo S (2011) Measuring the human factor of cyber security. In: 2011 IEEE International conference on technologies for homeland security (HST). IEEE, pp 230–235Google Scholar
  3. 3.
    Carlisle M, Chiaramonte M, Caswell D (2015) Using ctfs for an undergraduate cyber education. In: 2015 {USENIX} summit on gaming, games, and gamification in security education (3GSE 15)Google Scholar
  4. 4.
    Čeleda P, Čegan J, Vykopal J, Tovarňák D (2015) Kypo–a platform for cyber defence exercises. M&S Support to Operational Tasks Including War Gaming, Logistics,Cyber Defence. NATO Science and Technology OrganizationGoogle Scholar
  5. 5.
  6. 6.
    Evans M, He Y, Maglaras L, Janicke H (2019) Heart-is: a novel technique for evaluating human error-related information security incidents. Comput Secur 80:74–89CrossRefGoogle Scholar
  7. 7.
    Ferguson B, Tall A, Olsen D (2014) National cyber range overview. In: 2014 IEEE Military communications conference. IEEE, pp 123–128Google Scholar
  8. 8.
    Huang K, Siegel M, Stuart M (2018) Systematically understanding the cyber attack business: a survey. ACM Comput Surv (CSUR) 51(4):70CrossRefGoogle Scholar
  9. 9.
  10. 10.
    Jameel A, Shahzad K, Zafar A, Ahmed U, Hussain SJ, Sajid A (2018) The users experience quality of responsive web design on multiple devices. In: Proceedings of the 2nd international conference on future networks and distributed systems. ACM , p 69Google Scholar
  11. 11.
    Kordy B, Kordy P, Mauw S, Schweitzer P (2013) Adtool: security analysis with attack–defense trees. In: International conference on quantitative evaluation of systems. Springer, pp 173–176Google Scholar
  12. 12.
    Kraemer S, Carayon P, Clem J (2009) Human and organizational factors in computer and information security: pathways to vulnerabilities. Comput Secur 28(7):509–520CrossRefGoogle Scholar
  13. 13.
    Lampesberger H (2016) Technologies for web and cloud service interaction: a survey. SOCA 10(2):71–110CrossRefGoogle Scholar
  14. 14.
    Mauw S, Oostdijk M (2005) Foundations of attack trees. In: International conference on information security and cryptology. Springer, pp 186–198Google Scholar
  15. 15.
    Miehling E, Rasouli M, Teneketzis D (2015) Optimal defense policies for partially observable spreading processes on bayesian attack graphs. In: Proceedings of the second ACM workshop on moving target defense. ACM, pp 67–76Google Scholar
  16. 16.
    Ou X, Boyer WF, McQueen MA (2006) A scalable approach to attack graph generation. In: Proceedings of the 13th ACM conference on computer and communications security. ACM, pp 336–345Google Scholar
  17. 17.
    Pernik P (2014) Improving cyber security: Nato and the eu International Centre for Defense StudiesGoogle Scholar
  18. 18.
    Poolsappasit N, Dewri R, Ray I (2011) Dynamic security risk management using Bayesian attack graphs. IEEE Trans Depend Secur Comput 9(1):61–74CrossRefGoogle Scholar
  19. 19.
    Schneier B (1999) Attack trees. Dr Dobb’s J 24(12):21–29Google Scholar
  20. 20.
    Sheyner O, Haines J, Jha S, Lippmann R, Wing JM (2002) Automated generation and analysis of attack graphs. In: Proceedings 2002 IEEE symposium on security and privacy. IEEE , pp 273–284Google Scholar
  21. 21.
    Vykopal J, Vizváry M, Oslejsek R, Celeda P, Tovarnak D (2017) Lessons learned from complex hands-on defence exercises in a cyber range. In: 2017 IEEE frontiers in education conference (FIE). IEEE, pp 1–8Google Scholar
  22. 22.
    Zonouz SA, Khurana H, Sanders WH, Yardley TM (2013) Rre: a game-theoretic intrusion response and recovery engine. IEEE Trans Parallel Distrib Syst 25(2):395–406CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Mauro Andreolini
    • 1
    Email author
  • Vincenzo Giuseppe Colacino
    • 1
  • Michele Colajanni
    • 1
  • Mirco Marchetti
    • 1
  1. 1.University of Modena and Reggio EmiliaModenaItaly

Personalised recommendations