Abstract
Discussion of challenges and ways of improving Cyber Situational Awareness dominated our previous chapters. However, we have not yet touched on how to quantify any improvement we might achieve. Indeed, to get an accurate assessment of network security and provide sufficient Cyber Situational Awareness (CSA), simple but meaningful metrics—the focus of the Metrics of Security chapter—are necessary. The adage, “what can’t be measured can’t be effectively managed,” applies here. Without good metrics and the corresponding evaluation methods, security analysts and network operators cannot accurately evaluate and measure the security status of their networks and the success of their operations. In particular, this chapter explores two distinct issues: (i) how to define and use metrics as quantitative characteristics to represent the security state of a network, and (ii) how to define and use metrics to measure CSA from a defender’s point of view.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alberts C., et al. (2005). Mission Assurance Analysis Protocol (MAAP): Assessing Risk in Complex Environments. CMU/SEI-2005-TN-032. Pittsburgh, PA: Carnegie Mellon University.
Ammann P., et al. (2002). Scalable, Graph-based Network Vulnerability Analysis. the 9th ACM Conference on Computer and Communications Security.
Bolstad C. and Cuevas H. (2010). Integrating Situation Awareness Assessment into Test and Evaluation. The International Test and Evaluation Association (ITEA), 31: 240–246.
Cheung S., et al. (2003). Modeling Multi-Step Cyber Attacks for Scenario Recognition. the 3rd DARPA Information Survivability Conference and Exhibition. Washington D. C.
Dahl, O. (2005). Using colored petri nets in penetration testing. Master’s thesis. Gjøvik, Norway: Gjøvik University College.
Durso F., et al. (1995). Expertise and chess: A pilot study comparing situation awareness methodologies. In experimental analysis and measurement of situation awareness, (pp. 295–303).
Endsley, M. R. (1988). Situation awareness global assessment technique (SAGAT). the National Aerospace and Electronics Conference (NAECON).
Endsley, M. R. (1990). Predictive utility of an objective measure of situation awareness. the Human Factors Society 34th Annual Meeting, (pp. 41–45).
Endsley, M. R. (1995). Measurement of situation awareness in dynamic systems. Human Factors, 37(1), 65–84.
Endsley, M. R., et al. (1998). A comparative evaluation of SAGAT and SART for evaluations of situation awareness. the Human Factors and Ergonomics Society Annual Meeting, (pp. 82–86).
Fracker, M. (1991a). Measures of situation awareness: Review and future directions (Report No. AL-TR-1991-0128). Wright-Patterson Air Force Base, OH: Armstrong Laboratories.
Fracker, M. (1991b). Measures of situation awareness: An experimental evaluation (Report No. AL-TR-1991-0127). Wright-Patterson Air Force Base, OH: Armstrong Laboratories.
Gomez M., et al. (2008). An Ontology-Centric Approach to Sensor-Mission Assignment. Springer.
Goodall J., et al. (2009). Camus: Automatically Mapping Cyber Assets to Missions and Users. IEEE Military Communications Conference. Boston MA.
Grimaila M., et al. (2008). Improving the Cyber Incident Mission Impact Assessment Processes. the 4th Annual Workshop on Cyber Security and Information Intelligence Research.
Grimaila M., et al. (2009). Design Considerations for a Cyber Incident Mission Impact Assessment (CIMIA) Process. the 2009 International Conference on Security and Management (SAM09). Las Vegas, Nevada.
Harwood K., et al. (1988). Situational awareness: A conceptual and methodological framework. the 11th Biennial Psychology in the Department of Defense Symposium, (pp. pp. 23–27).
Hecker, A. (2008). On System Security Metrics and the Definition Approaches. the 2nd International Conference on Emerging Security Information, Systems and Technologies.
Heyman T., et al. (2008). Using security patterns to combine security metrics. the 3rd International Conference on Availability, Reliability and Security.
Holsopple J., et al. (2008). FuSIA: Future Situation and Impact Awareness. Information Fusion.
Jakobson G. (2011). Mission Cyber Security Situation Assessment Using Impact Dependency Graphs. the 14th International Conference on Information Fusion (FUSION) (pp. 1–8). Chicago, IL: IEEE.
Jansen, W. (2009). Directions in Security Metrics Research. National Institute of Standards and Technology, Computer Security Division.
Jones D. and Endsley M. R. (2000). Examining the validity of real-time probes as a metric of situation awareness. the 14th Triennial Congress of the International Ergonomics Association.
Kotenko I., et al. (2006). Attack graph based evaluation of network security. the 10th IFIP TC-6 TC-11 international conference on Communications and Multimedia Security, (pp. 216–227).
Lewis L., et al. (2008). Enabling Cyber Situation Awareness, Impact Assessment, and Situation Projection. Situation Management (SIMA).
Lindstrom, P. (2005). Security: Measuring Up. Retrieved from http://searchsecurity.techtarget.com/tip/Security-Measuring-Up
Manadhata P. and Wing J. (2011). An Attack Surface Metric. Software Engineering, IEEE Transactions on, vol. 37, no. 3, pp. 371–386.
Matthews M., et al. (2000). Measures of infantry situation awareness for a virtual MOUT environment. the Human Performance, Situation Awareness and Automation: User-Centered Design for the New Millennium.
McDermott, J. (2000). Attack net penetration testing. Workshop on New Security Paradigms.
Meland P. and Jensen J. (2008). Secure Software Design in Practice. the 3rd International Conference on Availability, Reliability and Security.
Musman S., et al. (2010). Evaluating the Impact of Cyber Attacks on Missions. MITRE Technical Paper #09-4577.
Natarajan A., et al. (2012). NSDMiner: Automated discovery of network service dependencies. INFOCOM (pp. 2507–2515). IEEE.
Nebel B., et al. (1995). Reasoning about temporal relations: a maximal tractable subclass of Allen's interval algebra. Journal of the ACM (JACM), vol. 42, no. 1, pp. 43–66.
Noel S., et al. (2004). Correlating Intrusion Events and Building Attack Scenarios through Attack Graph Distance. the 20th Annual Computer Security Conference. Tucson, Arizona.
Ou X., et al. (2006). A Scalable Approach to Attack Graph Generation. the 13th ACM Conference on Computer and Communication Security (CCS), (pp. 336–345).
Qin X. and Lee W. (2004). Attack Plan Recognition and prediction Using Causal Networks. the 20th Annual Computer Security Applications Conference.
Salerno J., et al. (2005). A Situation Awareness Model Applied to Multiple Domains. Multisensor, Multisource Information Fusion.
Salerno, J. (2008). Measuring situation assessment performance through the activities of interest score. the 11th International Conference on Information Fusion.
Sheyner O., et al. (2002). Automated Generation and Analysis of Attack Graphs. the 2002 IEEE Symposium on Security and Privacy, (pp. 254–265).
Singhal A., et al. (2010). Ontologies for modeling enterprise level security metrics. the 6th Annual Workshop on Cyber Security and Information Intelligence Research. ACM.
Strater L., et al. (2001). Measures of platoon leader situation awareness in virtual decision making exercises (No. Research Report 1770). Army Research Institute.
Tadda G., et al. (2006). Realizing Situation Awareness within a Cyber Environment. Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications (p. 1–8). Orlando: SPIE Vol.6242.
Taylor, R. (1989). Situational awareness rating technique (SART): The development of a tool for aircrew systems design. the AGARD AMP Symposium on Situational Awareness in Aerospace Operations, CP478.
Tu W., et. al. (2009). Automated Service Discovery for Enterprise Network Management. Stony Brook University. Retrieved May 8, 2014, from http://www.cs.sunysb.edu/~live3/research/asd_ppt.pdf
Vidulich M. (2000). Testing the sensitivity of situation awareness metrics in interface evaluations. Situation awareness analysis and measurement, 227–246.
Wang J., et al. (2009). Security Metrics for Software Systems. the 47th Annual Southeast Regional Conference.
Watters J., et al. (2009). The Risk-to-Mission Assessment Process (RiskMAP): A Sensitivity Analysis and an Extension to Treat Confidentiality Issues.
Zhou S., et al. (2003). Colored petri net based attack modeling. Rough Sets, Fuzzy Sets, Data Mining, and Granular Computing: the 9th International Conference (pp. vol. 2639, pp. 715–718). Chongqing, China: Springer.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Cheng, Y., Deng, J., Li, J., DeLoach, S.A., Singhal, A., Ou, X. (2014). Metrics of Security. In: Kott, A., Wang, C., Erbacher, R. (eds) Cyber Defense and Situational Awareness. Advances in Information Security, vol 62. Springer, Cham. https://doi.org/10.1007/978-3-319-11391-3_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-11391-3_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11390-6
Online ISBN: 978-3-319-11391-3
eBook Packages: Computer ScienceComputer Science (R0)