Abstract
For years, the research in intrusion detection field has been primarily focused on anomaly and misuse detection techniques. The latter method is traditionally favored in commercial products due to its predictability and high accuracy. In academic research, however, anomaly detection approach is perceived as a more powerful due to its theoretically higher potential to address novel attacks in comparison to misuse based methods. While academic community proposed a wide spectrum of anomaly based intrusion techniques, adequate comparison of the strengths and limitations of these techniques that can lead to potential commercial application is challenging. In this chapter we introduce the most significant criteria which have been proposed to have a more realistic evaluation of anomaly detection systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
MIT Lincoln Labs, 1998 DARPA Intrusion Detection Evaluation. Available on: http://www.ll.mit.edu/mission/communications/ist/corpora/ideval/index.html, February 2008.
KDD Cup 1999. Available on: http://kdd.ics.uci.edu/databases/kddcup 99/kddcup99.html, October 2007.
Nsl-kdd data set for network-based intrusion detection systems, Available on: http://iscx.cs.unb.ca/NSL-KDD/, March 2009.
Stefan Axelsson, The base-rate fallacy and its implications for the difficulty of intrusion detection, Proceedings of the 6th ACM conference on Computer and communication security (Kent Ridge Digital Labs, Singapore), ACM Press, November 1999, pp. 1–7.
——, The base-rate fallacy and the difficulty of intrusion detection, ACM Transactions on Information and System Security (TISSEC) 3 (2000), no. 3, 186–205.
P. Dokas, L. Ertoz, V. Kumar, A. Lazarevic, J. Srivastava, and P. Tan, Data mining for network intrusion detection, Proceedings of NSF Workshop on Next Generation Data Mining (Baltimore, MD), November 2002.
Mahesh V. Joshi, Ramesh C. Agarwal, and Vipin Kumar, Predicting rare classes: Can boosting make any weak lerner strong?, Proceedings of the SIG KDD (Edmonton, Alberta, Canada), 2002.
C. Kruegel, F. Valeur, G. Vigna, and R.A. Kemmerer, Stateful intrusion detection for high-speed networks, Proceedings of the IEEE Symposium on Security and Privacy (Oakland, CA), IEEE Press, May 2002, pp. 285–293.
W. Lee, W. Fan, M. Miller, s. Stolfo, and E. Zadok, Toward cost sensitive modeling for intrusion detection and response, Journal of Computer Security 10 (2002), no. 1,2, 5–22.
Wenke Lee, Joo B.D. Cabrera, Ashley Thomas, Niranjan Balwalli, Sunmeet Saluja, and Yi Zhang, Performance adaptation in real-time intrusion detection systems, Proceedings of Recent Advances in Intrusion Detection, 5th International Symposium, (RAID 2002) (Zurich, Switzerland) (A. Wespi, G. Vigna, and L. Deri, eds.), Lecture Notes in Computer Science, Springer-Verlag Heidelberg, October 2002, pp. 252–273.
R. Lippmann, D. Fried, I. Graf, J. Haines, K. Kendall, D. McClung, D. Weber, S. Webster, D. Wyschogrod, R. Cunningham, and M. Zissman, Evaluating intrusion detection systems: The 1998 darpa off-line intrusion detection evaluation, Proceedings of the 2000 DARPA Information Survivability Conference and Exposition (DISCEX-00), 2000, pp. 12–26.
M.V. Mahoney and P.K. Chan, An Analysis of the 1999 DARPA/Lincoln Laboratory Evaluation Data for Network Anomaly Detection, LECTURE NOTES IN COMPUTER SCIENCE (2003), 220–238.
John McHugh, Testing intrusion detection systems: a critique of the 1998 and 1999 darpa intrusion detection system evaluations as performed by lincoln laboratory, ACM Transactions on Information and System Security (TISSEC) 3 (2000), no. 4, 262–294.
NIST, Technology ITL, MITLL, Peter Mell, Vincent Hu, Richard Lippmann, Josh Haines, and Marc Zissman, An overview of issues in testing intrusion detection, July 2003.
Bryan Pfaffenberger, Webster's new world dictionary, ninth edition ed., ch. AC-3, p. 9, Hungry Minds, 2001.
Foster Provost and Tom Fawcett, Robust classification for imprecise environments, Machine Learning 42 (2001), no. 3, 203–231.
T. Ptacek and T.Newsham, Insertion, evasion, and denial of service: Eluding network intrusion detection, 1998.
Stuart Russell and Peter Norving, Artificial intelligence a modern approach, second edition ed., ch. Uncertainty, pp. 462–491, Prentice Hall, 2003.
——, Artificial intelligence a modern approach, second edition ed., ch. Probabilistic Reasoning, pp. 492–536, Prentice Hall, 2003.
Lambert Schaelicke, Thomas Slabach, Branden Moore, and Curt Freeland, Characterizing the performance of network intrusion detection sensors, Proceedings of Recent Advances in Intrusion Detection, 6th International Symposium, (RAID 2003) (Pittsburgh, PA, USA) (G. Vigna, E. Jonsson, and C. Kruegel, eds.), Lecture Notes in Computer Science, Springer-Verlag Heidelberg, September 2003, pp. 155–172.
R. Sekar, Y. Guang, S. Verma, and T. Shanbhag, A high-performance network intrusion detection system, CCS '99: Proceedings of the 6th ACM conference on Computer and communications security, ACM Press, 1999, pp. 8–17.
Eugene H. Spafford and Diego Zamboni, Intrusion detection using autonomous agents, Computer Networks 34 (2000), no. 4, 547–570, http://www.sciencedirect.com/science/article/B6VRG-*411FRK9-*2/2/f818f61028e80aa2cd740fdc4a3cd696.
SJ Stolfo, W. Fan, W. Lee, A. Prodromidis, and PK Chan, Cost-based modeling for fraud and intrusion detection: results fromthe JAM project, Proceedings of the DARPA Information Survivability Conference and Exposition (DISCEX), vol. 2, 2000.
T. Takada and H.Koike, Nigelog: Protecting logging information by hiding multiple backups in directories, Proceedings of the International Conference on Electronic Commerece and Security, IEEE, IEEE, 1999, pp. 874–878.
M. Tavallaee, E. Bagheri, W. Lu, and A.A. Ghorbani, A Detailed Analysis of the KDD CUP 99 Data Set, Proceedings of the IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA), 2009.
Sholom M. Weiss and Tong Zhang, The handbook of data mining, ch. Performance Alanysis and Evaluation, pp. 426–439, Lawrence Erlbaum Assoc Inc, 2003.
Q. Xue, J. Sun, and Z. Wei, Tjids: an intrusion detection architecture for distributed network, Proceedings of the Canadian Conference on Electrical and Computer Engineering, IEEE CCECE 2003, May 2003, pp. 709–712.
Dong Yu and D. Frincke, Towards survivable intrusion detection system, Proceedings of the 37th Annual Hawaii International Conference on System Sciences, January 2004, pp. 299–308.
D. Zamboni, Using internal sensors for computer intrusion detection, Ph.D. thesis, Purdue University, Center for Education and Research in Information Assurance and Security, August 2001.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2010 Springer-Verlag US
About this chapter
Cite this chapter
Ghorbani, A.A., Lu, W., Tavallaee, M. (2010). Evaluation Criteria. In: Network Intrusion Detection and Prevention. Advances in Information Security, vol 47. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-88771-5_7
Download citation
DOI: https://doi.org/10.1007/978-0-387-88771-5_7
Published:
Publisher Name: Springer, Boston, MA
Print ISBN: 978-0-387-88770-8
Online ISBN: 978-0-387-88771-5
eBook Packages: Computer ScienceComputer Science (R0)