For years, the research in intrusion detection field has been primarily focused on anomaly and misuse detection techniques. The latter method is traditionally favored in commercial products due to its predictability and high accuracy. In academic research, however, anomaly detection approach is perceived as a more powerful due to its theoretically higher potential to address novel attacks in comparison to misuse based methods. While academic community proposed a wide spectrum of anomaly based intrusion techniques, adequate comparison of the strengths and limitations of these techniques that can lead to potential commercial application is challenging. In this chapter we introduce the most significant criteria which have been proposed to have a more realistic evaluation of anomaly detection systems.
KeywordsFalse Alarm False Alarm Rate Receiver Operating Characteristic Receiver Operating Characteristic Curve Intrusion Detection
Unable to display preview. Download preview PDF.
- 1.MIT Lincoln Labs, 1998 DARPA Intrusion Detection Evaluation. Available on: http://www.ll.mit.edu/mission/communications/ist/corpora/ideval/index.html, February 2008.
- 2.KDD Cup 1999. Available on: http://kdd.ics.uci.edu/databases/kddcup 99/kddcup99.html, October 2007.
- 3.Nsl-kdd data set for network-based intrusion detection systems, Available on: http://iscx.cs.unb.ca/NSL-KDD/, March 2009.
- 4.Stefan Axelsson, The base-rate fallacy and its implications for the difficulty of intrusion detection, Proceedings of the 6th ACM conference on Computer and communication security (Kent Ridge Digital Labs, Singapore), ACM Press, November 1999, pp. 1–7.Google Scholar
- 5.——, The base-rate fallacy and the difficulty of intrusion detection, ACM Transactions on Information and System Security (TISSEC) 3 (2000), no. 3, 186–205.Google Scholar
- 6.P. Dokas, L. Ertoz, V. Kumar, A. Lazarevic, J. Srivastava, and P. Tan, Data mining for network intrusion detection, Proceedings of NSF Workshop on Next Generation Data Mining (Baltimore, MD), November 2002.Google Scholar
- 7.Mahesh V. Joshi, Ramesh C. Agarwal, and Vipin Kumar, Predicting rare classes: Can boosting make any weak lerner strong?, Proceedings of the SIG KDD (Edmonton, Alberta, Canada), 2002.Google Scholar
- 8.C. Kruegel, F. Valeur, G. Vigna, and R.A. Kemmerer, Stateful intrusion detection for high-speed networks, Proceedings of the IEEE Symposium on Security and Privacy (Oakland, CA), IEEE Press, May 2002, pp. 285–293.Google Scholar
- 9.W. Lee, W. Fan, M. Miller, s. Stolfo, and E. Zadok, Toward cost sensitive modeling for intrusion detection and response, Journal of Computer Security 10 (2002), no. 1,2, 5–22.Google Scholar
- 10.Wenke Lee, Joo B.D. Cabrera, Ashley Thomas, Niranjan Balwalli, Sunmeet Saluja, and Yi Zhang, Performance adaptation in real-time intrusion detection systems, Proceedings of Recent Advances in Intrusion Detection, 5th International Symposium, (RAID 2002) (Zurich, Switzerland) (A. Wespi, G. Vigna, and L. Deri, eds.), Lecture Notes in Computer Science, Springer-Verlag Heidelberg, October 2002, pp. 252–273.Google Scholar
- 11.R. Lippmann, D. Fried, I. Graf, J. Haines, K. Kendall, D. McClung, D. Weber, S. Webster, D. Wyschogrod, R. Cunningham, and M. Zissman, Evaluating intrusion detection systems: The 1998 darpa off-line intrusion detection evaluation, Proceedings of the 2000 DARPA Information Survivability Conference and Exposition (DISCEX-00), 2000, pp. 12–26.Google Scholar
- 12.M.V. Mahoney and P.K. Chan, An Analysis of the 1999 DARPA/Lincoln Laboratory Evaluation Data for Network Anomaly Detection, LECTURE NOTES IN COMPUTER SCIENCE (2003), 220–238.Google Scholar
- 14.NIST, Technology ITL, MITLL, Peter Mell, Vincent Hu, Richard Lippmann, Josh Haines, and Marc Zissman, An overview of issues in testing intrusion detection, July 2003.Google Scholar
- 15.Bryan Pfaffenberger, Webster's new world dictionary, ninth edition ed., ch. AC-3, p. 9, Hungry Minds, 2001.Google Scholar
- 17.T. Ptacek and T.Newsham, Insertion, evasion, and denial of service: Eluding network intrusion detection, 1998.Google Scholar
- 18.Stuart Russell and Peter Norving, Artificial intelligence a modern approach, second edition ed., ch. Uncertainty, pp. 462–491, Prentice Hall, 2003.Google Scholar
- 19.——, Artificial intelligence a modern approach, second edition ed., ch. Probabilistic Reasoning, pp. 492–536, Prentice Hall, 2003.Google Scholar
- 20.Lambert Schaelicke, Thomas Slabach, Branden Moore, and Curt Freeland, Characterizing the performance of network intrusion detection sensors, Proceedings of Recent Advances in Intrusion Detection, 6th International Symposium, (RAID 2003) (Pittsburgh, PA, USA) (G. Vigna, E. Jonsson, and C. Kruegel, eds.), Lecture Notes in Computer Science, Springer-Verlag Heidelberg, September 2003, pp. 155–172.Google Scholar
- 21.R. Sekar, Y. Guang, S. Verma, and T. Shanbhag, A high-performance network intrusion detection system, CCS '99: Proceedings of the 6th ACM conference on Computer and communications security, ACM Press, 1999, pp. 8–17.Google Scholar
- 22.Eugene H. Spafford and Diego Zamboni, Intrusion detection using autonomous agents, Computer Networks 34 (2000), no. 4, 547–570, http://www.sciencedirect.com/science/article/B6VRG-*411FRK9-*2/2/f818f61028e80aa2cd740fdc4a3cd696.
- 23.SJ Stolfo, W. Fan, W. Lee, A. Prodromidis, and PK Chan, Cost-based modeling for fraud and intrusion detection: results fromthe JAM project, Proceedings of the DARPA Information Survivability Conference and Exposition (DISCEX), vol. 2, 2000.Google Scholar
- 24.T. Takada and H.Koike, Nigelog: Protecting logging information by hiding multiple backups in directories, Proceedings of the International Conference on Electronic Commerece and Security, IEEE, IEEE, 1999, pp. 874–878.Google Scholar
- 25.M. Tavallaee, E. Bagheri, W. Lu, and A.A. Ghorbani, A Detailed Analysis of the KDD CUP 99 Data Set, Proceedings of the IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA), 2009.Google Scholar
- 26.Sholom M. Weiss and Tong Zhang, The handbook of data mining, ch. Performance Alanysis and Evaluation, pp. 426–439, Lawrence Erlbaum Assoc Inc, 2003.Google Scholar
- 27.Q. Xue, J. Sun, and Z. Wei, Tjids: an intrusion detection architecture for distributed network, Proceedings of the Canadian Conference on Electrical and Computer Engineering, IEEE CCECE 2003, May 2003, pp. 709–712.Google Scholar
- 28.Dong Yu and D. Frincke, Towards survivable intrusion detection system, Proceedings of the 37th Annual Hawaii International Conference on System Sciences, January 2004, pp. 299–308.Google Scholar
- 29.D. Zamboni, Using internal sensors for computer intrusion detection, Ph.D. thesis, Purdue University, Center for Education and Research in Information Assurance and Security, August 2001.Google Scholar