Abstract
As machine learning and cybersecurity continue to explode in the context of the digital ecosystem, the complexity of cybersecurity data combined with complicated and evasive machine learning algorithms leads to vast difficulties in designing an end-to-end system for intelligent, automatic anomaly classification. On the other hand, traditional systems use elementary statistics techniques and are often inaccurate, leading to weak centralized data analysis platforms. In this paper, we propose a novel system that addresses these two problems, titled CAMLPAD, for Cybersecurity Autonomous Machine Learning Platform for Anomaly Detection. The CAMLPAD system’s streamlined, holistic approach begins with retrieving a multitude of different species of cybersecurity data in real-time using elasticsearch, then running several machine learning algorithms, namely Isolation Forest, Histogram-Based Outlier Score (HBOS), Cluster-Based Local Outlier Factor (CBLOF), and K-Means Clustering, to process the data. Next, the calculated anomalies are visualized using Kibana and are assigned an outlier score, which serves as an indicator for whether an alert should be sent to the system administrator that there are potential anomalies in the network. After comprehensive testing of our platform in a simulated environment, the CAMLPAD system achieved an adjusted rand score of 95%, exhibiting the reliable accuracy and precision of the system. All in all, the CAMLPAD system provides an accurate, streamlined approach to real-time cybersecurity anomaly detection, delivering a novel solution that has the potential to revolutionize the cybersecurity sector.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Garcia-Teodoro, P., Diaz-Verdejo, J., Maciá-Fernández, G., Vázquez, E.: Anomaly-based network intrusion detection: techniques, systems and challenges. Comput. Secur. 28(1–2), 18–28 (2009)
Dasgupta, D. (ed.): Artificial Immune Systems and their Applications. Springer, Heidelberg (2012)
Demertzis, K., Iliadis, L., Spartalis, S.: A spiking one-class anomaly detection framework for cyber-security on industrial control systems. In: International Conference on Engineering Applications of Neural Networks, pp. 122–134. Springer, Cham (2017)
Dasgupta, D.: Immunity-based intrusion detection system: a general framework. In: Proceedings of the 22nd NISSC, vol. 1, pp. 147–160 (1999)
Abeshu, A., Chilamkurti, N.: Deep learning: the frontier for distributed attack detection in fog-to-things computing. IEEE Commun. Mag. 56(2), 169–175 (2018)
Patel, A., Qassim, Q., Wills, C.: A survey of intrusion detection and prevention systems. Inf. Manag. Comput. Secur. 18(4), 277–290 (2010)
Mylrea, M., Gourisetti, S.N.G.: Cybersecurity and optimization in smart “autonomous” buildings. In: Autonomy and Artificial Intelligence: A Threat or Savior?, pp. 263–294. Springer, Cham (2017)
Patel, A., Taghavi, M., Bakhtiyari, K., Junior, J.C.: An intrusion detection and prevention system in cloud computing: a systematic review. J. Netw. Comput. Appl. 36(1), 25–41 (2013)
Li, Y., Guo, L.: An active learning based TCM-KNN algorithm for supervised network intrusion detection. Comput. Secur. 26(7–8), 459–467 (2007)
Diro, A.A., Chilamkurti, N.: Distributed attack detection scheme using deep learning approach for Internet of Things. Futur. Gener. Comput. Syst. 82, 761–768 (2018)
Inacio, C.M., Trammell, B.: Yaf: yet another flowmeter. In: Proceedings of LISA10: 24th Large Installation System Administration Conference, p. 107 (2010)
Huang, M.Y., Jasper, R.J., Wicks, T.M.: A large scale distributed intrusion detection framework based on attack strategy analysis. Comput. Netw. 31(23–24), 2465–2475 (1999)
Russell, S., Dewey, D., Tegmark, M.: Research priorities for robust and beneficial artificial intelligence. Ai Mag. 36(4), 105–114 (2015)
Bilge, L., Balzarotti, D., Robertson, W., Kirda, E., Kruegel, C.: Disclosure: detecting botnet command and control servers through large-scale netflow analysis. In: Proceedings of the 28th Annual Computer Security Applications Conference, pp. 129–138. ACM (2012)
Chen, H., Chiang, R.H., Storey, V.C.: Business intelligence and analytics: from big data to big impact. MIS Q. 36(4) (2012)
Doelitzscher, F., Reich, C., Knahl, M., Passfall, A., Clarke, N.: An agent based business aware incident detection system for cloud environments. J. Cloud Comput.: Adv. Syst. Appl. 1(1), 9 (2012)
Ten, C.W., Hong, J., Liu, C.C.: Anomaly detection for cybersecurity of the substations. IEEE Trans. Smart Grid 2(4), 865–873 (2011)
Wressnegger, C., Schwenk, G., Arp, D., Rieck, K.: A close look on n-grams in intrusion detection: anomaly detection vs. classification. In: Proceedings of the 2013 ACM Workshop on Artificial Intelligence and Security, pp. 67–76. ACM (2013)
Aljawarneh, S., Aldwairi, M., Yassein, M.B.: Anomaly-based intrusion detection system through feature selection analysis and building hybrid efficient model. J. Comput. Sci. 25, 152–160 (2018)
Valeur, F., Mutz, D., Vigna, G.: A learning-based approach to the detection of SQL attacks. In: International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment, pp. 123–140 (2005)
Acknowledgments
We would like to thank the employees at Blue Cloak, LLC for their generous support throughout the duration of this research endeavor as well as for the cybersecurity data and tools used.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Hariharan, A., Gupta, A., Pal, T. (2020). CAMLPAD: Cybersecurity Autonomous Machine Learning Platform for Anomaly Detection. In: Arai, K., Kapoor, S., Bhatia, R. (eds) Advances in Information and Communication. FICC 2020. Advances in Intelligent Systems and Computing, vol 1130. Springer, Cham. https://doi.org/10.1007/978-3-030-39442-4_52
Download citation
DOI: https://doi.org/10.1007/978-3-030-39442-4_52
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-39441-7
Online ISBN: 978-3-030-39442-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)