CAMLPAD: Cybersecurity Autonomous Machine Learning Platform for Anomaly Detection
- 18 Downloads
As machine learning and cybersecurity continue to explode in the context of the digital ecosystem, the complexity of cybersecurity data combined with complicated and evasive machine learning algorithms leads to vast difficulties in designing an end-to-end system for intelligent, automatic anomaly classification. On the other hand, traditional systems use elementary statistics techniques and are often inaccurate, leading to weak centralized data analysis platforms. In this paper, we propose a novel system that addresses these two problems, titled CAMLPAD, for Cybersecurity Autonomous Machine Learning Platform for Anomaly Detection. The CAMLPAD system’s streamlined, holistic approach begins with retrieving a multitude of different species of cybersecurity data in real-time using elasticsearch, then running several machine learning algorithms, namely Isolation Forest, Histogram-Based Outlier Score (HBOS), Cluster-Based Local Outlier Factor (CBLOF), and K-Means Clustering, to process the data. Next, the calculated anomalies are visualized using Kibana and are assigned an outlier score, which serves as an indicator for whether an alert should be sent to the system administrator that there are potential anomalies in the network. After comprehensive testing of our platform in a simulated environment, the CAMLPAD system achieved an adjusted rand score of 95%, exhibiting the reliable accuracy and precision of the system. All in all, the CAMLPAD system provides an accurate, streamlined approach to real-time cybersecurity anomaly detection, delivering a novel solution that has the potential to revolutionize the cybersecurity sector.
KeywordsMachine learning Cybersecurity Anomaly detection Clustering Visualization
We would like to thank the employees at Blue Cloak, LLC for their generous support throughout the duration of this research endeavor as well as for the cybersecurity data and tools used.
- 2.Dasgupta, D. (ed.): Artificial Immune Systems and their Applications. Springer, Heidelberg (2012)Google Scholar
- 3.Demertzis, K., Iliadis, L., Spartalis, S.: A spiking one-class anomaly detection framework for cyber-security on industrial control systems. In: International Conference on Engineering Applications of Neural Networks, pp. 122–134. Springer, Cham (2017)Google Scholar
- 4.Dasgupta, D.: Immunity-based intrusion detection system: a general framework. In: Proceedings of the 22nd NISSC, vol. 1, pp. 147–160 (1999)Google Scholar
- 7.Mylrea, M., Gourisetti, S.N.G.: Cybersecurity and optimization in smart “autonomous” buildings. In: Autonomy and Artificial Intelligence: A Threat or Savior?, pp. 263–294. Springer, Cham (2017)Google Scholar
- 11.Inacio, C.M., Trammell, B.: Yaf: yet another flowmeter. In: Proceedings of LISA10: 24th Large Installation System Administration Conference, p. 107 (2010)Google Scholar
- 14.Bilge, L., Balzarotti, D., Robertson, W., Kirda, E., Kruegel, C.: Disclosure: detecting botnet command and control servers through large-scale netflow analysis. In: Proceedings of the 28th Annual Computer Security Applications Conference, pp. 129–138. ACM (2012)Google Scholar
- 15.Chen, H., Chiang, R.H., Storey, V.C.: Business intelligence and analytics: from big data to big impact. MIS Q. 36(4) (2012)Google Scholar
- 18.Wressnegger, C., Schwenk, G., Arp, D., Rieck, K.: A close look on n-grams in intrusion detection: anomaly detection vs. classification. In: Proceedings of the 2013 ACM Workshop on Artificial Intelligence and Security, pp. 67–76. ACM (2013)Google Scholar
- 20.Valeur, F., Mutz, D., Vigna, G.: A learning-based approach to the detection of SQL attacks. In: International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment, pp. 123–140 (2005)Google Scholar