CAMLPAD: Cybersecurity Autonomous Machine Learning Platform for Anomaly Detection

  • Ayush HariharanEmail author
  • Ankit Gupta
  • Trisha Pal
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1130)


As machine learning and cybersecurity continue to explode in the context of the digital ecosystem, the complexity of cybersecurity data combined with complicated and evasive machine learning algorithms leads to vast difficulties in designing an end-to-end system for intelligent, automatic anomaly classification. On the other hand, traditional systems use elementary statistics techniques and are often inaccurate, leading to weak centralized data analysis platforms. In this paper, we propose a novel system that addresses these two problems, titled CAMLPAD, for Cybersecurity Autonomous Machine Learning Platform for Anomaly Detection. The CAMLPAD system’s streamlined, holistic approach begins with retrieving a multitude of different species of cybersecurity data in real-time using elasticsearch, then running several machine learning algorithms, namely Isolation Forest, Histogram-Based Outlier Score (HBOS), Cluster-Based Local Outlier Factor (CBLOF), and K-Means Clustering, to process the data. Next, the calculated anomalies are visualized using Kibana and are assigned an outlier score, which serves as an indicator for whether an alert should be sent to the system administrator that there are potential anomalies in the network. After comprehensive testing of our platform in a simulated environment, the CAMLPAD system achieved an adjusted rand score of 95%, exhibiting the reliable accuracy and precision of the system. All in all, the CAMLPAD system provides an accurate, streamlined approach to real-time cybersecurity anomaly detection, delivering a novel solution that has the potential to revolutionize the cybersecurity sector.


Machine learning Cybersecurity Anomaly detection Clustering Visualization 



We would like to thank the employees at Blue Cloak, LLC for their generous support throughout the duration of this research endeavor as well as for the cybersecurity data and tools used.


  1. 1.
    Garcia-Teodoro, P., Diaz-Verdejo, J., Maciá-Fernández, G., Vázquez, E.: Anomaly-based network intrusion detection: techniques, systems and challenges. Comput. Secur. 28(1–2), 18–28 (2009)CrossRefGoogle Scholar
  2. 2.
    Dasgupta, D. (ed.): Artificial Immune Systems and their Applications. Springer, Heidelberg (2012)Google Scholar
  3. 3.
    Demertzis, K., Iliadis, L., Spartalis, S.: A spiking one-class anomaly detection framework for cyber-security on industrial control systems. In: International Conference on Engineering Applications of Neural Networks, pp. 122–134. Springer, Cham (2017)Google Scholar
  4. 4.
    Dasgupta, D.: Immunity-based intrusion detection system: a general framework. In: Proceedings of the 22nd NISSC, vol. 1, pp. 147–160 (1999)Google Scholar
  5. 5.
    Abeshu, A., Chilamkurti, N.: Deep learning: the frontier for distributed attack detection in fog-to-things computing. IEEE Commun. Mag. 56(2), 169–175 (2018)CrossRefGoogle Scholar
  6. 6.
    Patel, A., Qassim, Q., Wills, C.: A survey of intrusion detection and prevention systems. Inf. Manag. Comput. Secur. 18(4), 277–290 (2010)CrossRefGoogle Scholar
  7. 7.
    Mylrea, M., Gourisetti, S.N.G.: Cybersecurity and optimization in smart “autonomous” buildings. In: Autonomy and Artificial Intelligence: A Threat or Savior?, pp. 263–294. Springer, Cham (2017)Google Scholar
  8. 8.
    Patel, A., Taghavi, M., Bakhtiyari, K., Junior, J.C.: An intrusion detection and prevention system in cloud computing: a systematic review. J. Netw. Comput. Appl. 36(1), 25–41 (2013)CrossRefGoogle Scholar
  9. 9.
    Li, Y., Guo, L.: An active learning based TCM-KNN algorithm for supervised network intrusion detection. Comput. Secur. 26(7–8), 459–467 (2007)CrossRefGoogle Scholar
  10. 10.
    Diro, A.A., Chilamkurti, N.: Distributed attack detection scheme using deep learning approach for Internet of Things. Futur. Gener. Comput. Syst. 82, 761–768 (2018)CrossRefGoogle Scholar
  11. 11.
    Inacio, C.M., Trammell, B.: Yaf: yet another flowmeter. In: Proceedings of LISA10: 24th Large Installation System Administration Conference, p. 107 (2010)Google Scholar
  12. 12.
    Huang, M.Y., Jasper, R.J., Wicks, T.M.: A large scale distributed intrusion detection framework based on attack strategy analysis. Comput. Netw. 31(23–24), 2465–2475 (1999)CrossRefGoogle Scholar
  13. 13.
    Russell, S., Dewey, D., Tegmark, M.: Research priorities for robust and beneficial artificial intelligence. Ai Mag. 36(4), 105–114 (2015)CrossRefGoogle Scholar
  14. 14.
    Bilge, L., Balzarotti, D., Robertson, W., Kirda, E., Kruegel, C.: Disclosure: detecting botnet command and control servers through large-scale netflow analysis. In: Proceedings of the 28th Annual Computer Security Applications Conference, pp. 129–138. ACM (2012)Google Scholar
  15. 15.
    Chen, H., Chiang, R.H., Storey, V.C.: Business intelligence and analytics: from big data to big impact. MIS Q. 36(4) (2012)Google Scholar
  16. 16.
    Doelitzscher, F., Reich, C., Knahl, M., Passfall, A., Clarke, N.: An agent based business aware incident detection system for cloud environments. J. Cloud Comput.: Adv. Syst. Appl. 1(1), 9 (2012)CrossRefGoogle Scholar
  17. 17.
    Ten, C.W., Hong, J., Liu, C.C.: Anomaly detection for cybersecurity of the substations. IEEE Trans. Smart Grid 2(4), 865–873 (2011)CrossRefGoogle Scholar
  18. 18.
    Wressnegger, C., Schwenk, G., Arp, D., Rieck, K.: A close look on n-grams in intrusion detection: anomaly detection vs. classification. In: Proceedings of the 2013 ACM Workshop on Artificial Intelligence and Security, pp. 67–76. ACM (2013)Google Scholar
  19. 19.
    Aljawarneh, S., Aldwairi, M., Yassein, M.B.: Anomaly-based intrusion detection system through feature selection analysis and building hybrid efficient model. J. Comput. Sci. 25, 152–160 (2018)CrossRefGoogle Scholar
  20. 20.
    Valeur, F., Mutz, D., Vigna, G.: A learning-based approach to the detection of SQL attacks. In: International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment, pp. 123–140 (2005)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Blue Cloak LLCSterlingUSA

Personalised recommendations