Enhancing Network Security Via Machine Learning: Opportunities and Challenges

  • Mahdi Amrollahi
  • Shahrzad Hadayeghparast
  • Hadis KarimipourEmail author
  • Farnaz Derakhshan
  • Gautam Srivastava


Network security can be defined as the act of protecting any given network against threats that may lead to the availability of the network to be compromised. Moreover, we can also add that unauthorized access or even misuse of network-accessible resources are issues that network security must address. Traditional detection techniques are inefficient when dealing with huge amounts of data because their analysis processes are complex and time-consuming. Hence, the use of tools and techniques provided to us through big data can assist in the analysis and storage of data in intrusion detection systems to help reduce both processing and training time. This document presents a review of the work related to network security via machine learning.


Cyber security Network Machine learning Intrusion detection (IDS) Supervised learning Anomaly detection 


  1. 1.
    D. Kiwia, A. Dehghantanha, K.K.R. Choo, J. Slaughter, A cyber kill chain based taxonomy of banking Trojans for evolutionary computational intelligence. J. Comput. Sci. 27, 394–409 (2018)CrossRefGoogle Scholar
  2. 2.
    H. Karimipour, S. Geris, A. Dehghantanha, H. Leung, Intelligent Anomaly Detection for Large-Scale Smart Grids (IEEE CCECE, Edmonton, 2019), pp. 1–4Google Scholar
  3. 3.
    F.N. Dezfouli, A. Dehghantanha, R. Mahmod, N.F.B.M. Sani, S.B. Shamsuddin, F. Daryabar, A survey on malware analysis and detection techniques. Int. J. Adv. Comput. Technol. 5(14), 42 (2013)Google Scholar
  4. 4.
    N.B. Anuar, M. Papadaki, S. Furnell, N. Clarke, An investigation and survey of response options for Intrusion Response Systems (IRSs), in 2010 Information Security for South Africa (IEEE, 2010 August), pp. 1–8Google Scholar
  5. 5.
    M. Baig, P. Zavarsky, R. Ruhl, D. Lindskog, The Study of Evasion of Packed PE from Static Detection (World Congress on. Internet Security (WorldCIS), 2012), pp. 99–104Google Scholar
  6. 6.
    M. Conti, T. Dargahi, A. Dehghantanha, Cyber threat intelligence: challenges and opportunities, in Cyber Threat Intelligence (Springer, Cham, 2018), pp. 1–6Google Scholar
  7. 7.
    C. Cepeda, D.L.C. Tien, P. Ordónez, Feature selection and improving classification performance for malware detection, in 2016 IEEE International Conferences on Big Data and Cloud Computing (BDCloud), Social Computing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom) (BDCloud-SocialCom-SustainCom) (IEEE, 2016 October), pp. 560–566Google Scholar
  8. 8.
    M. Damshenas, A. Dehghantanha, R. Mahmoud, A survey on malware propagation, analysis, and detection. Int. J. Cyber-Secur. Digit. Forensics 2(4), 10–30 (2013)Google Scholar
  9. 9.
    S.R. Bragen, Malware detection through opcode sequence analysis using machine learning, Master’s thesis (2015)Google Scholar
  10. 10.
    H. Karimipour, V. Dinavahi, On false data injection attack against dynamic state estimation on smart power grids. in 2017 IEEE International Conference on Smart Energy Grid Engineering (SEGE) (IEEE, 2017), pp. 388–393Google Scholar
  11. 11.
    M.R. Begli, F. Derakhshan, H. Karimipour, A layered intrusion detection system for critical infrastructure using machine learning, in IEEE Int. Conf. on Smart Energy Grid Engineering (SEGE) (IEEE, 2019), pp. 1–5Google Scholar
  12. 12.
    E.M. Dovom, A. Azmoodeh, A. Dehghantanha, D.E. Newton, R.M. Parizi, H. Karimipour, Fuzzy pattern tree for edge malware detection and categorization in IoT. J. Syst. Archit. 97, 1–7 (2019)CrossRefGoogle Scholar
  13. 13.
    H. Haddad Pajouh, A. Dehghantanha, R. Khayami, K.K.R. Choo, A deep recurrent neural network based approach for internet of things malware threat hunting. Futur. Gener. Comput. Syst. 85, 88–96 (2018)CrossRefGoogle Scholar
  14. 14.
    E. Brynjolfsson, T. Mitchell, What can machine learning do? Workforce implications. Science 358(6370), 1530–1534 (2017)CrossRefGoogle Scholar
  15. 15.
    Z. Ghahramani, Probabilistic machine learning and artificial intelligence. Nature 521(7553), 452–459 (2015)CrossRefGoogle Scholar
  16. 16.
    A. Azmoodeh, A. Dehghantanha, K.K.R. Choo, Robust malware detection for internet of (battlefield) things devices using deep eigenspace learning. IEEE Trans. Sustain. Comput. 4(1), 88–95 (2018)CrossRefGoogle Scholar
  17. 17.
    O. Osanaiye, H. Cai, K.K.R. Choo, A. Dehghantanha, Z. Xu, M. Dlodlo, Ensemble-based multi-filter feature selection method for DDoS detection in cloud computing. EURASIP J. Wirel. Commun. Netw. 2016(1), 130 (2016)CrossRefGoogle Scholar
  18. 18.
    Y. Xin, L. Kong, Z. Liu, Y. Chen, Y. Li, H. Zhu, M. Gao, H. Hou, C. Wang, Machine learning and deep learning methods for cybersecurity. IEEE Access 6, 35365–35381 (2018)CrossRefGoogle Scholar
  19. 19.
    S. Dua, X. Du, Data Mining and Machine Learning in Cybersecurity (Auerbach Publications, Boca Raton, 2016)CrossRefGoogle Scholar
  20. 20.
    R. Verma, M. Kantarcioglu, D. Marchette, E. Leiss, T. Solorio, Security analytics: essential data analytics knowledge for cybersecurity professionals and students. IEEE Secur. Priv. 13(6), 60–65 (2015)CrossRefGoogle Scholar
  21. 21.
    J. Sakhnini, A. Dehghantanha, H. Karimipour, Smart grid cyber attacks detection using supervised learning and heuristic feature selection, in IEEE Int. Conf. on Smart Energy Grid Engineering (SEGE), Canada (IEEE, 2019), pp. 1–5Google Scholar
  22. 22.
    H. Karimipour, A. Dehghantanha, R.M. Parizi, K.R. Choo, H. Leung, A deep and scalable unsupervised machine learning system for cyber-attack detection in large-scale smart grids. IEEE Access 7, 80778–80788 (2019)CrossRefGoogle Scholar
  23. 23.
    S. Omar, A. Ngadi, H.H. Jebur, Machine learning techniques for anomaly detection: An overview. Int. J. Comput. Appl. 79(2), 33–41 (2013)Google Scholar
  24. 24.
    R. Sommer, V. Paxson, Outside the closed world: on using machine learning for network intrusion detection, in 2010 IEEE Symposium on Security and Privacy (IEEE, 2010), pp. 305–316Google Scholar
  25. 25.
    C.F. Tsai, Y.F. Hsu, C.Y. Lin, W.Y. Lin, Intrusion detection by machine learning: A review. Expert Syst. Appl. 36(10), 11994–12000 (2009)CrossRefGoogle Scholar
  26. 26.
    G. Apruzzese, M. Colajanni, L. Ferretti, A. Guido, M. Marchetti, On the effectiveness of machine and deep learning for cyber security, in 2018 10th International Conference on Cyber Conflict (CyCon) (IEEE, 2018), pp. 371–390Google Scholar
  27. 27.
    D.S. Berman, A.L. Buczak, J.S. Chavis, C.L. Corbett, A survey of deep learning methods for cyber security. Information 10(4), 122 (2019)CrossRefGoogle Scholar
  28. 28.
    C. Zhang, P. Patras, H. Haddadi, Deep learning in mobile and wireless networking: A survey. IEEE Commun. Surv. Tutor. 21, 2224–2287 (2019)CrossRefGoogle Scholar
  29. 29.
    Z. Yuan, Y. Lu, Z. Wang, Y. Xue, Droid-sec: deep learning in android malware detection, in ACM SIGCOMM Computer Communication Review, vol. 44, no. 4 (ACM, 2014), pp. 371–372Google Scholar
  30. 30.
    Z. Yuan, Y. Lu, Y. Xue, Droiddetector: Android malware characterization and detection using deep learning. Tsinghua Sci. Technol. 21(1), 114–123 (2016)CrossRefGoogle Scholar
  31. 31.
    R. Pascanu, J.W. Stokes, H. Sanossian, M. Marinescu, A. Thomas, Malware classification with recurrent networks, in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (IEEE, 2015 April), pp. 1916–1920Google Scholar
  32. 32.
    B. Kolosnjaji, A. Zarras, G. Webster, C. Eckert, Deep learning for classification of malware system call sequences, in Australasian Joint Conference on Artificial Intelligence (Springer, Cham, 2016 December), pp. 137–149Google Scholar
  33. 33.
    S. Tobiyama, Y. Yamaguchi, H. Shimada, T. Ikuse, T. Yagi, Malware detection with deep neural network using process behavior, in 2016 IEEE 40th Annual Computer Software and Applications Conference (COMPSAC), vol. 2 (IEEE, 2016 June), pp. 577–582Google Scholar
  34. 34.
    Y. Ding, S. Chen, J. Xu, Application of deep belief networks for opcode based malware detection, in 2016 International Joint Conference on Neural Networks (IJCNN) (IEEE, 2016 July), pp. 3901–3908Google Scholar
  35. 35.
    N. McLaughlin, J. Martinez del Rincon, B. Kang, S. Yerima, P. Miller, S. Sezer, Y. Safaei, E. Trickel, Z. Zhao, A. Doupé, G. Joon Ahn, Deep android malware detection, in Proceedings of the Seventh ACM on Conference on Data and Application Security and Privacy (ACM, 2017 March), pp. 301–308Google Scholar
  36. 36.
    W. Hardy, L. Chen, S. Hou, Y. Ye, X. Li, DL4MD: a deep learning framework for intelligent malware detection, in Proceedings of the International Conference on Data Mining (DMIN), (The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp), 2016), p. 61Google Scholar
  37. 37.
    G. Tzortzis, A. Likas, Deep belief networks for spam filtering, in 19th IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2007), vol. 2 (IEEE, 2007), pp. 306–309Google Scholar
  38. 38.
    G. Mi, Y. Gao, Y. Tan, Apply stacked auto-encoder to spam detection, in International Conference in Swarm Intelligence (Springer, Cham, 2015), pp. 3–15Google Scholar
  39. 39.
    O. Depren, M. Topallar, E. Anarim, M.K. Ciliz, An intelligent intrusion detection system (IDS) for anomaly and misuse detection in computer networks. Expert Syst. Appl. 29(4), 713–722 (2005)CrossRefGoogle Scholar
  40. 40.
    A.L. Buczak, E. Guven, A survey of data mining and machine learning methods for cyber security intrusion detection. IEEE Commun. Surv. Tutor. 18(2), 1153–1176 (2015)CrossRefGoogle Scholar
  41. 41.
    R. Boutaba, M.A. Salahuddin, N. Limam, S. Ayoubi, N. Shahriar, F. Estrada-Solano, O.M. Caicedo, A comprehensive survey on machine learning for networking: Evolution, applications and research opportunities. J. Int. Serv. Appl. 9(1), 16 (2018)CrossRefGoogle Scholar
  42. 42.
    D.E. Denning, An intrusion-detection model. IEEE Trans. Softw. Eng. SE-13(2), 222–232 (1987)CrossRefGoogle Scholar
  43. 43.
    S. Mohammadi, H. Mirvaziri, M. Ghazizadeh-Ahsaee, H. Karimipour, Cyber intrusion detection by combined feature selection algorithm. J. Inf. Secur. Appl. 44, 80–88 (2019)Google Scholar
  44. 44.
    H.H. Pajouh, R. Javidan, R. Khayami, D. Ali, K.K.R. Choo, A two-layer dimension reduction and two-tier classification model for anomaly-based intrusion detection in IoT backbone networks. IEEE Trans. Emerg. Top. Comput. 7(2), 314–323 (2016)CrossRefGoogle Scholar
  45. 45.
    S. Geris, H. Karimipour, A feature selection-based approach for joint cyber-attack detection and state estimation, in IEEE Int. Conf. on Smart Energy Grid Engineering (SEGE) (IEEE, 2019), pp. 1–5Google Scholar
  46. 46.
    S. Mohammadi, H. Mirvaziri, M. Ghazizadeh-Ahsaee, Multivariate correlation coefficient and mutual information-based feature selection in intrusion detection. Inf. Secur. J. Glob. Perspect. 26(5), 229–239 (2017)CrossRefGoogle Scholar
  47. 47.
    K. Wang, S.J. Stolfo, Anomalous payload-based network intrusion detection, in International Workshop on Recent Advances in Intrusion Detection (Springer, Berlin, 2004), pp. 203–222Google Scholar
  48. 48.
    S. Zanero, S.M. Savaresi, Unsupervised learning techniques for an intrusion detection system, in Proceedings of the 2004 ACM symposium on Applied computing (ACM, 2004), pp. 412–419Google Scholar
  49. 49.
    J. Beale, R. Deraison, H. Meer, R. Temmingh, C.V.D. Walt, Nessus Network Auditing (Syngress Publishing, Burlington, 2004)Google Scholar
  50. 50.
    N. Görnitz, M. Kloft, K. Rieck, U. Brefeld, Active learning for network intrusion detection, in Proceedings of the 2nd ACM Workshop on Security and Artificial Intelligence (ACM, 2009), pp. 47–54Google Scholar
  51. 51.
    L.L.C. Metasploit, The metasploit framework (2007),
  52. 52.
    A. Javaid, Q. Niyaz, W. Sun, M. Alam, A deep learning approach for network intrusion detection system, in Proceedings of the 9th EAI International Conference on Bio-inspired Information and Communications Technologies (Formerly BIONETICS) (ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), 2016), pp. 21–26Google Scholar
  53. 53.
    M.Z. Alom, V. Bontupalli, T.M. Taha, Intrusion detection using deep belief networks, in 2015 National Aerospace and Electronics Conference (NAECON) (IEEE, 2015), pp. 339–344Google Scholar
  54. 54.
    Y. Li, R. Ma, R. Jiao, A hybrid malicious code detection method based on deep learning. Int. J. Secur. Appl. 9(5), 205–216 (2015)Google Scholar
  55. 55.
    J. Kim, J. Kim, H.L.T. Thu, H. Kim, Long short term memory recurrent neural network classifier for intrusion detection, in 2016 International Conference on Platform Technology and Service (PlatCon) (IEEE, 2016), pp. 1–5Google Scholar
  56. 56.
    R. Raina, A. Battle, H. Lee, B. Packer, A.Y. Ng, Self-taught learning: transfer learning from unlabeled data, in Proceedings of the 24th International Conference on Machine learning (ACM, 2007), pp. 759–766Google Scholar
  57. 57.
    T.A. Tang, L. Mhamdi, D. McLernon, S.A.R. Zaidi, M. Ghogho, Deep learning approach for network intrusion detection in software defined networking, in 2016 International Conference on Wireless Networks and Mobile Communications (WINCOM). (IEEE, 2016), pp. 258–263Google Scholar
  58. 58.
    J. Cannady, Next generation intrusion detection: autonomous reinforcement learning of network attacks, in Proceedings of the 23rd National Information Systems Security Conference (NIST, 2000), pp. 1–12Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Mahdi Amrollahi
    • 1
  • Shahrzad Hadayeghparast
    • 1
  • Hadis Karimipour
    • 1
    Email author
  • Farnaz Derakhshan
    • 2
  • Gautam Srivastava
    • 3
  1. 1.School of EngineeringUniversity of GuelphGuelphCanada
  2. 2.Department of Electrical and Computer EngineeringUniversity of TabrizTabrizIran
  3. 3.Department of Mathematics and Computer ScienceBrandon UniversityBrandonCanada

Personalised recommendations