Deep Learning Meets Malware Detection: An Investigation

  • Biozid Bostami
  • Mohiuddin Ahmed
Part of the Advanced Sciences and Technologies for Security Applications book series (ASTSA)


From the dawn of computer programs, malware programs were originated and still with us. With evolving of technology, malware programs are also evolving. It is considered as one of the prime issues regarding cyber world security. Damage caused by the malware programs ranges from system failure to financial loss. Traditional approach for malware classification approach are not very suitable for advance malware programs. For the continuously evolving malware ecosystem deep learning approaches are more suitable as they are faster and can predict malware more effectively. To our best of knowledge, there has not substantial research done on deep learning based malware detection on different sectors like: IoT, Bio-medical sectors and Cloud platforms. The key contribution of this chapter will be creating directions of malware detection depending on deep learning. The chapter will be beneficial for graduate level students, academicians and researchers in this application domain.


  1. 1.
    McGraw G, Morrisett G (2000) Attacking malicious code: a report to the infosec research council. IEEE Softw 17(5):33–41CrossRefGoogle Scholar
  2. 2.
    Xufang L, Loh PKK, Tan F (2011) Mechanisms of polymorphic and metamorphic viruses. In 2011 European intelligence and security informatics conference (EISIC) 149–154Google Scholar
  3. 3.
    Cohen F (1987) Computer viruses. Comput Secur 6:22–35CrossRefGoogle Scholar
  4. 4.
    EroCarrera, Silberman P (2010) State of malware: family tiesGoogle Scholar
  5. 5.
    Egele M et al (2008) A survey on automated dynamic malware-analysis techniques and tools. ACM Comput Surv 44(2):1–42CrossRefGoogle Scholar
  6. 6.
    Vinod P et al (2009) Survey on malware detection methodsGoogle Scholar
  7. 7.
  8. 8.
    Yin H et al (2007) Panorama: capturing system-wide information flow for malware detection and analysis. In: Proceedings of the 14th ACM conference on computer and communications security. ACM, Alexandria, pp 116–127Google Scholar
  9. 9.
    Idika N, Mathur AP (2007) A survey of malware detection techniquesGoogle Scholar
  10. 10.
    Bostami B, Ahmed M, Choudhury S (2019) False data injection attacks in internet of things. In: Al-Turjman F (ed) Performability in internet of things. EAI/Springer innovations in communication and computing. Springer, ChamGoogle Scholar
  11. 11.
    Beaucamps P (2007) Advanced polymorphic techniques. Int J Comput Sci 2(3):194–205Google Scholar
  12. 12.
    Szor P (2005) The art of computer virus research and defense. Addison-Wesley Professional, Upper Saddle RiverGoogle Scholar
  13. 13.
    Shah A (2010) Approximate disassembly using dynamic programming [PhD. Thesis], San Jose State University, USGoogle Scholar
  14. 14.
    Szor P (1998) The Marburg situation. Virus Bull:8–10Google Scholar
  15. 15.
    Filiol E (2005) Computer viruses: from theory to applications. Springer, PariszbMATHGoogle Scholar
  16. 16.
    Walenstein A, Mathur R, Chouchane M et al (2007) The design space of metamorphic malware. In: Proceedings of the 2nd international conference on information warfare and security (ICIW 2007), pp 241–248Google Scholar
  17. 17.
    Lakhotia A, Kapoor A, Kumar E (2004) Are metamorphic viruses really invincible? Virus Bull:5–7Google Scholar
  18. 18.
    Ferrie P, Corporation S, Monica S (2001) Hunting for metamorphic. Proceedings of the Virus Bulletin Conference 2001, Czech Republic, Prague, 2001 September 27–28, 123144Google Scholar
  19. 19.
    Gutmann P (2007) The commercial malware industryGoogle Scholar
  20. 20.
    Islam MDR, Tian R, Batten LM, Versteeg S (2013) Classification of malware based on integrated static and dynamic features. J Netw Comput Appl 36(2):646–656CrossRefGoogle Scholar
  21. 21.
    Tahir R (2018) A study on malware and malware detection techniques. Int J Educ Manag Eng 8:20–30. Scholar
  22. 22.
    Jacob G, Debar H, Filiol E (2008) Behavioral detection of malware: from a survey towards established taxonomy. J Comput Virol 4(3):251–266CrossRefGoogle Scholar
  23. 23.
    Hofmeyr S, Forrest S, Somayaji A (1998) Intrusion detection using sequences of system calls. J Comput Secur 6:151–180CrossRefGoogle Scholar
  24. 24.
    Sato I, Okazaki Y, Goto S (2002) An improved intrusion detection method based on process profiling. IPSJ J 43:3316–3326Google Scholar
  25. 25.
    Mohata VB (2013) Mobile malware detection techniques. Int J Comput Sci Eng Technol (IJCSET)Google Scholar
  26. 26.
    Schultz M, Eskin E, Zadok E, Stolfo S (2001) Data mining methods for detection of new malicious executables. In IEEE symposium on security and privacy, pp 38–49. IEEE Computer SocietyGoogle Scholar
  27. 27.
    Henchiri O, Japkowicz N (2006) A feature selection and evaluation scheme for computer virus detection. In: Proceedings of ICDM-2006, Hong Kong, pp 891–895Google Scholar
  28. 28.
    Ye Y, Wang D, Li T, Ye D (2007) IMDS: intelligent malware detection system. In: Proceedings of the ACM international conference on knowledge discovery data mining, pp 1043–1047Google Scholar
  29. 29.
    Ye Y, Li T, Jiang Q, Wang Y (2010) CIMDS: adapting post processing techniques of associative classification for malware detection. IEEE Trans Syst Man Cybern C 40(3):298–307CrossRefGoogle Scholar
  30. 30.
    Jeong K, Lee H (2008) Code graph for malware detection. In information networking. In: ICOIN. International conference on, Jan 2008Google Scholar
  31. 31.
    Lee J, Jeong K, Lee H (2010) Detecting metamorphic malwares using computing, ser. ACM, New York, pp 1970–1977Google Scholar
  32. 32.
    Ye Y, Li T, Huang K, Jiang Q, Chen Y (2010) Hierarchical associative classifier (HAC) for malware detection from the large and imbalanced gray list. J Intell Inf Syst 35(1):1–20CrossRefGoogle Scholar
  33. 33.
    Ahmed F, Hameed H, Shafiq MZ, Farooq M (2009) Using spatio-temporal information in API calls with machine learning algorithms for malware detection. In: AISec ‘09 Proceedings of the 2nd ACM workshop on Security and artificial intelligence, pp 55–62Google Scholar
  34. 34.
    Bilar D (2007) OpCodes as predictor for malware. Int J Electron Secur Digit Forensics 1(2):156CrossRefGoogle Scholar
  35. 35.
    Santos I, Brezo F, Nieves J, Penya Y (2010) Idea: OpCode-sequencebased malware detection. In: Engineering secure software and system. Springer, Berlin/HeidelbergGoogle Scholar
  36. 36.
    Santos I, Brezo F, Ugarte-Pedrero X, Bringas PG (2011) OpCode sequences as representation of executables for data-mining-based unknown malware detection. Inf Sci 231:64–82MathSciNetCrossRefGoogle Scholar
  37. 37.
    Santos I, Brezo F, Sanz B, Laorden C, Bringas PG (2011) Using opCode sequences in single-class learning to detect unknown malware. IET Inf Secur 5(4):220CrossRefGoogle Scholar
  38. 38.
    Santos I, Laorden C, Bringas P (2011) Collective classification for unknown malware detection. In: Proceedings of the 6th ACM symposium on information, computer and communications securityGoogle Scholar
  39. 39.
    Santos I, Sanz B, Laorden C (2011) OpCode-sequence-based semisupervised unknown malware detection. In: Computational intelligence in security for information systems. Springer, Berlin/HeidelbergGoogle Scholar
  40. 40.
    Runwal N, Low RM, Stamp M (2012) OpCode graph similarity and metamorphic detection. J Comput Virol 8(1–2):37–52CrossRefGoogle Scholar
  41. 41.
    Shabtai A, Moskovitch R, Feher C, Dolev S, Elovici Y (2012) Detecting unknown malicious code by applying classification techniques on OpCode patterns. Secur Inf 1(1):1CrossRefGoogle Scholar
  42. 42.
    Gerald GBS, Tesauro J, Kephart JO (1996) Neural network for computer virus recognition. IEEE ExpertGoogle Scholar
  43. 43.
    Arnold W, Tesauro G (2000) Automatically generated Win32 heuristic virus detection. In Virus Bulletin ConferenceGoogle Scholar
  44. 44.
    Abou-assaleh, T, Cercone N, Keß V, Sweidan R (2004) N-gram-based detection of new malicious code, no. 1Google Scholar
  45. 45.
    Maloof MA, Kolter JZ (2006) Learning to detect malicious executables in the wild. In roc of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data MiningGoogle Scholar
  46. 46.
    Moskovitch EY, Stopel D, Feher C, Nissim N, Japkowicz N (2009) Unknown malcode detection and the imbalance problem. J Comput Virol 5(4):295–308CrossRefGoogle Scholar
  47. 47.
    Bruschi D, Martignoni L, Monga M (2006) Detecting self-mutating malware using control-flow graph matching. In: Büschkes R, Laskov P (eds) Detection of intrusions and malware & vulnerability assessment, volume 4064 of LNCS. Springer, Berlin, pp 129–143Google Scholar
  48. 48.
    Zhao Z (2011) A virus detection scheme based on features of control flow graph. 2nd International Conference on Artificial Intelligence, Management Science and Electronic Commerce (AIMSEC), pp 943–947Google Scholar
  49. 49.
    Bonfante G, Kaczmarek M, Marion JY (2007) Control flow graphs as malware signatures. WTCVGoogle Scholar
  50. 50.
    Eskandari M, Hashemi S (2011) Metamorphic malware detection using control flow graph mining. Int J Comput Sci Netw Secur 11:1–6Google Scholar
  51. 51.
    Kim K, Moon BR (2010) Malware detection based on dependency graph using hybrid genetic algorithm. In Proceedings of the 12th annual conference on Genetic and evolutionary computation, July 07–11, 2010Google Scholar
  52. 52.
    Nataraj L, Karthikeyan S, Jacob G, Manjunath BS (2011) Malware images: visualization and automatic classification. In: Proceedings of the 8th international symposium on visualization for cyber security, VizSec ‘11. ACM.. ISBN 978-1-4503-0679-9, New York, pp 4:1–4:7. Scholar
  53. 53.
    Dahl GE, Stokes JW, Deng L, Yu D (2013) Large-scale malware classification using random projections and neural networks. In Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on Acoustics. IEEE, 3422–3426Google Scholar
  54. 54.
    Saxe J, Berlin K (2015) Deep neural network based malware detection using two dimensional binary program features. In 2015 10th International Conference on Malicious and Unwanted Software (MALWARE). IEEEGoogle Scholar
  55. 55.
    Pascanu R, Stokes JW, Sanossian H, Marinescu M, Thomas A (2015) Malware classification with recurrent networks. In acoustics, speech and signal processing (ICASSP), 2015 IEEE International Conference on Acoustics. IEEE, 1916–1920Google Scholar
  56. 56.
    Cakir B, Dogdu E (2018) Malware classification using deep learning methods. In: Proceedings of the ACMSE 2018 conference (ACMSE ‘18). ACM, New York. Article 10, 5 pagesGoogle Scholar
  57. 57.
    Raff E, Barker J, Sylvester J, Brandon R, Catanzaro B, Nicholas C (2017) Malware detection by eating a whole exe. arXiv preprint arXiv:1710.09435Google Scholar
  58. 58.
    David OE, Netanyahu NS (2015) DeepSign: deep learning for automatic malware signature generation and classification. 2015 International Joint Conference on Neural Networks (IJCNN), Killarney, 2015, pp 1–8Google Scholar
  59. 59.
    Karbab E Debbabi M Derhab A Mouheb D (2017) Android malware detection using deep learning on API method sequencesGoogle Scholar
  60. 60.
    Choi S, Jang S, Kim Y, Kim J (2017) Malware detection using malware image and deep learning. 2017 International conference on information and communication technology convergence (ICTC), Jeju, 2017, pp 1193–1195Google Scholar
  61. 61.
    Le Q, Boydell O, Mac Namee B, Scanlon M (2018) Deep learning at the shallow end: malware classification for non-domain experts. Digit Investig 26:S118–S126CrossRefGoogle Scholar
  62. 62.
    Hardy W, Chen L, Hou S, Ye Y, Li X (2016) Dl4md: a deep learning framework for intelligent malware detection. Athens: The Steering Committee of The World Congress in computer science, computer engineering and applied computing (WorldComp), pp 61–67Google Scholar
  63. 63.
    Huang W, Stokes JW (2016) MtNet: a multi-task neural network for dynamic malware classification. In: In Proc. of the 13th international conference on detection of intrusions and malware, and vulnerability assessment, DIMVA 2016. Springer, Cham, pp 399–418Google Scholar
  64. 64.
    Davis A, Wolff M (2015) Deep learning on disassembly data. URL:
  65. 65.
    Tobiyama S, Yamaguchi Y, Shimada H, Ikuse T, Yagi T (2016) Malware detection with deep neural network using process behavior. In: 2016 IEEE 40th annual computer software and applications conference (COMPSAC), vol 2, pp 577–582. Scholar
  66. 66.
    Kang H, Jang JW, Mohaisen A, Kim HK (2015) Detecting and classifying android malware using static analysis along with creator information. Int J Distrib Sens Netw 11(6):479174CrossRefGoogle Scholar
  67. 67.
    Faruki P, Laxmi V, Bharmal A, Gaur MS, Ganmoor V (2015) AndroSimilar: robust signature for detecting variants of Android malware. J Inf Secur Appl 22:66–80Google Scholar
  68. 68.
    Song J, Han C, Wang K, Zhao J, Ranjan R, Wang L (2016) An integrated static detection and analysis framework for Android. Pervasive Mob Comput 32:15–25CrossRefGoogle Scholar
  69. 69.
    Sun M, Li X, Lui JC, Ma RT, Liang Z (2017) Monet: a user-oriented behavior-based malware variants detection system for Android. IEEE Trans Inf Forensics Secur 12(5):1103–1112CrossRefGoogle Scholar
  70. 70.
    Rovelli P, Vigfússon Ý (2014) PMDS: permission-based malware detection system. In: Prakash A, Shyamasundar R (eds) ICISS 2014. LNCS, vol 8880. Springer, Cham, pp 338–357. Scholar
  71. 71.
    Wu DJ, Mao CH, Wei TE, Lee HM, Wu KP (2012) DroidMat: android malware detection through manifest and API calls tracing. In: 2012 seventh Asia joint conference on information security (Asia JCIS), pp. 62–69. IEEEGoogle Scholar
  72. 72.
    Talha KA, Alper DI, Aydin C (2015) APK auditor: permission-based Android malware detection system. Digit Investig 13:1–14CrossRefGoogle Scholar
  73. 73.
    Sato R, Chiba D, Goto S (2013) Detecting Android malware by analyzing manifest files. Proc Asia Pac Adv Netw 36(23–31):17Google Scholar
  74. 74.
    Ping X, Xiaofeng W, Wenjia N, Tianqing Z, Gang L (2014) Android malware detection with contrasting permission patterns. China Commun 11(8):1–14Google Scholar
  75. 75.
    Vidal JM, Monge MAS, Villalba LJG (2018) A novel pattern recognition system for detecting Android malware by analyzing suspicious boot sequences. Knowl-Based Syst 150: 198–217CrossRefGoogle Scholar
  76. 76.
    Canfora G, Mercaldo F, Visaggio CA (2016) An HMM and structural entropy based detector for android malware: an empirical study. Comput Secur 61:1–18CrossRefGoogle Scholar
  77. 77.
    Karbab EB et al (2017) Android malware detection using deep learning on API method sequences. CoRR abs/1712.08996: n. PagGoogle Scholar
  78. 78.
    Karbab E, Debbabi M, Derhab A, Mouheb D (2018) MalDozer: automatic framework for android malware detection using deep learning. Digit Investig 24:S48–S59. Scholar
  79. 79.
    Yuan Z, Lu Y, Xue Y (2016) Droiddetector: android malware characterization and detection using deep learning. Tsinghua Sci Technol 21(1):114–123CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Biozid Bostami
    • 1
  • Mohiuddin Ahmed
    • 2
  1. 1.Islamic University of TechnologyDhakaBangladesh
  2. 2.Lecturer of Computing and Security, School of Science, Academic Centre of Cyber Security Excellence (ACCSE)Edith Cowan UniversityJoondalupAustralia

Personalised recommendations