Adaptive Semantics-Aware Malware Classification

  • Bojan KolosnjajiEmail author
  • Apostolis Zarras
  • Tamas Lengyel
  • George Webster
  • Claudia Eckert
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9721)


Automatic malware classification is an essential improvement over the widely-deployed detection procedures using manual signatures or heuristics. Although there exists an abundance of methods for collecting static and behavioral malware data, there is a lack of adequate tools for analysis based on these collected features. Machine learning is a statistical solution to the automatic classification of malware variants based on heterogeneous information gathered by investigating malware code and behavioral traces. However, the recent increase in variety of malware instances requires further development of effective and scalable automation for malware classification and analysis processes.

In this paper, we investigate the topic modeling approaches as semantics-aware solutions to the classification of malware based on logs from dynamic malware analysis. We combine results of static and dynamic analysis to increase the reliability of inferred class labels. We utilize a semi-supervised learning architecture to make use of unlabeled data in classification. Using a nonparametric machine learning approach to topic modeling we design and implement a scalable solution while maintaining advantages of semantics-aware analysis. The outcomes of our experiments reveal that our approach brings a new and improved solution to the reoccurring problems in malware classification and analysis.



The research was supported by the German Federal Ministry of Education and Research under grant 16KIS0328 (IUNO) and by the Bavarian State Ministry of Education, Science and the Arts as part of the FORSEC research association.


  1. 1.
    The Cuckoo Sandbox.
  2. 2.
  3. 3.
    Alvarez, V.M.: Yara.
  4. 4.
    Anderson, B., Storlie, C., Lane, T.: Improving malware classification: bridging the static/dynamic gap. In: Workshop on Security and Artificial Intelligence (AISec) (2012)Google Scholar
  5. 5.
    Bailey, M., Oberheide, J., Andersen, J., Mao, Z.M., Jahanian, F., Nazario, J.: Automated classification and analysis of internet malware. In: Kruegel, C., Lippmann, R., Clark, A. (eds.) RAID 2007. LNCS, vol. 4637, pp. 178–197. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  6. 6.
    Bayer, U., Comparetti, P.M., Hlauschek, C., Kruegel, C., Kirda, E.: Scalable, behavior-based malware clustering. In: ISOC Network and Distributed System Security Symposium (NDSS) (2009)Google Scholar
  7. 7.
    Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)zbMATHGoogle Scholar
  8. 8.
    Chau, D.H., Nachenberg, C., Wilhelm, J., Wright, A., Faloutsos, C.: Polonium: tera-scale graph mining and inference for malware detection. In: SIAM International Conference on Data Mining (SDM) (2011)Google Scholar
  9. 9.
    Dahl, G.E., Stokes, J.W., Deng, L., Yu, D.: Large-scale malware classification using random projections and neural networks. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2013)Google Scholar
  10. 10.
    Dumais, S.T.: Latent semantic analysis. Ann. Rev. Inf. Sci. Technol. 38(1), 188–230 (2004)CrossRefGoogle Scholar
  11. 11.
    Dumitras, T., Shou, D.: Toward a standard benchmark for computer security research: the Worldwide Intelligence Network Environment (WINE). In: Workshop on Building Analysis Datasets and Gathering Experience Returns for Security (BADGERS) (2011)Google Scholar
  12. 12.
    Ester, M., Kriegel, H.-P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: Kdd (1996)Google Scholar
  13. 13.
    Garcia-Teodoro, P., Diaz-Verdejo, J., Maciá-Fernández, G., Vázquez, E.: Anomaly-based network intrusion detection: techniques, systems and challenges. Comput. Secur. 28(1), 18–28 (2009)CrossRefGoogle Scholar
  14. 14.
    Hanif, Z., Calhoun, T., Trost, J.: Binarypig: Scalable Static Binary Analysis Over Hadoop. Black Hat, USA (2013)Google Scholar
  15. 15.
    Hanif, Z., Lengyel, T.K., Webster, G.D.: Internet-Scale File Analysis. Black Hat, USA (2015)Google Scholar
  16. 16.
    Heller, K., Svore, K., Keromytis, A.D., Stolfo, S.: One class support vector machines for detecting anomalous windows registry accesses. In: Workshop on Data Mining for Computer Security (DMSEC) (2003)Google Scholar
  17. 17.
    Jang, J., Brumley, D., Venkataraman, S.: Bitshred: feature hashing malware for scalable triage and semantic analysis. In: Conference on Computer and Communications Security (CCS) (2011)Google Scholar
  18. 18.
    Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley, New York (2004)CrossRefzbMATHGoogle Scholar
  19. 19.
    Lengyel, T.K., Maresca, S., Payne, B.D., Webster, G.D., Vogl, S., Kiayias, A.: Scalability, fidelity and stealth in the Drakvuf dynamic malware analysis system. In: Annual Computer Security Applications Conference (ACSAC) (2014)Google Scholar
  20. 20.
    Leung, K., Leckie, C.: Unsupervised anomaly detection in network intrusion detection using clusters. In: Australasian Conference on Computer Science (2005)Google Scholar
  21. 21.
  22. 22.
    Newman, D., Chemudugunta, C., Smyth, P., Steyvers, M.: Analyzing entities and topics in news articles using statistical topic models. In: Mehrotra, S., Zeng, D.D., Chen, H., Thuraisingham, B., Wang, F.-Y. (eds.) ISI 2006. LNCS, vol. 3975, pp. 93–104. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  23. 23.
    Perdisci, R., U, M.C.: VAMO: towards a fully automated malware clustering validity analysis. In: Annual Computer Security Applications Conference (ACSAC) (2012)Google Scholar
  24. 24.
    Pfoh, J., Schneider, C., Eckert, C.: Leveraging string kernels for malware detection. In: Lopez, J., Huang, X., Sandhu, R. (eds.) NSS 2013. LNCS, vol. 7873, pp. 206–219. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  25. 25.
    Ramage, D., Hall, D., Nallapati, R., Manning, C.D.: Labeled LDA: a supervised topic model for credit attribution in multi-labeled corpora. In: Conference on Empirical Methods in Natural Language Processing (2009)Google Scholar
  26. 26.
    Řehůřek, R., Sojka, P.: Software framework for topic modelling with large corpora. In: Workshop on New Challenges for NLP Frameworks (2010)Google Scholar
  27. 27.
    Rieck, K., Holz, T., Willems, C., Düssel, P., Laskov, P.: Learning and classification of malware behavior. In: Zamboni, D. (ed.) DIMVA 2008. LNCS, vol. 5137, pp. 108–125. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  28. 28.
    Roberts, J.-M.: Virus Share.
  29. 29.
    Schultz, M.G., Eskin, E., Zadok, E., Stolfo, S.J.: Data mining methods for detection of new malicious executables. In: Symposium on Security and Privacy (2001)Google Scholar
  30. 30.
    Stringhini, G., Egele, M., Zarras, A., Holz, T., Kruegel, C., Vigna, G.: B@bel: leveraging email delivery for spam mitigation. In: USENIX Security Symposium (2012)Google Scholar
  31. 31.
    Tegeler, F., Fu, X., Vigna, G., Kruegel, C.: Botfinder: finding bots in network traffic without deep packet inspection. In: International Conference on Emerging Networking Experiments and Technologies (CoNEXT) (2012)Google Scholar
  32. 32.
    Teh, Y.W., Jordan, M.I., Beal, M.J., Blei, D.M.: Hierarchical Dirichlet processes. J. Am. Stat. Assoc. 101(476), 1566–1581 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    The MITRE Corporation. CRITS.
  34. 34.
    VirusTotal. File Statistics.
  35. 35.
    Wainwright, M.J., Jordan, M.I.: Graphical models, exponential families, and variational inference. Found. Trends Mach. Learn. 1, 1–305 (2008)CrossRefzbMATHGoogle Scholar
  36. 36.
    Wang, C., Paisley, J.W., Blei, D.M.: Online variational inference for the hierarchical Dirichlet process. In: International Conference on Artificial Intelligence and Statistics (2011)Google Scholar
  37. 37.
    Warrender, C., Forrest, S., Pearlmutter, B.: Detecting intrusions using system calls: alternative data models. In: Symposium on Security and Privacy (1999)Google Scholar
  38. 38.
    Wicherski, G.: Pehash: a novel approach to fast malware clustering. In: USENIX Workshop on Large-Scale Exploits and Emergent Threats (LEET) (2009)Google Scholar
  39. 39.
    Xiao, H., Eckert, C.: Efficient online sequence prediction with side information. In: IEEE International Conference on Data Mining (ICDM) (2013)Google Scholar
  40. 40.
    Xiao, H., Stibor, T.: A supervised topic transition model for detecting malicious system call sequences. In: Workshop on Knowledge Discovery, Modeling and Simulation (2011)Google Scholar
  41. 41.
    Zarras, A., Papadogiannakis, A., Gawlik, R., Holz, T.: Automated generation of models for fast and precise detection of HTTP-based malware. In: Annual Conference on Privacy, Security and Trust (PST) (2014)Google Scholar
  42. 42.
    Zhou, D., Bousquet, O., Lal, T.N., Weston, J., Schölkopf, B.: Learning with local and global consistency. Adv. Neural Inf. Process. Syst. 16(16), 321–328 (2004)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Bojan Kolosnjaji
    • 1
    Email author
  • Apostolis Zarras
    • 1
  • Tamas Lengyel
    • 1
  • George Webster
    • 1
  • Claudia Eckert
    • 1
  1. 1.Technical University of MunichMunichGermany

Personalised recommendations