Obfuscation Code Localization Based on CFG Generation of Malware

  • Nguyen Minh Hai
  • Mizuhito OgawaEmail author
  • Quan Thanh Tho
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9482)


This paper presents a tool BE-PUM (Binary Emulator for PUshdown Model generation), which generates a precise control flow graph (CFG), under presence of typical obfuscation techniques of malware, e.g., indirect jump, self-modification, overlapping instructions, and structured exception handler (SEH), which cover packers. Experiments are performed on 2000 real-world malware examples taken from VX Heaven and compare the results of a popular commercial disassembler IDA Pro, a state-of-the-art tool JakStab, and BE-PUM. It shows that BE-PUM correctly traces CFGs, whereas IDA Pro and JakStab fail. By manual inspection on 300 malware examples, we also observe that the starts of these failures exactly locate the entries of obfuscation code.


Concolic testing Binary code analysis Malware  Obfuscation 



This work is supported by JSPS KAKENHI Grant-in-Aid for Scientific Research(B) 15H02684 and AOARD-144050 (14IOA053). It is also funded by Ho Chi Minh City University of Technology under grant number TNCS-2015-KHMT-06.


  1. 1.
    Avgerinos, T., Rebert, A., Cha, S.K., Brumley, D.: Enhancing symbolic execution with veritesting. In: 36th ICSE, pp. 1083–1094 (2014)Google Scholar
  2. 2.
    Balakrishnan, G., Gruian, R., Reps, T., Teitelbaum, T.: CodeSurfer/x86—a platform for analyzing x86 executables. In: Bodik, R. (ed.) CC 2005. LNCS, vol. 3443, pp. 250–254. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  3. 3.
    Balakrishnan, G., Reps, T.: Wysinwyx: what you see is not what you execute. ACM TOPLAS 32(6), 206–263 (2010)CrossRefGoogle Scholar
  4. 4.
    Balakrishnan, G., Reps, T., Kidd, N., Lal, A.K., Lim, J., Melski, D., Gruian, R., Yong, S., Chen, C.-H., Teitelbaum, T.: Model checking x86 executables with CodeSurfer/x86 and WPDS++. In: Etessami, K., Rajamani, S.K. (eds.) CAV 2005. LNCS, vol. 3576, pp. 158–163. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  5. 5.
    Bardin, S., Herrmann, P.: OSMOSE: automatic structural testing of executables. In: STVR, pp. 29–54 (2011)Google Scholar
  6. 6.
    Bardin, S., Herrmann, P., Leroux, J., Ly, O., Tabary, R., Vincent, A.: The BINCOA framework for binary code analysis. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 165–170. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  7. 7.
    Bonfante, G., Marion, J.-Y.,-Plantey, D.R.: A computability perspective on self-modifying programs. In: SEFM, pp. 231–239 (2009)Google Scholar
  8. 8.
    Bonfante, G., Fernandez, J., Marion, J.-Y., Rouxel, B., Sabatier, F., Thierry, A.: CoDisasm: medium scale concatic disassembly of self-modifying binaries with overlapping instructions. In: CCS (2015, to appear)Google Scholar
  9. 9.
    Brumley, D., et al.: Automatically identifying trigger-based behavior in malware. In: Lee, W., Wang, C., Dagon, D. (eds.) Botnet Analysis and Defense, pp. 65–88. Springer, Heidelberg (2008)Google Scholar
  10. 10.
    Brumley, D., Jager, I., Avgerinos, T., Schwartz, E.J.: BAP: a binary analysis platform. In: Gopalakrishnan, G., Qadeer, S. (eds.) CAV 2011. LNCS, vol. 6806, pp. 463–469. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  11. 11.
    Kolbitsch, C., Livshits, B., Zorn, B.G., Seifert, C.: Rozzle: de-cloaking internet malware. In: IEEE Symposium on Security and Privacy, pp. 443–457 (2012)Google Scholar
  12. 12.
    Moser, A., et al.: Exploring multiple execution paths for malware analysis. In: IEEE Symposium on Security and Privacy, pp. 231–245 (2007)Google Scholar
  13. 13.
    Bitblaze, D.S., et al.: A new approach to computer security via binary analysis. In: ICISS (2008)Google Scholar
  14. 14.
    Peng, F., et al.: Force-executing binary programs for security applications. In: USENIX Security, pp. 829–844 (2014)Google Scholar
  15. 15.
    Filiol, E.: Malware pattern scanning schemes secure against black-box analysis. J. Comput. Virol. 2, 35–50 (2006)CrossRefGoogle Scholar
  16. 16.
    Holzer, A., Kinder, J., Veith, H.: Using verification technology to specify and detect malware. In: Moreno Díaz, R., Pichler, F., Quesada Arencibia, A. (eds.) EUROCAST 2007. LNCS, vol. 4739, pp. 497–504. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  17. 17.
    Kang, M., Poosankam, P., Yin, H.: Renovo: a hidden code extractor for packed executables. In: Recurring Malcode, pp. 46–53 (2007)Google Scholar
  18. 18.
    Kinder, J., Katzenbeisser, S., Schallhart, C., Veith, H.: Detecting malicious code by model checking. In: Julisch, K., Kruegel, C. (eds.) DIMVA 2005. LNCS, vol. 3548, pp. 174–187. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  19. 19.
    Kinder, J., et al.: Proactive detection of computer worms using model checking. IEEE Trans. Dependable Secure Comput. 7, 424–438 (2010)CrossRefGoogle Scholar
  20. 20.
    Kinder, J., Zuleger, F., Veith, H.: An abstract interpretation-based framework for control flow reconstruction from binaries. In: Jones, N.D., Müller-Olm, M. (eds.) VMCAI 2009. LNCS, vol. 5403, pp. 214–228. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  21. 21.
    Kinder, J.: Static analysis of x86 executables. Ph.D thesis, Technische Universitat Darmstadt (2010)Google Scholar
  22. 22.
    King, J.C.: Symbolic execution and program testing. CACM 19(7), 385–394 (1976)CrossRefzbMATHGoogle Scholar
  23. 23.
    Labir, E.: VX reversing I, the basics. CodeBreakers-J. 1(1), 17–47 (2004)Google Scholar
  24. 24.
    Lakhotia, A., Preda, M.D., Giacobazzi, R.: Fast location of similar code fragments using semantic ‘juice’. In: PPREW, pp. 25–30 (2013)Google Scholar
  25. 25.
    Moser, A., Kruegel, C., Kirda, E.: Limits of static analysis for malware detection. In: ACSAC, pp. 215–225 (2007)Google Scholar
  26. 26.
    Nanda, S., Li, W., Lam, L., Chiueh, T.: BIRD: binary interpretation using runtime disassembly. In: CGO, pp. 358–370 (2006)Google Scholar
  27. 27.
    Nguyen, M.H., Nguyen, T.B., Quan, T.T., Ogawa, M.: A hybrid approach for control flow graph construction from binary code. In: APSEC, pp. 159–164 (2013)Google Scholar
  28. 28.
    Godefroid, P., Lahiri, S.K., Rubio-González, C.: Statically validating must summaries for incremental compositional dynamic test generation. In: Yahav, E. (ed.) Static Analysis. LNCS, vol. 6887, pp. 112–128. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  29. 29.
    Preda, M.D., Giacobazzi, R., Lakhotia, A., Mastroeni, I.: Abstract symbolic automata: mixed syntactic/semantic similarity analysis of executables. In: POPL, pp. 329–341 (2015)Google Scholar
  30. 30.
    Roundy, K.A., Miller, B.P.: Binary-code obfuscations in prevalent packer tools. ACM Comput. Surv. 46, 215–226 (2014)Google Scholar
  31. 31.
    Sharif, M., Lanzi, A., Giffin, J., Lee, W.: Impeding malware analysis using conditional code obfuscation. In: NDSS (2008)Google Scholar
  32. 32.
    Song, F., Touili, T.: Pushdown model checking for malware detection. In: Flanagan, C., König, B. (eds.) TACAS 2012. LNCS, vol. 7214, pp. 110–125. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  33. 33.
    Song, F., Touili, T.: LTL model-checking for malware detection. In: Piterman, N., Smolka, S.A. (eds.) TACAS 2013 (ETAPS 2013). LNCS, vol. 7795, pp. 416–431. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  34. 34.
    Person, S., Dwyer, M.B, Elbaum, S.G.,Pasareanu, C.S.: Differential symbolic execution. In: SIGSOFT FSE, pp. 226–237 (2008)Google Scholar
  35. 35.
    Szor, P.: The Art of Computer Virus Research and Defense. Addison-Wesley Professional, Reading (2005)Google Scholar
  36. 36.
    Dullien, T., Kornau, T., Weinmann, R.-P.: A framework for automated architecture-independent gadget search. In: WOOT (2009)Google Scholar
  37. 37.
    Thakur, A., Lim, J., Lal, A., Burton, A., Driscoll, E., Elder, M., Andersen, T., Reps, T.: Directed proof generation for machine code. In: Touili, T., Cook, B., Jackson, P. (eds.) CAV 2010. LNCS, vol. 6174, pp. 288–305. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  38. 38.
    Izumida, T., Futatsugi, K., Mori, A.: A generic binary analysis method for malware. In: Echizen, I., Kunihiro, N., Sasaki, R. (eds.) IWSEC 2010. LNCS, vol. 6434, pp. 199–216. Springer, Heidelberg (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Nguyen Minh Hai
    • 1
  • Mizuhito Ogawa
    • 2
    Email author
  • Quan Thanh Tho
    • 1
  1. 1.Ho Chi Minh City University of TechnologyHo Chi Minh CityVietnam
  2. 2.Japan Advanced Institute of Science and TechnologyNomiJapan

Personalised recommendations