RAMBO: Run-Time Packer Analysis with Multiple Branch Observation

  • Xabier Ugarte-PedreroEmail author
  • Davide Balzarotti
  • Igor Santos
  • Pablo G. Bringas
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9721)


Run-time packing is a technique employed by malware authors in order to conceal (e.g., encrypt) malicious code and recover it at run-time. In particular, some run-time packers only decrypt individual regions of code on demand, re-encrypting them again when they are not running. This technique is known as shifting decode frames and it can greatly complicate malware analysis. The first solution that comes to mind to analyze these samples is to apply multi-path exploration to trigger the unpacking of all the code regions. Unfortunately, multi-path exploration is known to have several limitations, such as its limited scalability for the analysis of real-world binaries. In this paper, we propose a set of domain-specific optimizations and heuristics to guide multi-path exploration and improve its efficiency and reliability for unpacking binaries protected with shifting decode frames.


Malware Unpacking Multi-path exploration 



We would like to thank the reviewers for their insightful comments and our shepherd Brendan Dolan-Gavitt for his assistance to improve the quality of this paper. This research was partially supported by the Basque Government under a pre-doctoral grant given to Xabier Ugarte-Pedrero.


  1. 1.
    Dinaburg, A., Royal, P., Sharif, M., Lee, W.: Ether: malware analysis via hardware virtualization extensions. In: Proceedings of the 15th ACM Conference on Computer and Communications Security, pp. 51–62. ACM (2008)Google Scholar
  2. 2.
    Royal, P., Halpin, M., Dagon, D., Edmonds, R., Lee, W.: Polyunpack: automating the hidden-code extraction of unpack-executing malware. In: Proceedings of the 22nd Annual Computer Security Applications Conference, pp. 289–300 (2006)Google Scholar
  3. 3.
    Kang, M., Poosankam, P., Yin, H.: Renovo: a hidden code extractor for packed executables. In: Proceedings of the 2007 ACM Workshop on Recurring Malcode, pp. 46–53 (2007)Google Scholar
  4. 4.
    Cesare, S., Xiang, Y.: Classification of malware using structured control flow. In: Proceedings of the Eighth Australasian Symposium on Parallel and Distributed Computing, vol. 107, pp. 61–70. Australian Computer Society, Inc. (2010)Google Scholar
  5. 5.
    Sharif, M., Yegneswaran, V., Saidi, H., Porras, P.A., Lee, W.: Eureka: a framework for enabling static malware analysis. In: Jajodia, S., Lopez, J. (eds.) ESORICS 2008. LNCS, vol. 5283, pp. 481–500. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  6. 6.
    Martignoni, L., Christodorescu, M., Jha, S.: Omniunpack: fast, generic, and safe unpacking of malware. In: Computer Security Applications Conference, 2007, ACSAC 2007, Twenty-Third Annual, pp. 431–441. IEEE (2007)Google Scholar
  7. 7.
    Coogan, K., Debray, S., Kaochar, T., Townsend, G.: Automatic static unpacking of malware binaries. In: 16th Working Conference on Reverse Engineering, 2009, pp. 167–176. IEEE (2009)Google Scholar
  8. 8.
    Ugarte-Pedrero, X., Balzarotti, D., Santos, I., Bringas, P.G.: [SoK] Deep packer inspection: a longitudinal study of the complexity of run-time packers. In: Proceedings of the IEEE Symposium on Security and Privacy. IEEE Computer Society, May 2015Google Scholar
  9. 9.
    Moser, A., Kruegel, C., Kirda, E.: Limits of static analysis for malware detection. In: Proceedings of the 23rd Annual Computer Security Applications Conference (ACSAC), pp. 421–430 (2007)Google Scholar
  10. 10.
    Brumley, D., Hartwig, C., Liang, Z., Newsome, J., Song, D., Yin, H.: Automatically identifying trigger-based behavior in malware. In: Lee, W., Wang, C., Dagon, D. (eds.) Botnet Detection, pp. 65–88. Springer, USA (2008)CrossRefGoogle Scholar
  11. 11.
    Brumley, D., Hartwig, C., Kang, M.G., Liang, Z., Newsome, J., Poosankam, P., Song, D., Yin, H.: Bitscope: Automatically dissecting malicious binaries. School of Computer Science, Carnegie Mellon University, Technical report CMU-CS-07-133 (2007)Google Scholar
  12. 12.
    Jia, C., Wang, Z., Lu, K., Liu, X., Liu, X.: Directed hidden-code extractor for environment-sensitive malwares. Phys. Procedia 24, 1621–1627 (2012)CrossRefGoogle Scholar
  13. 13.
    Peng, F., Deng, Z., Zhang, X., Xu, D., Lin, Z., Su, Z.: X-force: force-executing binary programs for security applications. In: Proceedings of the 2014 USENIX Security Symposium, San Diego, CA (2014)Google Scholar
  14. 14.
    Moser, A., Kruegel, C., Kirda, E.: Exploring multiple execution paths for malware analysis. In: IEEE Symposium on Security and Privacy, 2007, pp. 231–245. IEEE (2007)Google Scholar
  15. 15.
    Song, D., et al.: BitBlaze: a new approach to computer security via binary analysis. In: Sekar, R., Pujari, A.K. (eds.) ICISS 2008. LNCS, vol. 5352, pp. 1–25. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  16. 16.
    Schwartz, E.J., Avgerinos, T., Brumley, D.: All you ever wanted to know about dynamic taint analysis and forward symbolic execution (but might have been afraid to ask). In: IEEE Symposium on Security and Privacy 2010, pp. 317–331. IEEE (2010)Google Scholar
  17. 17.
    Cadar, C., Dunbar, D., Engler, D.R.: Klee: unassisted and automatic generation of high-coverage tests for complex systems programs. In: Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation (OSDI), vol. 8, pp. 209–224 (2008)Google Scholar
  18. 18.
    Chipounov, V., Kuznetsov, V., Candea, G.: S2e: a platform for in-vivo multi-path analysis of software systems. ACM SIGARCH Comput. Archit. News 39(1), 265–278 (2011)CrossRefGoogle Scholar
  19. 19.
    Brumley, D., Wang, H., Jha, S., Song, D.: Creating vulnerability signatures using weakest preconditions. In: 20th IEEE Computer Security Foundations Symposium (CSF), pp. 311–325. IEEE (2007)Google Scholar
  20. 20.
    Leino, K.R.M.: Efficient weakest preconditions. Inf. Process. Lett. 93(6), 281–288 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Clarke, E.M., Klieber, W., Nováček, M., Zuliani, P.: Model checking and the state explosion problem. In: Meyer, B., Nordio, M. (eds.) LASER 2011. LNCS, vol. 7682, pp. 1–30. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  22. 22.
    Bilge, L., Lanzi, A., Balzarotti, D.: Thwarting real-time dynamic unpacking. In: Proceedings of the 4th European Workshop on System Security, Article No. 5. ACM (2011)Google Scholar
  23. 23.
    Paleari, R., Martignoni, L., Roglia, G.F., Bruschi, D.: A fistful of red-pills: how to automatically generate procedures to detect cpu emulators. In: Proceedings of the USENIX Workshop on Offensive Technologies (WOOT), vol. 41, p. 86 (2009)Google Scholar
  24. 24.
    Deng, Z., Zhang, X., Xu, D.: Spider: stealthy binary program instrumentation and debugging via hardware virtualization. In: Proceedings of the 29th Annual Computer Security Applications Conference, pp. 289–298. ACM (2013)Google Scholar
  25. 25.
    Balzarotti, D., Cova, M., Karlberger, C., Kirda, E., Kruegel, C., Vigna, G.: Efficient detection of split personalities in malware. In: Network and Distributed System Security Symposium (NDSS) (2010)Google Scholar
  26. 26.
    Sharif, M.I., Lanzi, A., Giffin, J.T., Lee, W.: Impeding malware analysis using conditional code obfuscation. In: Network and Distributed System Security Symposium (NDSS) (2008)Google Scholar
  27. 27.
    Guo, F., Ferrie, P., Chiueh, T.C.: A study of the packer problem and its solutions. In: Lippmann, R., Kirda, E., Trachtenberg, A. (eds.) RAID 2008. LNCS, vol. 5230, pp. 98–115. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  28. 28.
    Stewart, J.: Ollybone: semi-automatic unpacking on ia-32. In: Proceedings of the 14th DEF CON Hacking Conference (2006)Google Scholar
  29. 29.
    Kim, H.C., Inoue, D., Eto, M., Takagi, Y., Nakao, K.: Toward generic unpacking techniques for malware analysis with quantification of code revelation. In: The 4th Joint Workshop on Information Security (2009)Google Scholar
  30. 30.
    Caballero, J., Johnson, N., McCamant, S., Song, D.: Binary code extraction and interface identification for security applications. In: Proceedings of the 17th Annual Network and Distributed System Security Symposium, ISOC, pp. 391–408 (2009)Google Scholar
  31. 31.
    Rolles, R.: Unpacking virtualization obfuscators. In: 3rd USENIX Workshop on Offensive Technologies (WOOT) (2009)Google Scholar
  32. 32.
    Sharif, M., Lanzi, A., Giffin, J., Lee, W.: Automatic reverse engineering of malware emulators. In: 30th IEEE Symposium on Security and Privacy, pp. 94–109. IEEE (2009)Google Scholar
  33. 33.
    Coogan, K., Lu, G., Debray, S.: Deobfuscation of virtualization-obfuscated software: a semantics-based approach. In: Proceedings of the 18th ACM Conference on Computer and Communications Security, pp. 275–284. ACM (2011)Google Scholar
  34. 34.
    Vasudevan, A., Yerraballi, R.: Cobra: fine-grained malware analysis using stealth localized-executions. In: IEEE Symposium on Security and Privacy, 15-pp (2006)Google Scholar
  35. 35.
    Kang, M.G., Yin, H., Hanna, S., McCamant, S., Song, D.: Emulating emulation-resistant malware. In: Proceedings of the 1st ACM Workshop on Virtual Machine Security, pp. 11–22. ACM (2009)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Xabier Ugarte-Pedrero
    • 1
    • 2
    Email author
  • Davide Balzarotti
    • 3
  • Igor Santos
    • 1
  • Pablo G. Bringas
    • 1
  1. 1.University of DeustoBilbaoSpain
  2. 2.Cisco Talos Security Intelligence and Research GroupSan JoseUSA
  3. 3.EurecomSophia AntipolisFrance

Personalised recommendations