Skip to main content
SpringerLink
Log in

Multi-stage Binary Code Obfuscation Using Improved Virtual Machine

  • Conference paper
Book cover Information Security (ISC 2011)

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 7001))

Included in the following conference series:

Abstract

A software obfuscator transforms a program into another executable one with the same functionality but unreadable code implementation. This paper presents an algorithm of multi-stage software obfuscation method using improved virtual machine techniques. The key idea is to iteratively obfuscate a program for many times in using different interpretations. An improved virtual machine (VM) core is appended to the protected program for byte-code interpretation. Adversaries will need to crack all intermediate results in order to figure out the structure of original code. Compared with existing obfuscators, our new obfuscator generates the protected code which performs more efficiently, and enjoys proven higher level security.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abadi, M., Plotkin, G.: On protection by layout randomization. In: 23rd IEEE Computer Security Foundations Symposium, pp. 337–351 (2010)

    Google Scholar 

  2. Anckaert, B., Madou, M., De Sutter, B., De Bus, B., De Bosschere, K., Preneel, B.: Program obfuscation: a quantitative approach. In: ACM Workshop on Quality of Protection, pp. 15–20 (2007)

    Google Scholar 

  3. Barak, B., Goldreich, O., Impagliazzo, R., Rudich, S., Sahai, A., Vadhan, S., Yang, K.: On the (Im)possibility of obfuscating programs. In: Kilian, J. (ed.) CRYPTO 2001. LNCS, vol. 2139, pp. 1–18. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  4. Beaucamps, P., Filiol, E.: On the possibility of practically obfuscating programs towards a unified perspective of code protection. Journal in Computer Virology 3, 3–21 (2007)

    Article  Google Scholar 

  5. Bitansky, N., Canetti, R.: On Strong Simulation and Composable Point Obfuscation. In: Rabin, T. (ed.) CRYPTO 2010. LNCS, vol. 6223, pp. 520–537. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  6. Canetti, R., Dakdouk, R.R.: Obfuscating Point Functions with Multibit Output. In: Smart, N.P. (ed.) EUROCRYPT 2008. LNCS, vol. 4965, pp. 489–508. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  7. Canetti, R., Tauman Kalai, Y., Varia, M., Wichs, D.: On Symmetric Encryption and Point Obfuscation. In: Micciancio, D. (ed.) TCC 2010. LNCS, vol. 5978, pp. 52–71. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  8. Cappaert, J., Preneel, B., Anckaert, B., Madou, M., De Bosschere, K.: Towards tamper resistant code encryption: Practice and experience. In: Chen, L., Mu, Y., Susilo, W. (eds.) ISPEC 2008. LNCS, vol. 4991, pp. 86–100. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  9. Ceccato, M., Di Penta, M., Nagra, J., Falcarin, P., Ricca, F., Torchiano, M., Tonella, P.: The effectiveness of source code obfuscation -an experimental assessment. In: The 17th IEEE International Conference on Program Comprehension (ICPC), pp. 178–187. IEEE Computer Society, Los Alamitos (2009)

    Google Scholar 

  10. Collberg, C.: Tutorial: code transformation techniques for software protection. In: ACM SIGPLAN 2009 Conference on Programming Language Design and Implementation, PLDI 2009 (2009)

    Google Scholar 

  11. Collberg, C., Thomborson, C.: Watermarking, tamper-proofing, and obfuscation - tools for software protection. IEEE Transactions on Software Engineering 28, 735–746 (2002)

    Article  Google Scholar 

  12. Collberg, C., Thomborson, C., Low, D.: A taxonomy of obfuscating transformations. Technical report (1997)

    Google Scholar 

  13. DataRescue. The ida pro disassembler and debugger (2005), http://www.hex-rays.com/idapro/

  14. Ge, J.: Control flow based obfuscation. In: Proceedings of the 5th ACM Workshop on Digital Rights Management (DRM), pp. 83–92. ACM Press, New York (2005)

    Chapter  Google Scholar 

  15. Goldweisser, S.: On the impossibility of obfuscation with auxiliary input, pp. 553–562. IEEE Computer Society, Los Alamitos (2005)

    Google Scholar 

  16. Hohenberger, S., Rothblum, G.N., Shelat, A., Vaikuntanathan, V.: Securely Obfuscating Re-encryption. In: Vadhan, S.P. (ed.) TCC 2007. LNCS, vol. 4392, pp. 233–252. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  17. Hohenberger, S., Waters, B.: Constructing Verifiable Random Functions with Large Input Spaces. In: Gilbert, H. (ed.) EUROCRYPT 2010. LNCS, vol. 6110, pp. 656–672. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  18. Jhala, R., Majumdar, R.: Path slicing. In: Proceedings of ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2005, pp. 38–47. ACM, New York (2005)

    Chapter  Google Scholar 

  19. Kanzaki, Y., Monden, A., Nakamura, M.: A software protection method based on instruction camouflage. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences (Japanese Edition) J87-A(6):755-767, 47–59 (2004)

    Google Scholar 

  20. Linn, C., Debray, S.: Obfuscation of executable code to improve resistance to static disassembly. In: ACM Conference on Computer and Communications Security (CCS), pp. 290–299. ACM Press, New York (2003)

    Google Scholar 

  21. Lynn, B., Prabhakaran, M., Sahai, A.: Positive Results and Techniques for Obfuscation. In: Cachin, C., Camenisch, J.L. (eds.) EUROCRYPT 2004. LNCS, vol. 3027, pp. 20–39. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  22. Madou, M., Anckaert, B., De Bus, B., De Bosschere, K.: On the effectiveness of source code transformations for binary obfuscation. In: Proc. of the Int’l Conf. on Software Engineering Research and Practice (SERP 2006), pp. 527–533 (2006)

    Google Scholar 

  23. Madou, M., Anckaert, B., Moseley, P., Debray, S.K., De Sutter, B., De Bosschere, K.: Software protection through dynamic code mutation. In: Song, J.-S., Kwon, T., Yung, M. (eds.) WISA 2005. LNCS, vol. 3786, pp. 194–206. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  24. Madou, M., Van Put, L., De Bosschere, K.: Understanding obfuscated code. In: 14th IEEE Int’l Conf. on Program Comprehension (ICPC), pp. 268–274 (2006)

    Google Scholar 

  25. Mit, M.E., Ernst, M.D.: Static and dynamic analysis: synergy and duality. In: WODA 2003: ICSE Workshop on Dynamic Analysis, pp. 24–27 (2003)

    Google Scholar 

  26. Monden, A., Monsifrot, A., Thomborson, C.: Security improvements for encrypted interpretation. In: Proc. 3rd Workshop on Application Specific Processors (WASP) Digest, pp. 19–26 (2004)

    Google Scholar 

  27. Naeem, N.A., Batchelder, M., Hendren, L.: Metrics for measuring the effectiveness of decompilers and obfuscator. In: 15th IEEE Int’l. Conf. on Program Comprehension, pp. 253–258 (2007)

    Google Scholar 

  28. Ogiso, T., Sakabe, Y., Soshi, M., Miyaji, A.: Software obfuscation on a theoretical basis and its implementation. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences E86-A(1), 176–186 (2003)

    Google Scholar 

  29. Popov, I.V., Debray, S.K., Andrews, G.R.: Binary obfuscation using signals. In: USENIX Security Symposium (2007)

    Google Scholar 

  30. Dalla Preda, M., Madou, M., De Bosschere, K., Giacobazzi, R.: Opaque Predicates Detection by Abstract Interpretation. In: Johnson, M., Vene, V. (eds.) AMAST 2006. LNCS, vol. 4019, pp. 81–95. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  31. Rolles, R.: X86 virtualizer (2008), http://rewolf.pl/

  32. Rolles, R.: Unpacking virtualization obfuscators. In: Proceedings of the 3rd USENIX Conference on Offensive Technologies, WOOT 2009, p. 1. USENIX Association (2009)

    Google Scholar 

  33. Schwarz, B., Debray, S.K., Andrews, G.R.: Disassembly of executable code revisited. In: 10th Working Conference on Reverse Engineering, pp. 45–54 (2002)

    Google Scholar 

  34. Sharif, M., Lanzi, A., Giffin, J., Lee, W.: Automatic reverse engineering of malware emulators. In: Proceedings of the 30th IEEE Symposium on Security and Privacy, pp. 94–109. IEEE Computer Society, Los Alamitos (2009)

    Google Scholar 

  35. Sivadasan, P., Sojan Lal, P.: Jconsthide: a framework for java source code constant hiding. CoRR (2009)

    Google Scholar 

  36. Smith, J.E., Nair, R.: Virtual machines: versatile platforms for systems and processes. Morgan Kaufmann, San Francisco (2005)

    MATH  Google Scholar 

  37. Oreans Technologies. Code virtualizer, http://oreans.com/codevirtualizer.php

  38. Udupa, S.K., Debray, S.K., Madou, M.: Deobfuscation: reverse engineering obfuscated code. In: 12th Working Conference on Reverse Engineering, pp. 45–54 (2005)

    Google Scholar 

  39. van Oorschot, P.C.: Revisiting Software Protection. In: Boyd, C., Mao, W. (eds.) ISC 2003. LNCS, vol. 2851, pp. 1–13. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  40. VMPsoft. Vmprotect software, http://www.vmprotect.ru/

  41. Wang, C., Hill, J., Knight, J.C., Davidson, J.W.: Protection of software-based survivability mechanism. In: Proceedings of the International Conference on Dependable Systems and Networks (formerly: FTCS), DSN 2001, pp. 193–202. IEEE Computer Society, Los Alamitos (2001)

    Chapter  Google Scholar 

  42. Wee, H.: On obfuscating point functions. In: Proceedings of the 37th Annual ACM Symposium on Theory of Computing, STOC 2005, pp. 523–532. ACM, New York (2005)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Infocomm Research, 1 Fusionpolis Way, 21-01, 138632, Singapore

    Hui Fang & Yongdong Wu

  2. Sumavision Soft Tech Co., Ltd., 15 Kaituo Road, Shangdi District, Beijing, 100085, China

    Shuhong Wang & Yin Huang

Authors
  1. Hui Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongdong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuhong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science and Engineering, Shanghai Jiao Tong University, 800 Dongchuan Road, 200240, Min Hang Shanghai, China

    Xuejia Lai

  2. Cryptography and Security Department, Institute for Infocomm Research, Singapore

    Jianying Zhou

  3. Key Laboratory of Computer Networks and Information, Security Xidian University, 2 South Taibai Road, Xi’an, 710071, Shaanxi, China

    Hui Li

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Fang, H., Wu, Y., Wang, S., Huang, Y. (2011). Multi-stage Binary Code Obfuscation Using Improved Virtual Machine. In: Lai, X., Zhou, J., Li, H. (eds) Information Security. ISC 2011. Lecture Notes in Computer Science, vol 7001. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24861-0_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-24861-0_12

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-24860-3

  • Online ISBN: 978-3-642-24861-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation