Skip to main content
SpringerLink
Log in

ROPOB: Obfuscating Binary Code via Return Oriented Programming

  • Conference paper
  • First Online:
Book cover Security and Privacy in Communication Networks (SecureComm 2017)

Abstract

Software reverse engineering has been widely employed for software reuse, serving malicious purposes, such as software plagiarism and malware camouflage. To raise the bar for adversaries to perform reverse engineering, plenty of work has been proposed to introduce obfuscation into the to-be-protected software. However, existing obfuscation methods are either inefficient or hard to be deployed. In this paper, we propose an obfuscation scheme for binaries based on Return Oriented Programming (ROP), which aims to serve as an efficient and deployable anti-reverse-engineering approach. Our basic idea is to transform direct control flow to indirect control flow. The strength of our scheme derives from the fact that static analysis is typically insufficient to pinpoint target address of indirect control flow. We implement a tool, ROPOB, to achieve obfuscation in Commercial-off-the-Shelf (COTS) binaries, and test ROPOB with programs in SPEC2006. The results show that ROPOB can successfully transform all identified direct control flow, without causing execution errors. The overhead is acceptable: the average performance overhead is less than 10% when obfuscation coverage is over 90%.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 143.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Alliance, B.S.: Global software privacy. http://globalstudy.bsa.org/2010/index.html

  2. Linn, C., Debray, S.: Obfuscation of executable code to improve resistance to static disassembly. In: 10th ACM Conference on Computer and Communications Security, pp. 290–299. ACM, October 2003

    Google Scholar 

  3. Debray, S.K., Popov, I.V., Andrews, G.R.: Binary obfuscation using signals. In: USENIX Security. USENIX, August 2007

    Google Scholar 

  4. Chen, H., et al.: Control flow obfuscation with information flow tracking. In: Proceedings of 42nd Annual IEEE/ACM International Symposium on Microarchitecture, pp. 391–400. ACM (2009)

    Google Scholar 

  5. Shacham, H.: The geometry of innocent flesh on the bone: return-into-libc without function calls (on the x86). In: Proceedings of the 14th ACM Conference on Computer and Communications Security, pp. 552–561. ACM (2007)

    Google Scholar 

  6. Lu, K., Xiong, S., Gao, D.: Ropsteg: program steganography with return oriented programming. In: Proceedings of 4th ACM Conference on Data and Application Security and Privacy (CODASPY), pp. 265–272. ACM (2014)

    Google Scholar 

  7. The IDA Pro disassembler and debugger. http://www.hex-rays.com/idapro/

  8. Backes, M., Nürnberger, S.: Oxymoron: making fine-grained memory randomization practical by allowing code sharing. In: Proceedings of 23rd Usenix Security Symposium, pp. 433–447 (2014)

    Google Scholar 

  9. Bletsch, T., Jiang, X., Freeh, V.W., Liang, Z.: Jump-oriented programming: a new class of code-reuse attack. In: Proceedings of 6th ACM Symposium on Information, Computer and Communications Security, pp. 30–40. ACM (2011)

    Google Scholar 

  10. Schwartz, E.J., Avgerinos, T., Brumley, D.: Q: exploit hardening made easy. In: USENIX Security Symposium (2011)

    Google Scholar 

  11. Lu, K., Zou, D., Wen, W., Gao, D.: Packed, printable, and polymorphic return-oriented programming. In: Sommer, R., Balzarotti, D., Maier, G. (eds.) RAID 2011. LNCS, vol. 6961, pp. 101–120. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23644-0_6

    Chapter  Google Scholar 

  12. Snow, K.Z., Monrose, F., Davi, L., Dmitrienko, A., Liebchen, C., Sadeghi, A.-R.: Just-in-time code reuse: on the effectiveness of fine-grained address space layout randomization. In: 2013 IEEE Symposium on Security and Privacy (SP), pp. 574–588. IEEE (2013)

    Google Scholar 

  13. Seibert, J., Okkhravi, H., Söderström, E.: Information leaks without memory disclosures: remote side channel attacks on diversified code. In: Proceedings of 2014 ACM SIGSAC Conference on Computer and Communications Security, pp. 54–65. ACM (2014)

    Google Scholar 

  14. Bittau, A., Belay, A., Mashtizadeh, A., Mazieres, D., Boneh, D.: Hacking blind. In: 2014 IEEE Symposium on Security and Privacy (SP), pp. 227–242. IEEE (2014)

    Google Scholar 

  15. Wang, T., Lu, K., Lu, L., Chung, S., Lee, W.: Jekyll on iOS: when benign apps become evil. In: Usenix Security, vol. 13 (2013)

    Google Scholar 

  16. Davi, L., Dmitrienko, A., Sadeghi, A.-R., Winandy, M.: Privilege escalation attacks on android. In: Burmester, M., Tsudik, G., Magliveras, S., Ilić, I. (eds.) ISC 2010. LNCS, vol. 6531, pp. 346–360. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-18178-8_30

    Chapter  Google Scholar 

  17. Lee, B., Lu, L., Wang, T., Kim, T., Lee, W.: From Zygote to Morula: fortifying weakened ASLR on android. In: 2014 IEEE Symposium on Security and Privacy (SP), pp. 424–439. IEEE (2014)

    Google Scholar 

  18. Cohen, F.B.: Operating system protection through program evolution. Comput. Secur. 12(6), 565–584 (1993)

    Article  Google Scholar 

  19. Xin, Z., Chen, H., Wang, X., Liu, P., Zhu, S., Mao, B., Xie, L.: Replacement attacks on behavior based software birthmark. In: Lai, X., Zhou, J., Li, H. (eds.) ISC 2011. LNCS, vol. 7001, pp. 1–16. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24861-0_1

    Chapter  Google Scholar 

  20. László, T., Kiss, Á.: Obfuscating C++ programs via control flow flattening. Annales Universitatis Scientarum Budapestinensis de Rolando Eötvös Nominatae, Sectio Computatorica 30, 3–19 (2009)

    MATH  Google Scholar 

  21. Sharif, M., Lanzi, A., Giffin, J., Lee, W.: Automatic reverse engineering of malware emulators. In: 2009 30th IEEE Symposium on Security and Privacy, pp. 94–109. IEEE (2009)

    Google Scholar 

  22. Wang, C.: A security architecture for survivability mechanisms. Ph.D. dissertation, University of Virginia (2001)

    Google Scholar 

  23. Oreans Technologies: Code virtualizer: total obfuscation against reverse engineering. http://www.oreans.com/codevirtualizer.php

  24. Oreans Technologies: Themida: advanced windows software protection system. http://www.oreans.com/themida.php

  25. VMProtect-Software: VMProtect - new-generation software protection. http://www.vmprotecct.ru/

  26. Christodorescu, M., Jha, S., Seshia, S.A., Song, D., Bryant, R.E.: Semantics-aware malware detection. In: 2005 IEEE Symposium on Security and Privacy, pp. 32–46. IEEE (2005)

    Google Scholar 

  27. Kruegel, C., Kirda, E., Mutz, D., Robertson, W., Vigna, G.: Polymorphic worm detection using structural information of executables. In: Valdes, A., Zamboni, D. (eds.) RAID 2005. LNCS, vol. 3858, pp. 207–226. Springer, Heidelberg (2006). https://doi.org/10.1007/11663812_11

    Chapter  Google Scholar 

  28. Lakhotia, A., Kumar, E.U., Venable, M.: A method for detecting obfuscated calls in malicious binaries. IEEE Trans. Softw. Eng. 31(11), 955–968 (2005)

    Article  Google Scholar 

  29. Singh, P.K., Moinuddin, M., Lakhotia, A.: Using static analysis and verification for analyzing virus and worm programs. In: Proceedings of 2nd European Conference on Information Warfare and Security, pp. 281–292 (2003)

    Google Scholar 

  30. Xu, Z., Miller, B.P., Reps, T.: Safety checking of machine code. In: ACM SIGPLAN Notices, vol. 35, pp. 70–82. ACM (2000)

    Article  Google Scholar 

  31. Guilfanov, I.: Decompilers and beyond. Black Hat USA (2008)

    Google Scholar 

  32. Schwartz, E.J., Lee, J., Woo, M., Brumley, D.: Native x86 decompilation using semantics-preserving structural analysis and iterative control-flow structuring. In: Proceedings of USENIX Security Symposium, p. 16 (2013)

    Google Scholar 

  33. Zeng, J., Fu, Y., Miller, K.A., Lin, Z., Zhang, X., Xu, D.: Obfuscation resilient binary code reuse through trace-oriented programming. In: Proceedings of 2013 ACM SIGSAC Conference on Computer & Communications Security, pp. 487–498. ACM (2013)

    Google Scholar 

  34. Wartell, R., Mohan, V., Hamlen, K.W., Lin, Z.: Binary stirring: self-randomizing instruction addresses of legacy x86 binary code. In: Proceedings of 2012 ACM Conference on Computer and Communications Security, pp. 157–168. ACM (2012)

    Google Scholar 

Download references

Acknowledgments

We thank the anonymous reviewers for their helpful feedback. This work was supported by Chinese National Natural Science Foundation 61272078.

Author information

Authors and Affiliations

  1. State Key Laboratory for Novel Software Technology, Department of Computer Science and Technology, Nanjing University, Nanjing, China

    Dongliang Mu, Jia Guo, Wenbiao Ding, Zhilong Wang & Bing Mao

  2. Zhengzhou University, Henan, China

    Lei Shi

Authors
  1. Dongliang Mu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jia Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenbiao Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhilong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bing Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lei Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dongliang Mu .

Editor information

Editors and Affiliations

  1. Wilfrid Laurier University, Waterloo, Ontario, Canada

    Xiaodong Lin

  2. University of New Brunswick, Fredericton, New Brunswick, Canada

    Ali Ghorbani

  3. University at Buffalo, Buffalo, New York, USA

    Kui Ren

  4. Pennsylvania State University, Philadelphia, Pennsylvania, USA

    Sencun Zhu

  5. Anhui Normal University, Wuhu, China

    Aiqing Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2018 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mu, D., Guo, J., Ding, W., Wang, Z., Mao, B., Shi, L. (2018). ROPOB: Obfuscating Binary Code via Return Oriented Programming. In: Lin, X., Ghorbani, A., Ren, K., Zhu, S., Zhang, A. (eds) Security and Privacy in Communication Networks. SecureComm 2017. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 238. Springer, Cham. https://doi.org/10.1007/978-3-319-78813-5_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-78813-5_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-78812-8

  • Online ISBN: 978-3-319-78813-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation