On Detecting Code Reuse Attacks


Today, a code reuse technique is often used when exploiting software vulnerabilities, such as a buffer overflow. These attacks bypass the protection against execution of code in the stack, which is implemented on the hardware and software levels in modern information systems. The attacks are based on finding suitable sections of executable code–gadgets–in the vulnerable program and linking these gadgets into chains. The article proposes a method to protect applications against code reuse attacks. The method is based on detecting properties that distinguish between chains of gadgets and typical chains of legitimate program basic blocks. The appearance of an atypical chain of basic blocks during program execution may indicate the execution of a malicious code. One of the properties of a chain of gadgets is that at the end of the chain a special processor instruction used to call a function of the operating system is executed. Experiments are carried out for the x86/64 Linux operating system which show the importance of this property for detecting malicious code execution. An algorithm for identifying atypical chains is developed which makes it possible to detect all currently known code reuse techniques.

This is a preview of subscription content, access via your institution.

Fig. 1.
Fig. 2.


  1. 1

    Shacham, H., The geometry of innocent flesh on the bone: Return-into-libc without function calls (on the x86), Proceedings of the 14th ACM Conference on Computer and Communications Security, 2007, pp. 552–561.

  2. 2

    Buchanan, E., Roemer, R., Shacham, H., and Savage, S., When good instructions go bad: Generalizing return-oriented programming to RISC, Proceedings of the 15th ACM Conference on Computer and Communications Security, 2008, pp. 27–38.

  3. 3

    http://ropshell.com. Accessed November 26, 2018.

  4. 4

    Binlin, C., Jianming, F., and Zhiyi, Y., Heap spraying attack detection based on sled distance, Int. J. Digital Content Technol. Its Appl., 2012, vol. 6, no. 14, pp. 379–386.

    Article  Google Scholar 

  5. 5

    Davi, L., Sadeghi, A., and Winandy, M., ROPdefender: A detection tool to defend against return-oriented programming attacks, Proceedings of the 6th ACM Symposium on Information, Computer and Communications Security, 2011, pp. 40–51.

  6. 6

    Davi, L., Koeberl, P., and Sadeghi, A., Hardware-assisted fine-grained control-flow integrity: Towards efficient protection of embedded systems against software exploitation, Proceedings of the 51st Annual Design Automation Conference, San Francisco, CA, 2014, pp. 1–6.

  7. 7

    Ge, X., Talele, N., Payer, M., and Jaeger, T., Fine-grained control-flow integrity for kernel software, IEEE European Symposium on Security and Privacy, 2016, pp. 179–194.

  8. 8

    Usui, T., Ikuse, T., Iwamura, M., and Yada, T., POSTER: Static ROP chain detection based on hidden Markov model considering ROP chain integrity, Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, 2016, pp. 1808–1810.

  9. 9

    Cawan, S.C., Arnold, S.R., Beattie, S.M., and Wagle, P.M., Pointguard: Method and system for protecting programs against pointer corruption attacks, US Patent 7752459B2, 2010.

  10. 10

    Cheng, Y., Zhou, Z., Miao, Y., Ding, X., and Deng, H.R., ROPecker: A generic and practical approach for defending against ROP attack, Symposium on Network and Distributed System Security (NDSS), 2014, pp. 1–14.

  11. 11

    Chen, P., Xiao, H., Shen, X., Yin, X., Mao, B., and Xie, L., DROP: Detecting return-oriented programming malicious code, Lect. Notes Comput. Sci., 2009, vol. 5905, pp. 163–177.

    Article  Google Scholar 

  12. 12

    Control-flow Enforcement Technology Preview, 2017. https://software.intel.com/sites/default/files/managed/4d/2a/control-flow-enforcement-technology-preview.pdf. Accessed November 26, 2018.

  13. 13

    Checkoway, S., Davi, L., Dmitrienko, A., Sadeghi, A.R., Shacham, H., and Winandy, M., Return-oriented programming without returns, Proceedings of the 17th ACM Conference on Computer and Communications Security, 2010, pp. 559–572.

  14. 14

    Sadeghi, A., Niksefat, S., and Rostamipour, M., Pure-call oriented programming (PCOP): Chaining the gadgets using call instructions, J. Comput. Virol. Hacking Tech., 2018, vol. 14, no. 2, pp. 139–156.

    Article  Google Scholar 

  15. 15

    Yao, F., Chen, J., and Venkataramani, G., Jop-alarm: Detecting jump-oriented programming-based anomalies in applications, IEEE 31st International Conference on Computer Design (ICCD), 2013, pp. 467–470.

  16. 16

    Goktas, E., Athanasopoulos, E., Polychronakis, M., Bos, H., and Portokalidis, G., Size does matter: Why using gadget-chain length to prevent code-reuse attacks is hard, Proceedings of the 23rd USENIX Security Symposium, 2014, pp. 417–432.

  17. 17

    Carlini, N. and Wagner, D., ROP is still dangerous: Breaking modern defenses, SEC'14 Proceedings of the 23rd USENIX Conference on Security Symposium, 2014, pp. 385–399.

  18. 18

    Aho, A.V., Sethi, R., and Ullman, J.D., Compilers: Principles, Techniques, and Tools, Pearson Education, Inc., 1986.

    Google Scholar 

  19. 19

    Kayaalp, M., Schmitt, T., Nomani, J., Ponomarev, D., and Abu-Ghazaleh, N., Scrap: Architecture for signature-based protection from code reuse attacks, Proceedings of IEEE 19th International Symposium on High Performance Computer Architecture (HPCA2013), 2013, pp. 258–269.

  20. 20

    https://sploitfun.wordpress.com/2015/05/08/bypassing-aslr-part-iii/. Accessed December 6, 2018.

  21. 21

    Katoch, V., Bypassing ASLR/DEP. https://www.exploit-db.com/docs/english/17914-bypassing-aslrdep.pdf. Accessed December 6, 2018.

  22. 22

    Pappas, V., Polychronakis, M., and Keromytis, A.D., Transparent ROP exploit mitigation using indirect branch tracing, Proc. of the 22nd USENIX Security Symposium, 2013, pp. 447–462.

  23. 23

    https://www.securityfocus.com/bid/62780/info. Accessed December 3, 2018.

  24. 24

    Moser, A., Kruegel, C., and Kirda, E., Limits of static analysis for malware detection, Proceedings of Twenty-Third Annual Computer Security Applications Conference (ACSAC 2007), 2008, pp. 421–430.

  25. 25

    Hu, H., Shinde, S., Adrian, S., Chua, Z.L., Saxena, P., and Liang, Z., Data-oriented programming: On the expressiveness of non-control data attacks, Security and Privacy (SP) Symposium, 2016, pp. 969–986.

  26. 26

    Ma, H., Lu, K., Ma, X., Zhang, H., Jia, C., and Gao, D., Software watermarking using return-oriented programming, Proceedings of the 10th ACM Symposium on Information, Computer and Communications Security, 2015, pp. 369–380.

  27. 27

    Gao, D., Method for obfuscation of code using return oriented programming, WO Patent 2016126206A1, 2015.

  28. 28

    Lu, K., Xiong, S., and Gao, D., Ropsteg: Program steganography with return oriented programming, Proceedings of the 4th ACM Conference on Data and Application Security and Privacy, 2014, pp. 265–272.

Download references

Author information



Corresponding author

Correspondence to Y. V. Kosolapov.

Ethics declarations


The authors declare that they have no conflicts of interest.


Yury V. Kosolapov, orcid.org/0000-0002-1491-524X, PhD.

Additional information

Translated by O. Pismenov

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kosolapov, Y.V. On Detecting Code Reuse Attacks. Aut. Control Comp. Sci. 54, 573–583 (2020). https://doi.org/10.3103/S0146411620070111

Download citation


  • code reuse
  • software vulnerabilities