Prevent Kernel Return-Oriented Programming Attacks Using Hardware Virtualization

  • Tian Shuo
  • He Yeping
  • Ding Baozeng
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7232)


ROP attack introduced briefly in this paper is a serious threat to compute systems. Kernel ROP attack is great challenge to existing defenses because attackers have system privilege, little prerequisite to mount attacks, and the disability of existing countermeasures against runtime attacks. A method preventing kernel return-oriented programming attack is proposed, which creates a separated secret address space for control data taking advantage of VMM architecture. The secret address space is implemented as a shadow stack on the same host with the target OS facilited by hardware virtualization techniques. The experience result shows the performance overhead in our implementation is about 10% and acceptable in practical.


Return-Oriented Programming kernel attacks virtualization shadow stack 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Shacham, H.: The geometry of innocent flesh on the bone: return-into-libc without function calls (on the x86). In: Proceedings of the 14th ACM Conference on Computer and Communications Security, pp. 552–561 (2007)Google Scholar
  2. 2.
    Buchanan, E., Roemer, R., Shacham, H., Savage, S.: When good instructions go bad: generalizing return-oriented programming to RISC. In: Proceedings of the 15th ACM Conference on Computer and Communications Security, pp. 27–38 (2008)Google Scholar
  3. 3.
    Checkoway, S., A. J. F., Kantor, B., Halderman, J.A., Felten, E.W., Schacham, H.: Can DREs provide long-lasing security? The case of return-oriented programming and the AVC Advantage. USENIX/ACCURATE/IVAoSS (2009)Google Scholar
  4. 4.
    Kornau, T.: Return oriented programming for the ARM achitecture (2010)Google Scholar
  5. 5.
    Lidner, F.: Developments in Cisco IOS forensics (2009)Google Scholar
  6. 6.
    Dullien, T., Kornau, T., Weinmann, R.-P.: A framework for automated architecture-independent gadget search. In: Proceedings of the 4th USENIX Conference on Offensive Technologies, p. 1 (2010)Google Scholar
  7. 7.
    PaXTeam. Documentation for the PaX projectGoogle Scholar
  8. 8.
    Barrantes, E.G., Ackley, D.H., Palmer, T.S., Stefanovic, D., Zovi, D.D.: Randomized instruction set emulation to disrupt binary code injection attacks. In: Proceedings of the 10th ACM Conference on Computer and Communications Security, pp. 281–289 (2003)Google Scholar
  9. 9.
    Cowan, C., Pu, C., Maier, D., Hintony, H., Walpole, J., Bakke, P., Beattie, S., Grier, A., Wagle, P., Zhang, Q.: StackGuard: automatic adaptive detection and prevention of buffer-overflow attacks. In: Proceedings of the 7th Conference on USENIX Security Symposium, vol. 7, pp. 63–78 (1998)Google Scholar
  10. 10.
    Madan, B., Phoha, S., Trivedi, K.: StackOFFence: a technique for defending against buffer overflow attacks. In: Information Technology: Coding and Computing, ITCC 2005, pp. 656–661 (2005)Google Scholar
  11. 11.
    Tian Shuo, H.Y.: Ding Liping: SSGuard: a Nonlinear-enhanced Countermeasure against Stack-smashing Attacks. In: Proceedings of ICIMT 2010, vol. 1, pp. 427–433 (2010)Google Scholar
  12. 12.
    Strackx, R., Younan, Y., Philippaerts, P., Piessens, F., Lachmund, S., Walter, T.: Breaking the memory secrecy assumption. In: Proceedings of the Second European Workshop on System Security, pp. 1–8 (2009)Google Scholar
  13. 13.
    Shacham, H., Page, M., Pfaff, B., Goh, E.-J., Modadugu, N., Boneh, D.: On the effectiveness of address-space randomization. In: Proceedings of the 11th ACM Conference on Computer and communications Security, pp. 298–307 (2004)Google Scholar
  14. 14.
    Roglia, G.F., Martignoni, L., Paleari, R., Bruschi, D.: Surgically returning to randomized lib (c). In: Computer Security Applications Conference, pp. 60–69 (2009)Google Scholar
  15. 15.
    Le, L.: Payload already inside: data re-use for ROP exploits. Black Hat (2010)Google Scholar
  16. 16.
    Li, J., Wang, Z., Jiang, X., Grace, M., Bahram, S.: Defeating return-oriented rootkits with ”Return-Less” kernels. In: Proceedings of the 5th European Conference on Computer systems, pp. 195–208 (2010)Google Scholar
  17. 17.
    Chen, P., Xiao, H., Shen, X., Yin, X., Mao, B., Xie, L.: DROP: Detecting Return-Oriented Programming Malicious Code. In: Prakash, A., Sen Gupta, I. (eds.) ICISS 2009. LNCS, vol. 5905, pp. 163–177. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  18. 18.
    Davi, L., Sadeghi, A.-R., Winandy, M.: ROPdefender: a detection tool to defend against return-oriented programming attacks. In: Proceedings of the 6th ACM Symposium on Information, Computer and Communications Security, pp. 40–51 (2011)Google Scholar
  19. 19.
    Vladimir Kiriansky, D.B.: Saman Amarasinghe Secure Execution via Program Shepherding. In: 11th USENIX Security Symposium, pp. 191–206 (2002)Google Scholar
  20. 20.
    Intel. IA-32 Intel Architecture Software Developer’s Mannual Volume 3B: System Programming Guide, Part 1 (January 2006)Google Scholar
  21. 21.
    Sharif, M.I., Lee, W., Cui, W., Lanzi, A.: Secure in-VM monitoring using hardware virtualization. In: Proceedings of the 16th ACM Conference on Computer and Communications Security, pp. 477–487 (2009)Google Scholar
  22. 22.
  23. 23.
    Microsoft. Data Execution Prevention (2006)Google Scholar
  24. 24.
    Eto, H., Yoda, K.: Propolice: Improved stack-smashing attack detection. Transactions of Information Processing Society of Japan 43(12), 4034–4041 (2002)Google Scholar
  25. 25.
    Younan, Y., Pozza, D., Piessens, F., Joosen, W.: Extended protection against stack smashing attacks without performance loss. In: 22nd Annual Computer Security Applications Conference, ACSAC 2006, pp. 429–438 (2006)Google Scholar
  26. 26.
    Francillon, A., Perito, D., Castelluccia, C.: Defending embedded systems against control flow attacks. In: Proceedings of the First ACM Workshop on Secure Execution of Untrusted Code, pp. 19–26 (2009)Google Scholar
  27. 27.
    Frantzen, M., Shuey, M.: StackGhost: Hardware facilitated stack protection. In: SSYM 2001: Proceedings of the 10th Conference on USENIX Security Symposium, pp. 55–66 (2001)Google Scholar
  28. 28.
    Onarlioglu, K., Bilge, L., Lanzi, A., Balzarotti, D., Kirda, E.: G-Free: defeating return-oriented programming through gadget-less binaries. In: Proceedings of the 26th Annual Computer Security Applications Conference, pp. 49–58 (2010)Google Scholar
  29. 29.
    Davi, L., Sadeghi, A.-R., Winandy, M.: Dynamic integrity measurement and attestation: towards defense against return-oriented programming attacks. In: Proceedings of the 2009 ACM Workshop on Scalable Trusted Computing, pp. 49–54 (2009)Google Scholar
  30. 30.
    Abadi, M., Erlingsson, M.B., Ligatti, J.: Control-flow integrity principles, implementations, and applications. ACM Trans. Inf. Syst. Secur. 13(1), 1–40 (2009)CrossRefGoogle Scholar
  31. 31.
    Nick, L., Petroni, J., Hicks, M.: Automated detection of persistent kernel control-flow attacks. In: Proceedings of the 14th ACM Conference on Computer and Communications Security, pp. 103–115 (2007)Google Scholar
  32. 32.
    Riley, R., Jiang, X., Xu, D.: Guest-transparent prevention of kernel rootkits with vmm-based memory shadowing (2008)Google Scholar
  33. 33.
    Tzi-Cker Chiueh, F.-H.H.: RAD: a compile-time solution to buffer overflow attacks. icdcs. In: 21st IEEE International Conference on Distributed Computing Systems (ICDCS 2001), pp. 409–417 (2001)Google Scholar
  34. 34.
    Vendicator. Stack Shield: A ”stack smashing” technique protection tool for LinuxGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Tian Shuo
    • 1
    • 2
  • He Yeping
    • 1
  • Ding Baozeng
    • 1
    • 2
  1. 1.Institution of SoftwareChinese Academy of SciencesChina
  2. 2.Graduate University of Chinese Academy of SciencesChina

Personalised recommendations