Skip to main content
SpringerLink
Log in

Transparent Protection of Commodity OS Kernels Using Hardware Virtualization

  • Conference paper
Security and Privacy in Communication Networks (SecureComm 2010)

Abstract

Kernel rootkits are among the most insidious threats to computer security today. By employing various code injection techniques, they are able to maintain an omnipotent presence in the compromised OS kernels. Existing preventive countermeasures typically employ virtualization technology as part of their solutions. However, they are still limited in either (1) requiring modifying the OS kernel source code for the protection or (2) leveraging software-based virtualization techniques such as binary translation with a high overhead to implement a Harvard architecture (which is robust to various code injection techniques used by kernel rootkits). In this paper, we introduce hvmHarvard, a hardware virtualization-based Harvard architecture that transparently protects commodity OS kernels from kernel rootkit attacks and significantly reduces the performance overhead. Our evaluation with a Xen-based prototype shows that it can transparently protect legacy OS kernels with rootkit resistance while introducing < 5% performance overhead.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. W ∧ X, http://en.wikipedia.org/wiki/W_xor_X

  2. Rootkit Numbers Rocketing UP, McAfee Says (2006), http://news.cnet.com/2100-7349_3-6061878.html

  3. AMD Virtualization (AMD-V) Technology (2009), http://sites.amd.com/us/business/it-solutions/usage-models/virtualization/Pages/amd-v.aspx

  4. Cooperation Grows in Fight Against Cybercrime (2010), http://www.avertlabs.com/research/blog/index.php/category/rootkits-and-stealth-malware/

  5. Intel 64 and IA-32 Architectures Software Developers Manual, Volume 3B: System Programming Guide (2010), http://www.intel.com/assets/pdf/manual/253669.pdf

  6. Abadi, M., Budiu, M., Erlingsson, U., Ligatti, J.: Control-Flow Integrity Principles, Implementations, and Applications. ACM Transactions on Information and System Security 13(1), 1–40 (2009)

    Article  Google Scholar 

  7. Apache Http Server Project, http://httpd.apache.org/

  8. ab - Apache Benchmarking Tool, http://httpd.apache.org/docs/2.2/programs/ab.html

  9. Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T.L., Ho, A., Neugebauer, R., Pratt, I., Warfield, A.: Xen and the Art of Virtualization. In: SOSP 2003: Proceedings of the 19th ACM Symposium on Operating Systems Principles, pp. 164–177. ACM, New York (2003)

    Google Scholar 

  10. Buchanan, E., Roemer, R., Shacham, H., Savage, S.: When Good Instructions Go Bad: Generalizing Return-Oriented Programming to RISC. In: CCS 2008: Proceedings of the 15th ACM Conference on Computer and Communications Security, pp. 27–38. ACM, New York (2008)

    Chapter  Google Scholar 

  11. Castro, M., Costa, M., Harris, T.: Securing Software by Enforcing Data-Flow Integrity. In: OSDI 2006: Proceedings of the 7th Symposium on Operating Systems Design and Implementation, pp. 147–160. USENIX Association, Berkeley (2006)

    Google Scholar 

  12. Chen, X., Garfinkel, T., Lewis, E.C., Subrahmanyam, P., Waldspurger, C.A., Boneh, D., Dwoskin, J., Ports, D.R.: Overshadow: A Virtualization-based Approach to Retrofitting Protection in Commodity Operating Systems. In: ASPLOS XIII: Proceedings of the 13th International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 2–13. ACM, New York (2008)

    Chapter  Google Scholar 

  13. Dunlap, G.W., King, S.T., Cinar, S., Basrai, M.A., Chen, P.M.: ReVirt: Enabling Intrusion Analysis Through Virtual-Machine Logging and Replay. In: OSDI 2002: Proceedings of the 5th Symposium on Operating Systems Design and Implementation, pp. 211–224. ACM, New York (2002)

    Google Scholar 

  14. Garfinkel, T., Rosenblum, M.: A Virtual Machine Introspection Based Architecture for Intrusion Detection. In: Proceedings of the Network and Distributed Systems Security Symposium, pp. 191–206 (2003)

    Google Scholar 

  15. Grizzard, J.B.: Towards Self-Healing Systems: Re-establishing Trust in Compromised Systems. Ph.D. thesis, Georgia Institute of Technology (2006)

    Google Scholar 

  16. Hund, R., Holz, T., Freiling, F.C.: Return-Oriented Rootkits: Bypassing Kernel Code Integrity Protection Mechanisms. In: Security 2009: Proceedings of the 18th USENIX Security Symposium (2009)

    Google Scholar 

  17. Jiang, X., Wang, X., Xu, D.: Stealthy Malware Detection through VMM-based “Out-of-the-Box” Semantic View Reconstruction. In: CCS 2007: Proceedings of the 14th ACM Conference on Computer and Communications Security, pp. 128–138. ACM, New York (2007)

    Google Scholar 

  18. Klein, T.: ScoopyNG (2010), http://www.trapkit.de/research/vmm/scoopyng/

  19. Kortchinsky, K.: Honeypots: Counter Measures to VMware Fingerprinting (2004), http://seclists.org/lists/honeypots/2004/Jan-Mar/0015.html

  20. Liakh, S., Jiang, X.: [2/4,tip:x86/mm] Set First MB as RW+NX (2010), https://patchwork.kernel.org/patch/90048/

  21. Liakh, S., Jiang, X.: [3/4,tip:x86/mm] NX Protection for Kernel Data (2010), https://patchwork.kernel.org/patch/90046/

  22. Liakh, S., Jiang, X.: [4/4,tip:x86/mm] RO/NX Protection for Loadable Kernel Modules (2010), https://patchwork.kernel.org/patch/90047/

  23. Liston, T., Skoudis, E.: On the Cutting Edge: Thwarting Virtual Machine Detection (2006), http://handlers.sans.org/tliston/ThwartingVMDetection_Liston_Skoudis.pdf

  24. LMbench - Tools for Performance Analysis (1998), http://www.bitmover.com/lmbench/

  25. Lombardi, F., Di Pietro, R.: KvmSec: A Security Extension for Linux Kernel Virtual Machines. In: SAC 2009: Proceedings of the 2009 ACM Symposium on Applied Computing, New York, NY, pp. 2029–2034 (2009)

    Google Scholar 

  26. Murray, D.G., Milos, G., Hand, S.: Improving Xen Security through Disaggregation. In: VEE 2008: Proceedings of the 4th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, pp. 151–160. ACM, New York (2008)

    Google Scholar 

  27. Payne, B.D., Carbone, M., Sharif, M.I., Lee, W.: Lares: An Architecture for Secure Active Monitoring Using Virtualization. In: Oakland 2008: IEEE Symposium on Security and Privacy (S&P 2008), pp. 233–247. IEEE Computer Society, Los Alamitos (2008)

    Chapter  Google Scholar 

  28. Petroni Jr., N.L., Fraser, T., Molina, J., Arbaugh, W.A.: Copilot - A Coprocessor-based Kernel Runtime Integrity Monitor. In: Security 2004: Proceedings of the 13th USENIX Security Symposium, pp. 179–194. USENIX Association, Berkeley (2004)

    Google Scholar 

  29. Petroni, Jr., N.L., Fraser, T., Walters, A., Arbaugh, W.A.: An Architecture for Specification-based Detection of Semantic Integrity Violations in Kernel Dynamic Data. In: Security 2006: Proceedings of the 15th USENIX Security Symposium, pp. 289–304. USENIX Association, Berkeley (2006)

    Google Scholar 

  30. Petroni, Jr., N.L., Hicks, M.: Automated Detection of Persistent Kernel Control-Flow Attacks. In: CCS 2007: Proceedings of the 14th ACM Conference on Computer and Communications Security, pp. 103–115 (2007)

    Google Scholar 

  31. Riley, R., Jiang, X., Xu, D.: Guest-Transparent Prevention of Kernel Rootkits with VMM-Based Memory Shadowing. In: Lippmann, R., Kirda, E., Trachtenberg, A. (eds.) RAID 2008. LNCS, vol. 5230, pp. 1–20. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  32. Rosenblum, N.E., Cooksey, G., Miller, B.P.: Virtual Machine-provided Context Sensitive Page Mappings. In: VEE 2008: Proceedings of the 4th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, pp. 81–90. ACM, New York (2008)

    Google Scholar 

  33. Rutkowska, J.: Red Pill (2004), http://invisiblethings.org/papers/redpill.html

  34. Rutkowska, J.: System Virginity Verifier: Defining the Roadmap for Malware Detection on Windows System (2005), http://www.invisiblethings.org/papers/hitb05_virginity_verifier.ppt

  35. Rutkowska, J., Wojtczuk, R.: Qubes OS Architecture (2010), http://qubes-os.org/

  36. Seshadri, A., Luk, M., Qu, N., Perrig, A.: SecVisor: A Tiny Hypervisor to Provide Lifetime Kernel code Integrity for Commodity OSes. In: SOSP 2007: Proceedings of the 21st ACM SIGOPS Symposium on Operating Systems Principles, pp. 335–350. ACM, New York (2007)

    Google Scholar 

  37. Shacham, H.: The Geometry of Innocent Flesh on the Bone: Return-into-libc without Function Calls (on the x86). In: CCS 2007: Proceedings of the 14th ACM Conference on Computer and Communications Security, pp. 552–561. ACM, New York (2007)

    Google Scholar 

  38. Sharif, M.I., Lee, W., Cui, W., Lanzi, A.: Secure In-VM Monitoring Using Hardware Virtualization. In: CCS 2009: Proceedings of the 16th ACM Conference on Computer and Communications Security, pp. 477–487. ACM, New York (2009)

    Google Scholar 

  39. Sparks, S., Butler, J.: Shadow Walker.: Raising the Bar for Rootkit Detection. In: Black Hat Japan (2005)

    Google Scholar 

  40. Wang, Y.M., Beck, D., Vo, B., Roussev, R., Verbowski, C.: Detecting Stealth Software with Strider GhostBuster. In: DSN 2005: Proceedings of the 2005 International Conference on Dependable Systems and Networks, pp. 368–377. IEEE Computer Society, Los Alamitos (2005)

    Google Scholar 

  41. Wang, Z., Jiang, X.: HyperSafe: A Lightweight Approach to Provide Lifetime Hypervisor Control-Flow Integrity. In: Oakland 2010: IEEE Symposium on Security and Privacy (S&P 2010), pp. 380–398. IEEE Computer Society, Los Alamitos (2010)

    Chapter  Google Scholar 

  42. Wang, Z., Jiang, X., Cui, W., Ning, P.: Countering Kernel Rootkits with Lightweight Hook Protection. In: CCS 2009: Proceedings of the 16th ACM Conference on Computer and Communications Security, pp. 545–554. ACM, New York (2009)

    Google Scholar 

  43. Wang, Z., Jiang, X., Cui, W., Wang, X.: Countering Persistent Kernel Rootkits through Systematic Hook Discovery. In: Lippmann, R., Kirda, E., Trachtenberg, A. (eds.) RAID 2008. LNCS, vol. 5230, pp. 21–38. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  44. Wurster, G., Oorschot, P.C.v., Somayaji, A.: A Generic Attack on Checksumming-Based Software Tamper Resistance. In: Oakland 2005: Proceedings of the 2005 IEEE Symposium on Security and Privacy (S&P 2005), pp. 127–138. IEEE Computer Society, Los Alamitos (2005)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, North Carolina State University, USA

    Michael Grace, Zhi Wang, Deepa Srinivasan, Jinku Li, Xuxian Jiang & Siarhei Liakh

  2. School of Computing, National University of Singapore, Singapore

    Zhenkai Liang

Authors
  1. Michael Grace
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Deepa Srinivasan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jinku Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xuxian Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhenkai Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Siarhei Liakh
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Center for Secure Information Systems, George Mason University, 4400 University Drive, 22030-4422, Fairfax, VA, USA

    Sushil Jajodia

  2. Institute for Infocomm Research, 1 Fusionopolis Way, 21-01 Connexis, 138632, South Tower, Singapore

    Jianying Zhou

Rights and permissions

Reprints and permissions

Copyright information

© 2010 ICST Institute for Computer Science, Social Informatics and Telecommunications Engineering

About this paper

Cite this paper

Grace, M. et al. (2010). Transparent Protection of Commodity OS Kernels Using Hardware Virtualization. In: Jajodia, S., Zhou, J. (eds) Security and Privacy in Communication Networks. SecureComm 2010. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 50. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16161-2_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-16161-2_10

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-16160-5

  • Online ISBN: 978-3-642-16161-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation