Abstract
In cloud environments, the typical response to a malware attack is to snapshot and shutdown the virtual machine (VM), and revert it to a prior state. This approach often leads to service disruption and loss of availability, which can have much more damaging consequences than the original attack. Critical evidence needed to understand and permanently remedy the original vulnerability may also be lost. In this work, we propose an alternative solution, which seeks to automatically identify and disable rootkit malware by restoring normal system control flows. Our approach employs virtual machine introspection (VMI), which allows a privileged VM to view and manipulate the physical memory of other VMs with the aid of the hypervisor. This opens up the opportunity to identify common attacks on the integrity of kernel data structures and code, and to restore them to their original state.
To produce an automated solution, we monitor a pool of VMs running the same kernel version to identify kernel invariants, and deviations from them, and use the observed invariants to restore the normal state of the kernel. In the process, we automatically handle address space layout randomization, and are able to protect critical kernel data structures and all kernel code. We evaluate a proof-of-concept prototype of the proposed system, called Nixer, against real-world malware samples in different scenarios. The results show that changes caused by the rootkits are properly identified and patched at runtime, and that the malware functionality has been disabled. We were able to repair kernel memory in all scenarios considered with no impairment of the functionality and minimal performance impact on the infected VMs.
This work was supported in part by the NSF grant # 1623276.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Basic\(\_\)6 rootkit (2016). https://github.com/bowlofstew/rootkit.com/tree/master/hoglund/basic_6
BinDiff (2016). https://www.zynamics.com/bindiff.html/
kBouncer (2016). http://www.cs.columbia.edu/~vpappas/papers/kbouncer.pdf
Libguestfs (2016). http://libguestfs.org/
LibVMI (2016). http://libvmi.com
Opdis (2016). http://mkfs.github.io/content/opdis/
Suterusu rootkit (2016). https://github.com/mncoppola/suterusu
Understanding and Defeating Windows 8.1 Kernel Patch Protection (2016). http://www.nosuchcon.org/talks/2014/D2_01_Andrea_Allievi_Win8.1_Patch_protections.pdf
Volatility (2016). http://www.volatilityfoundation.org/
Ahmed, I., Richard, G.G., Zoranic, A., Roussev, V.: Integrity checking of function pointers in kernel pools via virtual machine introspection. In: Desmedt, Y. (ed.) ISC 2013. LNCS, vol. 7807, pp. 3–19. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-27659-5_1
Ahmed, I., Zoranic, A., Javaid, S., Richard, G., Roussev, V.: Rule-based integrity checking of interrupt descriptor tables in cloud environments. In: Peterson, G., Shenoi, S. (eds.) DigitalForensics 2013. IAICT, vol. 410, pp. 305–328. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-41148-9_21
Ahmed, I., Zoranic, A., Javaid, S., Richard III, G.G.: ModChecker: kernel module integrity checking in the cloud environment. In: 2012 41st International Conference on Parallel Processing Workshops, pp. 306–313. IEEE (2012)
Bahram, S., Jiang, X., Wang, Z., Grace, M., Li, J., Srinivasan, D., Rhee, J., Xu, D.: DKSM: subverting virtual machine introspection for fun and profit. In: Proceedings of the 29th IEEE Symposium on Reliable Distributed Systems (2010)
Bauman, E., Ayoade, G., Lin, Z.: A survey on hypervisor-based monitoring: approaches, applications, and evolutions. ACM Comput. Surv. (CSUR) 48(1), 10 (2015)
Burow, N., Carr, S.A., Brunthaler, S., Payer, M., Nash, J., Larsen, P., Franz, M.: Control-flow integrity: precision, security, and performance. ACM Comput. Surv. 50, 16 (2016)
Criswell, J., Dautenhahn, N., Adve, V.: KCoFI: complete control-flow integrity for commodity operating system kernels. In: Proceedings of the IEEE Symposium on Security and Privacy (2014)
Garfinkel, T., Rosenblum, M., et al.: A virtual machine introspection based architecture for intrusion detection. In: NDSS, vol. 3, pp. 191–206 (2003)
Hoglund, G., Butler, J.: Rootkits: Subverting the Windows Kernel. Addison-Wesley Professional, Boston (2006)
Jain, B., Baig, M.B., Zhang, D., Porter, D.E., Sion, R.: SoK: introspections on trust and the semantic gap. In: 2014 IEEE Symposium on Security and Privacy, pp. 605–620. IEEE (2014)
Jiang, X., Wang, X., Xu, D.: Stealthy malware detection through VMM-based out-of-the-box semantic view reconstruction. In: Proceedings of the 14th ACM Conference on Computer and Communications Security, pp. 128–138. ACM (2007)
Win, T.Y., Tianfield, H., Mair, Q.: Detection of malware and kernel-level rootkits in cloud computing environments. In: 2nd IEEE International Conference on Cyber Security and Cloud Computing (CSCloud), pp. 295–300 (2015)
Yuan, P., Zeng, Q., Ding, X.: Hardware-assisted fine-grained code-reuse attack detection. In: Bos, H., Monrose, F., Blanc, G. (eds.) RAID 2015. LNCS, vol. 9404, pp. 66–85. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-26362-5_4
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Grimm, J., Ahmed, I., Roussev, V., Bhatt, M., Hong, M. (2018). Automatic Mitigation of Kernel Rootkits in Cloud Environments. In: Kang, B., Kim, T. (eds) Information Security Applications. WISA 2017. Lecture Notes in Computer Science(), vol 10763. Springer, Cham. https://doi.org/10.1007/978-3-319-93563-8_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-93563-8_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-93562-1
Online ISBN: 978-3-319-93563-8
eBook Packages: Computer ScienceComputer Science (R0)