Abstract
Malicious programs like the viruses, worms, Trojan horses, and backdoors infect host computers by taking advantage of flaws of the software and thereby introducing some kind of secret functionalities. The authors of these malicious programs attempt to find new methods to get avoided from detection engines. They use different obfuscation techniques such as dead code insertion, instruction substitution to make the malicious programs more complex. Initially, obfuscation techniques those are used by software developers to protect their software from piracy are now misused by these malware authors. This paper intends to detect such obfuscated programs or malware using control flow graph (CFG) matching technique, using VF2 algorithm. If the original CFG of the executable is found to be isomorphic to subgraph of obfuscated CFG (under examination), then it can be classified as an obfuscated one.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
You, I., Yim, K.: Malware obfuscation techniques: a brief survey. In: International Conference on Broadband, Wireless Computing, Communication and Applications, IEEE Computer Society, pp. 297–300 (2010)
Sharif, M., et al.: Impeding malware analysis using conditional code obfuscation. In: Network and Distributed System Security Symposium (2008)
Walenstein, A., Lakhotia, A.: A transformation-based model of malware derivation. In: 7th IEEE International Conference on Malicious and Unwanted Software, pp. 17–25 (2012)
Durfina, L., Kroustek, J., Zemek, P.: Psyb0t malware: a step-by-step decompilation—case study. In: Working Conference on Reverse Engineering (WCRE), pp. 449–456. IEEE Computer Society (2013)
Ernst, M., et al.: Quickly detecting relevant program invariants. In: 22nd International Conference on Software Engineering, pp. 449–458 (2000)
Cordella, L.P., Foggia, P., Sansone, C., Vento, M.: Evaluating performance of the VF graph matching algorithm. In: Proceedings of the 10th International Conference on Image Analysis and Processing, pp. 1172–1177. IEEE Computer Society Press (1999)
Cordella, L.P., Foggia, P., Sansone, C., Vento, M.: An improved algorithm for matching large graphs. In: 3rd International Workshop on Graph-based Representations, Italy (2001)
McKay, B.D.: Practical graph isomorphism. Congressus Numerantium 30, 45–87 (1981)
Messmer, B.T., Bunke, H.: A decision tree approach to graph and subgraph isomorphism detection. J. Pattern Recog. 32, 1979–1998 (1999)
Gold, R.: Reductions of control flow graphs. Int. J. Comput., Electr. Autom. Control Inf. Eng. 8(3), (2014)
Sadiq, W., Orlowska, M.E.: Analyzing process models using graph reduction techniques. Inf. Syst. 25(2), 117–134 (2000)
Bondy, J.A., Murty. U.S.R.: Graph Theory. Springer, Berlin (2008)
Abadi, M., Budiu, M., Erlingsson, U’., Ligatti, J.: Control flow integrity principles, implementations, and applications. ACM Trans. Inf. Syst. Secur. 13(1), 4:1–4:40 (2009)
Brunel, J., Doligez, D., Hansen, R.R., Lawall, J.L., Muller, G.: A foundation for flow-based program matching, using temporal logic and model checking POPL. ACM (2009)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Behera, C.K., Sanjog, G., Lalitha Bhaskari, D. (2019). Control Flow Graph Matching for Detecting Obfuscated Programs. In: Hoda, M., Chauhan, N., Quadri, S., Srivastava, P. (eds) Software Engineering. Advances in Intelligent Systems and Computing, vol 731. Springer, Singapore. https://doi.org/10.1007/978-981-10-8848-3_26
Download citation
DOI: https://doi.org/10.1007/978-981-10-8848-3_26
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-8847-6
Online ISBN: 978-981-10-8848-3
eBook Packages: EngineeringEngineering (R0)