Abstract
Reverse engineering of communication protocols is aimed at providing methods and tools allowing to infer a model of these protocols. It is very relevant for many application domains, such as interoperability or security audits. Recently, several tools have been developed in order to automate, entirely or partially, the protocol inference process. These tools rely on several techniques, that are usually tuned and adapted according to the final goal of the reverse engineering task. The aim of this paper is (1) to present an overview of the main challenges related to reverse engineering, and (2) to introduce the use of obfuscation techniques to make the reverse engineering process more complex and difficult in particular to malicious users.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Antunes, J., Neves, N., Verissimo, P.: Reverse engineering of protocols from network traces. In: 2011 18th Working Conference on Reverse Engineering (WCRE), pp. 169–178. IEEE, New York (2011)
Bossert, G., Hiet, G., Henin, T.: Modelling to simulate botnet command and control protocols for the evaluation of network intrusion detection systems. In: 2011 Conference on Network and Information Systems Security (SAR-SSI), pp. 1–8. IEEE, La Rochelle (2011)
Bossert, G.: Exploiting semantic for the automatic reverse engineering of communication protocols. Ph.D. Thesis, Suplec December 2014
Bridger, H., Rishab, N., Phillipa, G., Rob, J.: Games without frontiers: investigating video games as a covert channel. In: Proceedings of the 2016 IEEE European Symposium on Security and Privacy, IEEE European Symposium on Security and Privacy. IEEE (2015)
Caballero, J., Poosankam, P., Kreibich, C., Song, D.: Dispatcher: enabling active botnet infiltration using automatic protocol reverse-engineering. In: Proceedings of the 16th ACM Conference on Computer and Communications Security, CCS 2009, pp. 621–634. ACM, New York (2009)
Caballero, J., Song, D.: Rosetta: extracting protocol semantics using binary analysis with applications to protocol replay and NAT rewriting. Technical Report CMU-CyLab-07-014, Carnegie Mellon University, Pittsburgh, USA (2007)
Caballero, J., Yin, H., Liang, Z., Song, D.: Polyglot: automatic extraction of protocol message format using dynamic binary analysis. In: Proceedings of the 14th ACM Conference on Computer and Communications Security, CCS 2007, pp. 317–329. ACM, New York (2007)
Caballero Bayerri, J.: Grammar and model extraction for security applications using dynamic program binary analysis. Ph.D. Thesis, Carnegie Mellon University, Pittsburgh, PA, USA (2010)
Cui, W., Paxson, V., Weaver, N., Katz, R.H.: Protocol-independent adaptive replay of application dialog. In: Proceedings of the 13th Annual Network and Distributed System Security Symposium (NDSS). Internet Society, San Diego, USA, February 2006. http://research.microsoft.com/apps/pubs/default.aspx?id=153197
Cui, W., Peinado, M., Chen, K., Wang, H.J., Irun-Briz, L.: Tupni: automatic reverse engineering of input formats. In: Proceedings of the 15th ACM Conference on Computer and Communications Security, CCS 2008, pp. 391–402. ACM, New York (2008)
de la Higuera, C.: Grammatical Inference: Learning Automata and Grammars. Cambridge University Press, New York (2010)
Hjelmvik, E., John, W.: Breaking and Improving Protocol Obfuscation. Technical Report 2010–05, Chalmers University of Technology, Gothenburg, Sweden (2010). http://publications.lib.chalmers.se/cpl/record/index.xsql?pubid=123751
Leita, C., Mermoud, K., Dacier, M.: ScriptGen: an automated script generation tool for Honeyd. In: 21st Annual Computer Security Applications Conference, p. 214. IEEE, Tucson (2005)
Li, X., Chen, L.: A survey on methods of automatic protocol reverse engineering. In: 2011 Seventh International Conference on Computational Intelligence and Security (CIS), pp. 685–689. IEEE, Hainan (2011)
Newsome, J., Brumley, D., Franklin, J., Song, D.: Replayer: automatic protocol replay by binary analysis. In: Proceedings of the 13th ACM Conference on Computer and Communications Security, CCS 2006, pp. 311–321. ACM, New York (2006)
Samba Team: Opening windows to a wider world. http://www.samba.org
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Duchêne, J., Le Guernic, C., Alata, E., Nicomette, V., Kaâniche, M. (2017). Protocol Reverse Engineering: Challenges and Obfuscation. In: Cuppens, F., Cuppens, N., Lanet, JL., Legay, A. (eds) Risks and Security of Internet and Systems. CRiSIS 2016. Lecture Notes in Computer Science(), vol 10158. Springer, Cham. https://doi.org/10.1007/978-3-319-54876-0_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-54876-0_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-54875-3
Online ISBN: 978-3-319-54876-0
eBook Packages: Computer ScienceComputer Science (R0)