Differentiating Code from Data in x86 Binaries

  • Richard Wartell
  • Yan Zhou
  • Kevin W. Hamlen
  • Murat Kantarcioglu
  • Bhavani Thuraisingham
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6913)


Robust, static disassembly is an important part of achieving high coverage for many binary code analyses, such as reverse engineering, malware analysis, reference monitor in-lining, and software fault isolation. However, one of the major difficulties current disassemblers face is differentiating code from data when they are interleaved. This paper presents a machine learning-based disassembly algorithm that segments an x86 binary into subsequences of bytes and then classifies each subsequence as code or data. The algorithm builds a language model from a set of pre-tagged binaries using a statistical data compression technique. It sequentially scans a new binary executable and sets a breaking point at each potential code-to-code and code-to-data/data-to-code transition. The classification of each segment as code or data is based on the minimum cross-entropy. Experimental results are presented to demonstrate the effectiveness of the algorithm.


statistical data compression segmentation classification x86 binary disassembly 


  1. 1.
    Balakrishnan, G., Gruian, R., Reps, T., Teitelbaum, T.: CodeSurfer/x86—a platform for analyzing x86 executables. In: Proceedings of the 14th International Conference on Compiler Construction (CC), pp. 250–254 (2005)Google Scholar
  2. 2.
    Bratko, A., Cormack, G.V., Filipič, B., Lynam, T.R., Zupan, B.: Spam filtering using statistical data compression models. Journal of Machine Learning Research 7, 2673–2698 (2006)zbMATHMathSciNetGoogle Scholar
  3. 3.
    Cleary, J.G., Teahan, W.J.: Unbounded length contexts for PPM. The Computer Journal 40(2/3), 67–75 (1997)CrossRefGoogle Scholar
  4. 4.
    Cleary, J.G., Witten, I.H.: Data compression using adaptive coding and partial string matching. IEEE Transactions on Communications 32(4), 396–402 (1984)CrossRefGoogle Scholar
  5. 5.
    Cormack, G.V., Horspool, R.N.: Data compression using dynamic Markov modeling. The Computer Journal 30(6), 541–550 (1987)CrossRefGoogle Scholar
  6. 6.
    Eagle, C.: The IDA Pro Book: The Unofficial Guide to the World’s Most Popular Disassembler. No Starch Press, Inc., San Francisco (2008)Google Scholar
  7. 7.
    Erdélyi, G.: IDAPython: User scripting for a complex application. Bachelor’s thesis, EVTEK University of Applied Sciences (2008)Google Scholar
  8. 8.
    Hamlen, K.W., Mohan, V., Wartell, R.: Reining in Windows API abuses with in-lined reference monitors. Tech. Rep. UTDCS-18-10, The University of Texas at Dallas, Richardson, Texas (June 2010)Google Scholar
  9. 9.
    Hex-Rays: The IDA Pro disassembler and debugger (2011),
  10. 10.
    Hunt, G., Brubacher, D.: Detours: Binary interception of Win32 functions. In: Proceedings of the 3rd USENIX Windows NT Symposium (WINSYM), pp. 14–21 (1999)Google Scholar
  11. 11.
    Intel: Intel® 64 and IA-32 Architectures Software Developer’s Manual, vol. 2A & 2B: Instruction Set Reference. Intel Corporation (2011)Google Scholar
  12. 12.
    Kinder, J., Veith, H.: Jakstab: A static analysis platform for binaries. In: Gupta, A., Malik, S. (eds.) CAV 2008. LNCS, vol. 5123, pp. 423–427. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  13. 13.
    Kinder, J., Zuleger, F., Veith, H.: An abstract interpretation-based framework for control flow reconstruction from binaries. In: Jones, N.D., Müller-Olm, M. (eds.) VMCAI 2009. LNCS, vol. 5403, pp. 214–228. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  14. 14.
    Moffat, A., Turpin, A.: Compression and Coding Algorithms. Kluwer Academic Publishers, Boston (2002)CrossRefzbMATHGoogle Scholar
  15. 15.
    Teahan, W.J., Wen, Y., McNab, R.J., Witten, I.H.: A compression-based algorithm for Chinese word segmentation. Computational Linguistics 26(3), 375–393 (2000)CrossRefGoogle Scholar
  16. 16.
    Teahan, W.J.: Text classification and segmentation using minimum cross-entropy. In: Proceedings of the 6th International Conference on Computer-Assisted Information Retrieval (RIAO), pp. 943–961 (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Richard Wartell
    • 1
  • Yan Zhou
    • 1
  • Kevin W. Hamlen
    • 1
  • Murat Kantarcioglu
    • 1
  • Bhavani Thuraisingham
    • 1
  1. 1.Computer Science DepartmentUniversity of TexasDallasUSA

Personalised recommendations