Typestate Checking of Machine Code

  • Zhichen Xu
  • Thomas Reps
  • Barton P. Miller
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2028)


We check statically whether it is safe for untrusted foreign machine code to be loaded into a trusted host system. Our technique works on ordinary machine code, and mechanically synthesizes (and verifies) a safety proof. Our earlier work along these lines was based on a C-like type system, which does not suffice for machine code whose origin is C++ source code. In the present paper, we address this limitation with an improved typestate system and introduce several new techniques, including: summarizing the effects of function calls so that our analysis can stop at trusted boundaries, inferring information about the sizes and types of stack-allocated arrays, and a symbolic range analysis for propagating information about array bounds. These techniques make our approach to safety checking more precise, more efficient, and able to handle a larger collection of real-life code sequences than was previously the case.


Range Analysis Program Point Machine Code Access Permission Safety Check 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    M. Abadi, and L. Cardelli. A Theory of Objects. Monographs in Computer Science, D. Gries, and F. B. Schneider (Ed.). Springer-Verlag New York (1996).Google Scholar
  2. 2.
    R. Bodik, R. Gupta, and V. Sarkar. ABCD: Eliminating Array Bounds Checks on Demand. SIGPLAN Conference on Programming Language Design and Implementation. Vancouver B.C., Canada (June 2000).Google Scholar
  3. 3.
    S. Chandra, and T. Reps. Physical Type Checking for C. PASTE’ 99: SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering. Toulouse, France (September 1999).Google Scholar
  4. 4.
    D. R. Chase, M. Wegman, and F. Zadeck. Analysis of Pointers and Structures. SIGPLAN Conference on Programming Language Design and Implementation. New York, NY (1990).Google Scholar
  5. 5.
    B. Chatterjee, B. G. Ryder, and W. A. Landi. Relevant Context Inference. ACM Symposium on Principles of Programming Languages. San Antonio, TX (January 1999).Google Scholar
  6. 6.
    P. Cousot, R. Cousot: Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints. The 4th ACM Symposium on Principles of Programming Languages. Los Angeles, California (January 1977).Google Scholar
  7. 7.
    P. Cousot, and N. Halbwachs. Automatic Discovery of Linear Restraints Among Variables of a Program. Fifth Annual ACM Symposium on Principles of Programming Languages. Tucson, AZ (January 1978).Google Scholar
  8. 8.
    JavaSoft. Java Native Interface Specification. Release 1.1 (May 1997).Google Scholar
  9. 9.
    jPVM: A Native Methods Interface to PVM for the Java Platform. (2000).
  10. 10.
    A. Mycroft. Type-Based Decompilation (or Program Reconstruction via Type Reconstruction). 8th European Symposium on Programming, ESOP’99. Amsterdam, The Netherlands (March 1999).Google Scholar
  11. 11.
    B. P. Miller, M. D. Callaghan, J. M. Cargille, J. K. Hollingsworth, R. B. Irvin, K. L. Karavanic, K. Kunchithapadam, and T. Newhall.The Paradyn Parallel Performance Measurement Tools. IEEE Computer 28, 11 (November 1995).Google Scholar
  12. 12.
    R. Milner. A Theory of Type Polymorphism in Programming. Journal of Computer and System Sciences 17, 3 (1978).Google Scholar
  13. 13.
    R. Rivest. The MD5 Message-Digest Algorithm. Request for Comments: 1321. MIT Laboratory for Computer Science and RSA Data Security, Inc (April 1992).Google Scholar
  14. 14.
    R. Rugina and M. Rinard. Symbolic Bounds Analysis of Pointers, Array Indices, and Accessed Memory Regions. SIGPLAN Conference on Programming Language Design and Implementation. Vancouver B.C., Canada (June 2000).Google Scholar
  15. 15.
    M. Siff, S. Chandra, T. Ball, K. Kunchithapadam, and T. Reps. Coping with type casts in C. Seventh European Software Engineering Conference and Seventh ACM SIGSOFT Symposium on the Foundations of Software Engineering. Toulouse, France (September 1999).Google Scholar
  16. 16.
    N. P. Smith. Stack Smashing Vulnerabilities in the UNIX Operating System. (2000).
  17. 17.
    C. Small, and M. A. Seltzer. Comparison of OS Extension Technologies. USENIX 1996 Annual Technical Conference. San Diego, CA (January 1996).Google Scholar
  18. 18.
    F. Smith, D. Walker, and G. Morrisett. Alias Types. European Symposium on Programming. Berlin, Germany (March 2000).Google Scholar
  19. 19.
    N. Susuki, and K. Ishihata. Implementation of an Array Bound Checker. 4th ACM Symposium on Principles of Programming Languages. Los Angeles, CA (January 1977).Google Scholar
  20. 20.
    A. Tamches, and B. P. Miller. Fine-Grained Dynamic Instrumentation of Commodity Operating System Kernels. Third Symposium on Operating System Design and Implementation. New Orleans, LA (February 1999).Google Scholar
  21. 21.
    C. Verbrugge, P. Co, and L. Hendren. Generalized Constant Propagation A Study in C. 6th International Conference on Compiler Construction. Linköping, Sweden (April 1996).Google Scholar
  22. 22.
    P. Wadler. A taste of linear logic. Mathematical Foundations of Computer Science, Lecture Notes in Computer Science 711. Springer-Verlag. Gdansk, Poland (August 1993).Google Scholar
  23. 23.
    D. Wegner, J. Foster, E. Brewer, and A. Aiken. A First Step Towards Automated Detection of Buffer Overrun Vulnerabilities. The 2000 Network and Distributed Systems Security Conference. San Diego, CA (February 2000).Google Scholar
  24. 24.
    Z. Xu, B. P. Miller, and T. W. Reps. Safety Checking of Machine Code. SIGPLAN Conference on Programming Language Design and Implementation. Vancouver B.C., Canada (June 2000).Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Zhichen Xu
    • 1
  • Thomas Reps
    • 2
  • Barton P. Miller
    • 2
  1. 1.Hewlett-Packard LaboratoriesPalo Alto
  2. 2.University of Wisconsin at MadisonMadison

Personalised recommendations