Black-Box Testing of Grey-Box Behavior

  • Benjamin Tyler
  • Neelam Soundarajan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2931)


Object-oriented frameworks are designed to provide functionality common to a variety of applications. Developers use these frameworks in building their own specialized applications, often without having the source code of the original framework. Unfortunately, the interactions between the framework components and the new application code can lead to behaviors that could not be predicted even if valid black-box specifications were provided for the framework components. What is needed are grey-box specifications that include information about sequences of method calls made by the original framework code. Our focus is on how to test frameworks against such specifications, which requires the ability to monitor such method calls made by the framework during testing. The problem is that without the source code of the framework, we cannot resort to code instrumentation to track these calls. We develop an approach that allows us to do this, and demonstrate it on a simple case study.


Source Code Base Class Template Method Test Eater Framework Component 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alexander, R., Offutt, J.: Criteria for testing polymorphic relationships. In: Int. Symp. on Softw. Reliability Eng., pp. 15–23 (2000)Google Scholar
  2. 2.
    Barnett, M., Grieskamp, W., Kerer, C., Schulte, W., Szyperski, C., Tilmann, N., Watson, A.: Serious specifications for composing components. In: 6th ICSE Workshop on Component-based Software Engineering, pp. 1–6 (2003)Google Scholar
  3. 3.
    Binder, R.: Testing Object-Oriented Systems. Addison-Wesley, Reading (1999)Google Scholar
  4. 4.
    Booch, G., Rumbaugh, J., Jacobson, I.: The Unified Modeling Language User Guide. Addison-Wesley, Reading (1999)Google Scholar
  5. 5.
    Buchi, M., Weck, W.: The greybox approach: when blackbox specifications hide too much. Technical Report TUCS TR No. 297, Turku Centre for Computer Science (1999), available at
  6. 6.
    Chen, M.-H., Kao, H.M.: Testing object-oriented programs — an integrated approach. In: Int. Symp. on Softw. Reliability Eng., pp. 73–83 (1999)Google Scholar
  7. 7.
    Cheon, Y., Leavens, G.: A simple and practical approach to unit testing: The JML and JUnit way. In: Magnusson, B. (ed.) ECOOP 2002. LNCS, vol. 2374, pp. 231–255. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  8. 8.
    Duran, J., Ntafos, S.: An evaluation of random testing. IEEE Trans. on Software Eng. 10, 438–444 (1984)CrossRefGoogle Scholar
  9. 9.
    Fayad, M.E., Schmidt, D.C.: Special issue on object oriented application frameworks. Comm. of the ACM 40 (October 1997)Google Scholar
  10. 10.
    Froehlich, G., Hoover, H., Liu, L., Sorenson, P.: Hooking into object-oriented application frameworks. In: Proc. of 1997 Int. Conf. on Software Engineering, pp. 141–151. ACM, New York (1997)Google Scholar
  11. 11.
    Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design Patterns: Elements of Reusable OO Software. Addison-Wesley, Reading (1995)Google Scholar
  12. 12.
    Hamlet, D., Taylor, R.: Partition testing does not inspire confidence. IEEE Trans. on Software Eng. 16(12), 1402–1411 (1990)CrossRefMathSciNetGoogle Scholar
  13. 13.
    Johnson, R., Foote, B.: Designing reusable classes. Journal of OOP 1, 26–49 (1988)Google Scholar
  14. 14.
    Jones, C.: Systematic Software Development Using VDM. Prentice-Hall, Englewood Cliffs (1990)zbMATHGoogle Scholar
  15. 15.
    Liskov, B., Wing, J.: A behavioral notion of subtyping. ACM Trans. on Prog. Lang. and Systems 16, 1811–1841 (1994)CrossRefGoogle Scholar
  16. 16.
    Marinov, D., Khurshid, S.: TestEra: A novel framework for automated testing of Java programs. In: Proc. of 16th ASE. IEEE, Los Alamitos (2001)Google Scholar
  17. 17.
    McDaniel, R., McGregor, J.D.: Testing the polymorphic interactions between classes. Technical Report 94-103, Dept. of Computer Sc., Clemson University (1994)Google Scholar
  18. 18.
    Meyer, B.: Object-Oriented Software Construction. Prentice-Hall, Englewood Cliffs (1997)zbMATHGoogle Scholar
  19. 19.
    Owicki, S., Gries, D.: An axiomatic proof technique for parallel programs. Acta Informatica 6(1), 319–340 (1976)zbMATHCrossRefMathSciNetGoogle Scholar
  20. 20.
    Pree, W.: Meta patterns: a means for capturing the essentials of reusable OO design. In: Tokoro, M., Pareschi, R. (eds.) ECOOP 1994. LNCS, vol. 821, pp. 150–162. Springer, Heidelberg (1994)CrossRefGoogle Scholar
  21. 21.
    Rountev, A., Milanova, A., Ryder, B.: Fragment class analysis for testing of polymorphism in Java software. In: Int. Conf. on Softw. Eng., pp. 210–220 (2003)Google Scholar
  22. 22.
    Soundarajan, N., Fridella, S.: Framework-based applications: From incremental development to incremental reasoning. In: Frakes, W.B. (ed.) ICSR 2000. LNCS, vol. 1844, pp. 100–116. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  23. 23.
    Weyuker, E.: Testing component-based software: A cautionary tale. IEEE Software 15(5), 54–59 (1998)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Benjamin Tyler
    • 1
  • Neelam Soundarajan
    • 1
  1. 1.Computer and Information ScienceOhio State UniversityColumbusUSA

Personalised recommendations