Skip to main content
SpringerLink
Log in

On the Effects of Pair Programming on Thoroughness and Fault-Finding Effectiveness of Unit Tests

  • Conference paper
Product-Focused Software Process Improvement (PROFES 2007)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 4589))

Abstract

Code coverage and mutation score measure how thoroughly tests exercise programs and how effective they are, respectively. The objective is to provide empirical evidence on the impact of pair programming on both, thoroughness and effectiveness of test suites, as pair programming is considered one of the practices that can make testing more rigorous, thorough and effective. A large experiment with MSc students working solo and in pairs was conducted. The subjects were asked to write unit tests using JUnit, and to follow test-driven development approach, as suggested by eXtreme Programming methodology. It appeared that branch coverage, as well as mutation score indicator (the lower bound on mutation score), was not significantly affected by using pair programming, instead of solo programming. However, slight but insignificant positive impact of pair programming on mutations score indicator was noticeable. The results do not support the positive impact of pair programming on testing to make it more effective and thorough. The generalization of the results is limited due to the fact that MSc students participated in the study. It is possible that the benefits of pair programming will exceed the results obtained in this experiment for larger, more complex and longer projects.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Williams, L., Kessler, R.: Pair Programming Illuminated. Addison-Wesley, London (2002)

    Google Scholar 

  2. Beck, K., Andres, C.: Extreme Programming Explained: Embrace Change, 2nd edn. Addison-Wesley, London (2004)

    Google Scholar 

  3. Williams, L., Kessler, R.R., Cunningham, W., Jeffries, R.: Strengthening the Case for Pair Programming. IEEE Software 17(4), 19–25 (2000)

    Article  Google Scholar 

  4. Williams, L.A., Kessler, R.R.: All I really need to know about pair programming I learned in kindergarten. Communications of the ACM 43(5), 108–114 (2000)

    Article  Google Scholar 

  5. Arisholm, E., Gallis, H., Dybå, T., Sjøberg, D.I.K.: Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise. IEEE Transactions on Software Engineering 33(2), 65–86 (2007)

    Article  Google Scholar 

  6. Beck, K.: Test Driven Development: By Example. Addison-Wesley, London (2002)

    Google Scholar 

  7. Kobayashi, O., Kawabata, M., Sakai, M., Parkinson, E.: Analysis of the Interaction between Practices for Introducing XP Effectively. In: ICSE 2006. Proceeding of the 28th International Conference on Software Engineering, pp. 544–550. ACM Press, New York, NY, USA (2006)

    Chapter  Google Scholar 

  8. Marick, B.: How to Misuse Code Coverage. In: Proceedings of the 16th International Conference on Testing Computer Software (1999), http://www.testing.com/writings/coverage.pdf

  9. Kaner, C.: Software Negligence and Testing Coverage. In: STAR 1996: Proceedings the 5th International Conference, Software Testing, Analysis and Review. pp. 299–327 (1996)

    Google Scholar 

  10. Cai, X., Lyu, M.R.: The Effect of Code Coverage on Fault Detection under Different Testing Profiles. SIGSOFT Softw. Eng. Notes 30(4), 1–7 (2005)

    Google Scholar 

  11. Cornett, S.: Code Coverage Analysis (Retrieved 2006), http://www.bullseye.com/coverage.html

  12. DeMillo, R.A., Lipton, R.J., Sayward, F.G.: Hints on Test Data Selection: Help for the Practicing Programmer. IEEE Computer 11(4), 34–41 (1978)

    Google Scholar 

  13. Hamlet, R.G.: Testing Programs with the Aid of a Compiler. IEEE Transactions on Software Engineering 3(4), 279–290 (1977)

    Article  MathSciNet  Google Scholar 

  14. Offutt, A.J., Untch, R.H.: Mutation 2000: Uniting the Orthogonal. In: Mutation testing for the new century, pp. 34–44. Kluwer Academic Publishers, Norwell, MA, USA (2001)

    Google Scholar 

  15. Zhu, H., Hall, P.A.V., May, J.H.R.: Software Unit Test Coverage and Adequacy. ACM Computing Surveys 29(4), 366–427 (1997)

    Article  Google Scholar 

  16. Walsh, P.J.: A Measure of Test Case Completeness. PhD thesis, Univ. New York (1985)

    Google Scholar 

  17. Frankl, P.G., Weiss, S.N., Hu, C.: All-Uses vs Mutation Testing: An Experimental Comparison of Effectiveness. Journal of Systems and Software 38(3), 235–253 (1997)

    Article  Google Scholar 

  18. Offutt, A.J., Pan, J., Tewary, K., Zhang, T.: An Experimental Evaluation of Data Flow and Mutation Testing. Software Practice and Experience 26(2), 165–176 (1996)

    Article  Google Scholar 

  19. Venners, B.: Test-Driven Development. A Conversation with Martin Fowler, Part V (Retrieved 2007), http://www.artima.com/intv/testdrivenP.html

  20. Moore, I.: Jester a JUnit test tester. In: Marchesi, M., Succi, G. (eds.) XP 2001: Proceedings of the 2nd International Conference on Extreme Programming and Flexible Processes in Software Engineering, pp. 84–87 (2001)

    Google Scholar 

  21. Gamma, E., Beck, K.: JUnit Project Home Page (Retrieved 2006), http://www.junit.org/

  22. Offutt, J., Ma, Y.S., Kwon, Y.R.: An Experimental Mutation System for Java. SIGSOFT Software Engineering Notes 29(5), 1–4 (2004)

    Google Scholar 

  23. Ma, Y.S., Offutt, J., Kwon, Y.R.: MuJava: A Mutation System for Java. In: ICSE 2006. Proceeding of the 28th International Conference on Software Engineering, New York, NY, USA, pp. 827–830. ACM Press, New York (2006)

    Chapter  Google Scholar 

  24. Chevalley, P., Thévenod-Fosse, P.: A mutation analysis tool for Java programs. International Journal on Software Tools for Technology Transfer (STTT) 5(1), 90–103 (2003)

    Article  Google Scholar 

  25. Madeyski, L., Radyk, N.: Judy mutation testing tool project (Retrieved 2007), http://www.e-informatyka.pl/sens/Wiki.jsp?page=Projects.Judy

  26. Offutt, A.J., Lee, A., Rothermel, G., Untch, R.H., Zapf, C.: An Experimental Determination of Sufficient Mutant Operators. ACM Transactions on Software Engineering and Methodology 5(2), 99–118 (1996)

    Article  Google Scholar 

  27. Ammann, P., Offutt, J.: Introduction to Software Testing. (In progress) (2008)

    Google Scholar 

  28. Ma, Y.S., Harrold, M.J., Kwon, Y.R.: Evaluation of Mutation Testing for Object-Oriented Programs. In: ICSE 2006. Proceeding of the 28th International Conference on Software Engineering, New York, NY, USA, pp. 869–872. ACM Press, New York (2006)

    Chapter  Google Scholar 

  29. Nosek, J.T.: The Case for Collaborative Programming. Communications of the ACM 41(3), 105–108 (1998)

    Article  Google Scholar 

  30. Nawrocki, J.R., Wojciechowski, A.: Experimental Evaluation of Pair Programming. In: Richardson, I., Abrahamsson, P., Messnarz, R. (eds.) Software Process Improvement. LNCS, vol. 3792, pp. 269–276. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  31. Nawrocki, J.R., Jasiński, M., Olek, L., Lange, B.: Pair Programming vs. Side-by-Side Programming. In: Richardson, I., Abrahamsson, P., Messnarz, R. (eds.) Software Process Improvement. LNCS, vol. 3792, pp. 28–38. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  32. Hulkko, H., Abrahamsson, P.: A Multiple Case Study on the Impact of Pair Programming on Product Quality. In: Inverardi, P., Jazayeri, M. (eds.) ICSE 2005, pp. 495–504. ACM Press, New York (2006)

    Google Scholar 

  33. Müller, M.M.: Are Reviews an Alternative to Pair Programming? Empirical Software Engineering 9(4), 335–351 (2004)

    Article  Google Scholar 

  34. Madeyski, L.: Preliminary Analysis of the Effects of Pair Programming and Test-Driven Development on the External Code Quality. In: Zieliński, K., Szmuc, T. (eds.) Software Engineering: Evolution and Emerging Technologies Frontiers in Artificial Intelligence and Applications. Frontiers in Artificial Intelligence and Applications, vol. 130, pp. 113–123. IOS Press, Amsterdam (2005)

    Google Scholar 

  35. Madeyski, L.: The Impact of Pair Programming and Test-Driven Development on Package Dependencies in Object-Oriented Design — An Experiment. In: Münch, J., Vierimaa, M. (eds.) PROFES 2006. LNCS, vol. 4034, pp. 278–289. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  36. Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering: An Introduction. Kluwer Academic Publishers, Norwell, MA, USA (2000)

    MATH  Google Scholar 

  37. Madeyski, L.: Is External Code Quality Correlated with Programming Experience or Feelgood Factor? In: Abrahamsson, P., Marchesi, M., Succi, G. (eds.) XP 2006. LNCS, vol. 4044, pp. 65–74. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  38. Shadish, W.R., Cook, T.D., Campbell, D.T.: Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin (2002)

    Google Scholar 

  39. Cenqua Pty Ltd: Clover project (Retrieved 2006), http://www.cenqua.com/clover/

  40. Cook, T.D., Campbell, D.T.: Quasi-Experimentation: Design and Analysis Issues. Houghton Mifflin Company (1979)

    Google Scholar 

  41. Sørumgård, L.S.: Verification of Process Conformance in Empirical Studies of Software Development. PhD thesis, The Norwegian University of Science and Technology (1997)

    Google Scholar 

  42. Kitchenham, B., Pfleeger, S.L., Pickard, L., Jones, P., Hoaglin, D.C., Emam, K.E., Rosenberg, J.: Preliminary Guidelines for Empirical Research in Software Engineering. IEEE Transactions on Software Engineering 28(8), 721–734 (2002)

    Article  Google Scholar 

  43. Höst, M., Regnell, B., Wohlin, C.: Using Students as Subjects — A Comparative Study of Students and Professionals in Lead-Time Impact Assessment. Empirical Software Engineering 5(3), 201–214 (2000)

    Article  MATH  Google Scholar 

  44. Tichy, W.F.: Hints for Reviewing Empirical Work in Software Engineering. Empirical Software Engineering 5(4), 309–312 (2000)

    Article  MathSciNet  Google Scholar 

  45. American Psychological Association: Publication manual of the American Psychological Association (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Applied Informatics, Wroclaw University of Technology, Wyb.Wyspianskiego 27, 50370 Wroclaw, Poland

    Lech Madeyski

Authors
  1. Lech Madeyski
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Jürgen Münch Pekka Abrahamsson

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Madeyski, L. (2007). On the Effects of Pair Programming on Thoroughness and Fault-Finding Effectiveness of Unit Tests. In: Münch, J., Abrahamsson, P. (eds) Product-Focused Software Process Improvement. PROFES 2007. Lecture Notes in Computer Science, vol 4589. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73460-4_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-73460-4_20

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-73459-8

  • Online ISBN: 978-3-540-73460-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation