Qualitative Survey-Based Content Analysis and Validation of Measures of Software Development Team Performance

  • Amar Nath Chatterjee
  • Duraipandian Israel
Part of the Communications in Computer and Information Science book series (CCIS, volume 141)


Performance measurement of software development teams is an elusive and complex issue. Many IT organizations have tried evolving their own measures without focus on reliability and validity. There is yet no widely accepted scale for measuring software development team performance (SDTP). An examination of available measurement models of team performance/team effectiveness throws up gaps that call for identification and synchronization of dimensions. Based on expert surveys, this paper attempts to identify and short-list a set of content-validated dimensions of SDTP. First, SPSS Text Analysis package was used to content-analyze 94 industry experts’ textual responses to an open-ended qualitative survey questionnaire, which led to extraction, categorization and short-listing of 34 measures of SDTP. Then followed another round of expert survey (N=30) that led to a distilled set of 20 content-validated measures of SDTP, based on Content Validity Ratios. This list of measures should help future research for SDTP scale development.


Software Development Team Performance (SDTP) Measure Dimension Latent Construct Multi-dimensional Construct (MDC) Unidimensional Construct (UDC) Instrument Scale Development SPSS Text Analyzer Qualitative Data Analysis Reliability Validity Content Validity Ratio (CVR) 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Balijepalli, V., Mahapatra, R., Nerur, S., Price, K.H.: Are two heads better than one for software development? The productivity paradox of pair programming. MIS Quarterly 33(1), 91–118 (2009)Google Scholar
  2. 2.
    Bose, P., Mishra, B.R.: Master technology first; be a manager later, says Infosys. Business Standard XXXV, 229 (2009)Google Scholar
  3. 3.
    Bradley, J.H., Hebert, F.J.: The effect of personality type on team performance. Journal of Management Development 16(5), 337–353 (1997)CrossRefGoogle Scholar
  4. 4.
    Brannick, M.T., Prince, C.: An overview of team performance measurement. In: Brannick, M.T., Salas, E., Prince, C. (eds.) Team Performance Assessment and Measurement: Theory, Methods, and Applications, ch. 1, pp. 3–16. Lawrence Erlbaum Associates, NJ (1997)Google Scholar
  5. 5.
    Brown, C.V., DeHayes, D.W., Hoffer, J.A., Martin, E.W., Perkins, W.C.: Managing Information Technology, 6th edn. Pearson / Prentice Hall, NJ (2009)Google Scholar
  6. 6.
    Chatterjee, A.N.: A critical evaluation of the instruments for measurement of software development team performance. Global Digital Business Review 3(1), 134–143 (2008) ISSN 1931-8146Google Scholar
  7. 7.
    Kuo, C.-C.: Research on impacts of team leadership on team effectiveness. The Journal of American Academy of Business, 266–277 (September 2004)Google Scholar
  8. 8.
    Doolen, T.L., Hacker, M.E., Aken, E.M.V.: The impact of organizational context on work team effectiveness: A study of production team. IEEE Transactions on Engineering Management 50(3), 285–296 (2003)CrossRefGoogle Scholar
  9. 9.
    Doolen, T.L., Hacker, M.E., Aken, E.M.V.: Managing organizational context for engineering team effectiveness. Team Performance Management 12(5/6), 138–154 (2006)CrossRefGoogle Scholar
  10. 10.
    Faraj, S., Sambamurthy, V.: Leadership of information systems development projects. IEEE Transactions on Engineering Management 53(2), 238–249 (2006)CrossRefGoogle Scholar
  11. 11.
    Forsyth, D.R.: Group Dynamics, 3rd edn. Wadsworth Publishing Company, CA (1998)Google Scholar
  12. 12.
    Gorla, N., Lam, Y.W.: Who should work with whom? Building effective software project teams. Communications of the ACM 47(6), 79–82 (2004)CrossRefGoogle Scholar
  13. 13.
    Hackman, J.R.: The design of work teams. In: Lorsch, J.W. (ed.) Handbook of Organizational Behavior, pp. 315–342. Prentice-Hall, Englewood Cliffs (1987)Google Scholar
  14. 14.
    Hardesty, D.M., Bearden, W.O.: The use of expert judges in scale development – implications for improving face validity of measures of unobservable constructs. Journal of Business Research 57(2), 98–107 (2004)CrossRefGoogle Scholar
  15. 15.
    Hayes, F.: Chaos is back. Computer World 38(45), 70 (2004)Google Scholar
  16. 16.
    Hoegl, M., Gemuenden, H.: Teamwork Quality and the Success of Innovative Projects: A Theoretical Concept and Empirical Evidence. Organizational Science 12(4), 435–449 (2001)CrossRefGoogle Scholar
  17. 17.
    Hui, A.K.T., Liu, D.B.: A Bayesian Belief Network Model and tool to evaluate risk and impact in software development projects, pp. 297–301. IEEE, Los Alamitos (2004)Google Scholar
  18. 18.
    Imbrie, P.K., Immekus, J.C., Maller, S.J.: Work in progress – a model to evaluate team effectiveness. In: 35th ASEE/IEEE Frontiers in Education Conference. Session T4F, pp. 12–13 (2005)Google Scholar
  19. 19.
    Kraiger, K., Wenzel, L.H.: Conceptual development and empirical evaluation of measures of shared mental models as indicators of team effectiveness. In: Brannick, M.T., Salas, E., Prince, C. (eds.) Team Performance Assessment and Measurement: Theory, Methods, and Applications, ch. 4, pp. 63–84. Lawrence Erlbaum Associates, NJ (1997)Google Scholar
  20. 20.
    Law, K.S., Wong, C.: Multidimensional constructs in structural equation analysis: an illustration using the job perception and job satisfaction constructs. Journal of Management 25(2), 143–160 (1999)CrossRefGoogle Scholar
  21. 21.
    Lawshe, C.H.: A quantitative approach to content validity. Personnel Psychology 28, 563–575 (1975)CrossRefGoogle Scholar
  22. 22.
    Lee, G., Xia, W.: Toward agile: an integrated analysis of quantitative and qualitative field data on software development agility. MIS Quarterly 34(1), 87–114 (2010)Google Scholar
  23. 23.
    May, T.A.: Demographics and IT team performance. Computer World 38(20), 21–22 (2004)Google Scholar
  24. 24.
    McIntyre, R.M., Salas, E.: Team performance in complex environments: what we have learned so far. In: Guzzo, R., Salas, E. (eds.) Team Effectiveness and Decision Making in Organizations, pp. 333–380. Jossey-Bass, San Francisco (1995)Google Scholar
  25. 25.
    Nidumolu, S.R.: The effect of coordination and uncertainty on software project performance: residual performance risk as an intervening variable. Information Systems Research 6(3), 191–219 (1995)CrossRefGoogle Scholar
  26. 26.
    Pattit, J.M., Wilemon, D.: Creating high-performing software development teams. R&D Management 35(4), 375–393 (2005)CrossRefGoogle Scholar
  27. 27.
    Pinto, J.K., Mantel Jr., S.J.: The causes of project failure. IEEE Transactions on Engineering Management 37(4), 269–276 (1990)CrossRefGoogle Scholar
  28. 28.
    Rico, R., Manzanares, M.S., Gil, F., Gibson, C.: Team Implicit Coordination Process: A Team Knowledge-Based Approach. Academy of Management Review 33(1), 163–184 (2008)CrossRefGoogle Scholar
  29. 29.
    Rubinstein, D.: Standish Group Report: There’s Less Development Chaos Today. Software Development Times (2007), (accessed on October 24, 2009)
  30. 30.
    Sandblad, B., Gulliksen, J., Aborg, C., Boivie, I., Persson, J., Goransson, B., Kavathatzopoulos, I., Blomkvist, S., Cajander, A.: Work environment and computer systems development. Behaviour & Information Technology 22(6), 375–387 (2003)CrossRefGoogle Scholar
  31. 31.
    Sethi, V., King, R.C., Quick, J.C.: What causes stress in information system professionals? Communications of the ACM 47(3), 99–102 (2004)CrossRefGoogle Scholar
  32. 32.
    Sonnentag, S., Frese, M., Brodbeck, F.C., Heinbokel, T.: Use of design methods, team leaders’ goal orientation, and team effectiveness: a follow-up study in software development projects. International Journal of Human-computer Interaction 9(4), 443–454 (1997)CrossRefGoogle Scholar
  33. 33.
    SPSS Inc.: SPSS Text Analysis for SurveysTM 2.0 User’s Guide. SPSS Inc., Chicago (2006) ISBN 1-56827-379-7Google Scholar
  34. 34.
    Stevens, K.T.: The effects of roles and personality characteristics on software development team effectiveness, Dissertation submitted to the Faculty of Virginia Polytechnic Institute and State University for Ph.D (approved), p. 127 (1998)Google Scholar
  35. 35.
    Sundstrom, E., De Meuse, K.P., Futrell, D.: Work teams: applications and effectiveness. American Psychology 45(2), 120–133 (1990)CrossRefGoogle Scholar
  36. 36.
    Tannenbaum, S.L., Beard, R.l., Salas, E.: Team building and its influence on team effectiveness: an examination of conceptual and empirical developments. In: Kelly, K. (ed.) Issues, Theory, and Research in Industrial / Organizational Psychology, pp. 117–153. Elsevier, Amsterdam (1992)CrossRefGoogle Scholar
  37. 37.
    Tesluk, P., Mathieu, J.E., Zaccaro, S.J., Marks, M.: Task and aggregation issues in the analysis and assessment of team performance. In: Brannick, M.T., Salas, E., Prince, C. (eds.) Team Performance Assessment and Measurement: Theory, Methods, and Applications, ch. 10, pp. 197–224. Lawrence Erlbaum Associates, NJ (1997)Google Scholar
  38. 38.
    Thamhain, H.J.: Effective project leadership in complex self-directed team environments. In: Proceedings of the 32nd Hawaii International Conference on System Sciences (HICSS 1999), p. 12. IEEE, Los Alamitos (1999)Google Scholar
  39. 39.
    Thite, M.: Leadership: A critical success factor in IT project management. Management of Engineering and Technology. In: Portland International Conference on Technology and Innovation Management “PICMET 1999”, vol. 1, pp. 298–303 (1999)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Amar Nath Chatterjee
    • 1
  • Duraipandian Israel
    • 1
  1. 1.XLRI JamshedpurIndia

Personalised recommendations