An Empirical Assessment on Affective Reactions of Novice Developers When Applying Test-Driven Development

  • Simone RomanoEmail author
  • Davide Fucci
  • Maria Teresa Baldassarre
  • Danilo Caivano
  • Giuseppe Scanniello
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11915)


We study whether and in which phase Test-Driven Development (TDD) influences affective states of novice developers in terms of pleasure, arousal, dominance, and liking. We performed a controlled experiment with 29 novice developers. Developers in the treatment group performed a development task using TDD, whereas those in the control group used a non-TDD development approach. We compared the affective reactions to the development approaches, as well as to the implementation and testing phases, exploiting a lightweight, powerful, and widely used tool, i.e., Self-Assessment Manikin. We observed that there is a difference between the two development approaches in terms of affective reactions. Therefore, it seems that affective reactions play an important role when applying TDD and their investigation could help researchers to better understand such a development approach.


Test-Driven Development TDD Affective state SAM 


  1. 1.
    Aniche, M.F., Ferreira, T.M., Gerosa, M.A.: What concerns beginner test-driven development practitioners: a qualitative analysis of opinions in an agile conference. In: Proceedings of Brazilian Workshop on Agile Methods. Springer (2011)Google Scholar
  2. 2.
    Beck, K.: Test-Driven Development: By Example. Addison-Wesley (2003)Google Scholar
  3. 3.
    Bergersen, G.R., Sjøberg, D.I.K., Dybå, T.: Construction and validation of an instrument for measuring programming skill. IEEE Trans. Softw. Eng. 40(12), 1163–1184 (2014) CrossRefGoogle Scholar
  4. 4.
    Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)CrossRefGoogle Scholar
  5. 5.
    Brunner, E., Dette, H., Munk, A.: Box-type approximations in nonparametric factorial designs. J. Am. Stat. Assoc. 92(440), 1494–1502 (1997)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Carver, J., Jaccheri, L., Morasca, S., Shull, F.: Issues in using students in empirical studies in software engineering education. In: Proceedings of International Symposium on Software Metrics, pp. 239–249. IEEE (2003)Google Scholar
  7. 7.
    Cliff, N.: Ordinal Methods for Behavioral Data Analysis. Psychology Press (1996)Google Scholar
  8. 8.
    Colomo-Palacios, R., Hernández-López, A., García-Crespo, Á., Soto-Acosta, P.: A study of emotions in requirements engineering. In: Lytras, M.D., Ordonez de Pablos, P., Ziderman, A., Roulstone, A., Maurer, H., Imber, J.B. (eds.) WSKS 2010. CCIS, vol. 112, pp. 1–7. Springer, Heidelberg (2010). Scholar
  9. 9.
    Csikszentmihalyi, M.: Finding Flow: The Psychology of Engagement with Everydaylife. Basic Books (1997)Google Scholar
  10. 10.
    Erdogmus, H., Morisio, M., Torchiano, M.: On the effectiveness of the test-first approach to programming. IEEE Trans. Softw. Eng. 31(3), 226–237 (2005)CrossRefGoogle Scholar
  11. 11.
    Ford, D., Parnin, C.: Exploring causes of frustration for software developers. In: Proceedings of International Workshop on Cooperative and Human Aspects of Software Engineering, pp. 115–116. IEEE (2015)Google Scholar
  12. 12.
    Fucci, D., Erdogmus, H., Turhan, B., Oivo, M., Juristo, N.: A dissection of the test-driven development process: does it really matter to test-first or to test-last? IEEE Trans. Softw. Eng. 43(7), 597–614 (2017)CrossRefGoogle Scholar
  13. 13.
    Fucci, D., et al.: A longitudinal cohort study on the retainment of test-driven development. In: Proceedings of International Symposium on Empirical Software Engineering and Measurement, pp. 18:1–18:10. ACM (2018)Google Scholar
  14. 14.
    Fucci, D., et al.: An external replication on the effects of test-driven development using a multi-site blind analysis approach. In: Proceedings of International Symposium on Empirical Software Engineering and Measurement, pp. 3:1–3:10. ACM (2016)Google Scholar
  15. 15.
    Gachechiladze, D., Lanubile, F., Novielli, N., Serebrenik, A.: Anger and its direction in collaborative software development. In: Proceedings of International Conference on Software Engineering: New Ideas and Emerging Technologies Results Track, pp. 11–14. IEEE (2017)Google Scholar
  16. 16.
    Girardi, D., Lanubile, F., Novielli, N., Fucci, D.: Sensing developers’ emotions: the design of a replicated experiment. In: Proceedings of International Workshop on Emotion Awareness in Software Engineering, pp. 51–54. IEEE (2018)Google Scholar
  17. 17.
    Graziotin, D., Fagerholm, F., Wang, X., Abrahamsson, P.: What happens when software developers are (un)happy. J. Syst. Softw. 140, 32–47 (2018)CrossRefGoogle Scholar
  18. 18.
    Graziotin, D., Wang, X., Abrahamsson, P.: Are happy developers more productive? In: Heidrich, J., Oivo, M., Jedlitschka, A., Baldassarre, M.T. (eds.) PROFES 2013. LNCS, vol. 7983, pp. 50–64. Springer, Heidelberg (2013). Scholar
  19. 19.
    Herbon, A., Peter, C., Markert, L., Van Der Meer, E., Voskamp, J.: Emotion studies in HCI-a new approach. In: Proceedings of International Conference on Human-Computer Interaction (2005)Google Scholar
  20. 20.
    Höst, M., Regnell, B., Wohlin, C.: Using students as subjects–a comparative study of students and professionals in lead-time impact assessment. Empirical Softw. Eng. 5(3), 201–214 (2000)CrossRefGoogle Scholar
  21. 21.
    Jedlitschka, A., Ciolkowski, M., Pfahl, D.: Guide to advanced empirical software engineering. In: Shull, F., Singer, J., Sjoberg, D.I.K. (eds.) Guide to Advanced Empirical Software Engineering, pp. 201–228. Springer, London (2008). Scholar
  22. 22.
    Jeffries, R., Melnik, G.: Guest editors’ introduction: TDD-the art of fearless programming. IEEE Softw. 24(3), 24–30 (2007)CrossRefGoogle Scholar
  23. 23.
    Kaptein, M.C., Nass, C., Markopoulos, P.: Powerful and consistent analysis of likert-type ratingscales. In: Proceedings of International Conference on Human Factors in Computing Systems, pp. 2391–2394. ACM (2010)Google Scholar
  24. 24.
    Karac, I., Turhan, B.: What do we (really) know about test-driven development? IEEE Softw. 35(4), 81–85 (2018)CrossRefGoogle Scholar
  25. 25.
    Khan, I.A., Brinkman, W.P., Hierons, R.M.: Do moods affect programmers’ debug performance? Cogn. Technol. Work 13(4), 245–258 (2011)CrossRefGoogle Scholar
  26. 26.
    Koelstra, S., et al.: Deap: a database for emotion analysis using physiological signals. IEEE Trans. Affect. Comput. 3(1), 18–31 (2012)CrossRefGoogle Scholar
  27. 27.
    Mäntylä, M., Adams, B., Destefanis, G., Graziotin, D., Ortu, M.: Mining valence, arousal, and dominance: possibilities for detecting burnout and productivity? In: Proceedings of International Conference on Mining Software Repositories, pp. 247–258. ACM (2016)Google Scholar
  28. 28.
    Müller, S.C., Fritz, T.: Stuck and frustrated or in flow and happy: sensing developers’ emotions and progress. In: International Conference on Software Engineering, vol. 1, pp. 688–699. IEEE (2015)Google Scholar
  29. 29.
    Munir, H., Moayyed, M., Petersen, K.: Considering rigor and relevance when evaluating test driven development: a systematic review. Inf. Softw. Technol. 56(4), 375–394 (2014)CrossRefGoogle Scholar
  30. 30.
    Murgia, A., Tourani, P., Adams, B., Ortu, M.: Do developers feel emotions? an exploratory analysis of emotions in software artifacts. In: Proceedings of Working Conference on Mining Software Repositories, pp. 262–271. ACM (2014)Google Scholar
  31. 31.
    Ortu, M., et al.: The emotional side of software developers in JIRA. In: Proceedings of International Conference on Mining Software Repositories, pp. 480–483. ACM (2016)Google Scholar
  32. 32.
    Rafique, Y., Mišić, V.B.: The effects of test-driven development on external quality and productivity: a meta-analysis. IEEE Trans. Softw. Eng. 39(6), 835–856 (2013)CrossRefGoogle Scholar
  33. 33.
    Romano, J., Kromrey, J., Coraggio, J., Skowronek, J.: Appropriate statistics for ordinal level data: should we really be using t-test and Cohen’sd for evaluating group differences on the NSSE and other surveys? In: Annual Meeting of the Florida Association of Institutional Research, pp. 1–3 (2006)Google Scholar
  34. 34.
    Romano, S., Fucci, D., Scanniello, G., Turhan, B., Juristo, N.: Results from an ethnographically-informed study in the context of test driven development. In: Proceedings of the International Conference on Evaluation and Assessment in Software Engineering, pp. 10:1–10:10. ACM (2016)Google Scholar
  35. 35.
    Romano, S., Fucci, D., Scanniello, G., Turhan, B., Juristo, N.: Findings from a multi-method study on test-driven development. Inf. Softw. Technol. 89, 64–77 (2017)CrossRefGoogle Scholar
  36. 36.
    Rudmann, D.S., McConkie, G.W., Zheng, X.S.: Eyetracking in cognitive state detection for HCI. In: Proceedings of international conference on Multimodal interfaces, pp. 159–163. ACM (2003)Google Scholar
  37. 37.
    Russell, J.A.: Core affect and the psychological construction of emotion. Psychol. Rev. 110(1), 145–172 (2003)CrossRefGoogle Scholar
  38. 38.
    Salman, I., Misirli, A.T., Juristo, N.: Are students representatives of professionals in software engineering experiments? In: International Conference on Software Engineering, vol. 1, pp. 666–676. IEEE (2015)Google Scholar
  39. 39.
    Turhan, B., Layman, L., Diep, M., Erdogmus, H., Shull, F.: How effective is test-driven development. In: Making Software: What Really Works, and Why We Believe It, pp. 207–217. O’Reilly Media (2010)Google Scholar
  40. 40.
    Wohlin, C., Runeson, P., Hst, M., Ohlsson, M.C., Regnell, B., Wessln, A.: Experimentation in Software Engineering. Springer, New York (2012). Scholar
  41. 41.
    Wrobel, M.R.: Emotions in the software development process. In: Proceedings of International Conference on Human System Interactions, pp. 518–523. IEEE (2013)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.University of BariBariItaly
  2. 2.University of HamburgHamburgGermany
  3. 3.University of BasilicataPotenzaItaly

Personalised recommendations