Enablers and Inhibitors of Experimentation in Early-Stage Software Startups

  • Jorge MelegatiEmail author
  • Rafael Chanin
  • Xiaofeng Wang
  • Afonso Sales
  • Rafael Prikladnicki
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11915)


Software startups are temporary organizations that develop innovative software-intensive products or services. Despite of numerous successful stories, most startups fail. Several methodologies were proposed both in the scientific and commercial literature to improve their success rate, and a common element among them is the idea of experimentation. This concept was brought to software development as an approach focused on taking critical product assumptions as hypotheses and developing experiments to support or refute them. Although well-known methodologies are based on this idea, the literature shows that software startups still do not follow this approach. The goal of this paper is to identify the enablers and inhibitors of experimentation in early-stage software startups. To achieve the goal, we performed a multiple-case study of four software startups. The results comprise a set of enablers and inhibitors divided into the categories of individual, organizational context, and environment.


Software startups Experimentation Experiment-driven software development 



This work is partially funded by FAPERGS (17/2551-0001/205-4).


  1. 1.
    Ahrend, J.M.: Requirements Elicitation in Startup Companies. Research Topics in HCI (2013)Google Scholar
  2. 2.
    Andries, P., Debackere, K., van Looy, B.: Simultaneous experimentation as a learning strategy: business model development under uncertainty. Strateg. Entrep. J. 7(4), 288–310 (2013)CrossRefGoogle Scholar
  3. 3.
    Berg, V., Birkeland, J., Nguyen-Duc, A., Pappas, I.O., Jaccheri, L.: Software startup engineering: a systematic mapping study. J. Syst. Softw. 144(February), 255–274 (2018)CrossRefGoogle Scholar
  4. 4.
    Blank, S.: The Four Steps to the Epiphany: Successful Strategies for Products that Win., Louisville (2007)Google Scholar
  5. 5.
    Bortolini, R.F., Nogueira Cortimiglia, M., Danilevicz, A.D.M.F., Ghezzi, A.: Lean Startup: a comprehensive historical review. Manag. Decis. (August) (2018).
  6. 6.
    Bosch, J., Holmström Olsson, H., Björk, J., Ljungblad, J.: The early stage software startup development model: a framework for operationalizing lean principles in software startups. In: Fitzgerald, B., Conboy, K., Power, K., Valerdi, R., Morgan, L., Stol, K.-J. (eds.) LESS 2013. LNBIP, vol. 167, pp. 1–15. Springer, Heidelberg (2013). Scholar
  7. 7.
    Coleman, G., O’Connor, R.V.: An investigation into software development process formation in software start-ups. J. Enterp. Inf. Manag. 21(6), 633–648 (2008)CrossRefGoogle Scholar
  8. 8.
    Cruzes, D.S., Dyba, T.: Recommended steps for thematic synthesis in software engineering. In: 2011 International Symposium on Empirical Software Engineering and Measurement, vol. 7491, pp. 275–284 (2011).
  9. 9.
    Cukier, D., Kon, F.: A maturity model for software startup ecosystems. J. Innov. Entrep. 7(1), 14 (2018)CrossRefGoogle Scholar
  10. 10.
    Eisenhardt, K.M.: Building theories from case study research. Acad. Manag. Rev. 14(4), 532–550 (1989)CrossRefGoogle Scholar
  11. 11.
    Eisenhardt, K.M., Tabrizi, B.N.: Accelerating adaptive processes: product innovation in the global computer industry. Adm. Sci. Q. 40(1), 84 (1995)CrossRefGoogle Scholar
  12. 12.
    Fabijan, A., et al.: Experimentation growth: evolving trustworthy A/B testing capabilities in online software companies. J. Softw. Evol. Process. 30(12), e2113 (2018). (December 2017)CrossRefGoogle Scholar
  13. 13.
    Fabijan, A., Olsson, H.H., Bosch, J.: Customer feedback and data collection techniques in software R&D: a literature review. In: Fernandes, J., Machado, R., Wnuk, K. (eds.) ICSOB. Lecture Notes in Business Information Processing, vol. 210. Springer, Cham (2015). Scholar
  14. 14.
    Festinger, L.: A Theory of Cognitive Dissonance, vol. 2. Stanford university press, Redwood City (1957)Google Scholar
  15. 15.
    Frederiksen, D.L., Brem, A.: How do entrepreneurs think they create value? A scientific reflection of Eric Ries’ Lean startup approach. Int. Entrep. Manag. J. 13(1), 169–189 (2017)CrossRefGoogle Scholar
  16. 16.
    Garcia, R., Calantone, R.: A critical look at technological innovation typology and innovativeness terminology: a literature review. J. Product Innov. Manag. 19(2), 110–132 (2002)CrossRefGoogle Scholar
  17. 17.
    Giardino, C., Wang, X., Abrahamsson, P.: Why early-stage software startups fail: a behavioral framework. In: Lassenius, C., Smolander, K. (eds.) ICSOB 2014. LNBIP, vol. 182, pp. 27–41. Springer, Cham (2014). Scholar
  18. 18.
    Gutbrod, M., Münch, J., Tichy, M.: How do software startups approach experimentation? empirical results from a qualitative interview study. In: Felderer, M., Méndez Fernández, D., Turhan, B., Kalinowski, M., Sarro, F., Winkler, D. (eds.) PROFES 2017. LNCS, vol. 10611, pp. 297–304. Springer, Cham (2017). Scholar
  19. 19.
    Hayward, M.L.A., Shepherd, D.A., Griffin, D.: A hubris theory of entrepreneurship. Manag. Sci. 52(2), 160–172 (2006)CrossRefGoogle Scholar
  20. 20.
    Herrmann, B.L., Marmer, M., Dogrultan, E., Holtschke, D.: Startup ecosystem report 2012. In: Telefonica Digital and Startup Genome (2012)Google Scholar
  21. 21.
    Kerr, W.R., Nanda, R., Rhodes-Kropf, M.: Entrepreneurship as experimentation. J. Econ. Perspect. 28(3), 25–48 (2014)CrossRefGoogle Scholar
  22. 22.
    Klotins, E., Unterkalmsteiner, M., Gorschek, T.: Software engineering in startup companies: an analysis of 88 experience reports. Empirical Softw. Eng. 24(1), 68–102 (2019). CrossRefGoogle Scholar
  23. 23.
    Lindgren, E., Münch, J.: Raising the odds of success: the current state of experimentation in product development. Inf. Softw. Technol. 77, 80–91 (2016)CrossRefGoogle Scholar
  24. 24.
    Melegati, J., Goldman, A., Kon, F., Wang, X.: A model of requirements engineering in software startups. Inf. Softw. Technol. 109, 92–107 (2019)CrossRefGoogle Scholar
  25. 25.
    Mohanani, R., Salman, I., Turhan, B., Rodriguez, P., Ralph, P.: Cognitive Biases in Software Engineering: A Systematic Mapping Study. IEEE Trans. Softw. Eng. 5589(c), 1–20 (2018).
  26. 26.
    Olsson, H.H., Bosch, J.: From opinions to data-driven software R&D: a multi-case study on how to close the ‘Open Loop’ problem. In: 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications, pp. 9–16. IEEE, August 2014Google Scholar
  27. 27.
    Pantiuchina, J., Mondini, M., Khanna, D., Wang, X., Abrahamsson, P.: Are software startups applying agile practices? the state of the practice from a large survey. In: Baumeister, H., Lichter, H., Riebisch, M. (eds.) XP 2017. LNBIP, vol. 283, pp. 167–183. Springer, Cham (2017). Scholar
  28. 28.
    Patel, S.K., Prof Acharya, A., Patel, M., Rathod, V.R., Prajapati, J.B.: Performance analysis of content management systems-joomla, drupal and wordpress general terms open source content management system. Int. J. Comput. Appl. 21(4), 39–43 (2011)Google Scholar
  29. 29.
    Paternoster, N., Giardino, C., Unterkalmsteiner, M., Gorschek, T., Abrahamsson, P.: Software development in startup companies: a systematic mapping study. Inf. Softw. Technol. 56(10), 1200–1218 (2014)CrossRefGoogle Scholar
  30. 30.
    Riemenschneider, C.K., Hardgrave, B.C., Davis, F.D.: Explaining software developer acceptance of methodologies: a comparison of five theoretical models. IEEE Trans. Softw. Eng. 28(12), 1135–1145 (2002)CrossRefGoogle Scholar
  31. 31.
    Ries, E.: The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses. Crown Business, New York City (2011)Google Scholar
  32. 32.
    Rogers, E.M.: Diffusion of Innovations. Simon and Schuster, New York City (2010)Google Scholar
  33. 33.
    Ros, R., Runeson, P.: Continuous experimentation and A/B testing. In: Proceedings of the 4th International Workshop on Rapid Continuous Software Engineering - RCoSE 2018, pp. 35–41. ACM Press, New York (2018)Google Scholar
  34. 34.
    Runeson, P., Höst, M.: Guidelines for conducting and reporting case study research in software engineering. Empir. Softw. Eng. 14(2), 131–164 (2009)CrossRefGoogle Scholar
  35. 35.
    Seppänen, P., Oivo, M., Liukkunen, K.: The initial team of a software startup. In: 2016 International Conference on Engineering, Technology and Innovation (ICE) & IEEE International Technology Management Conference, pp. 57–65 (2016)Google Scholar
  36. 36.
    Steininger, D.M.: Linking information systems and entrepreneurship: a review and agenda for IT-associated and digital entrepreneurship research. Inf. Syst. J. 29, 363–407 (2019).
  37. 37.
    Tanabian, M., ZahirAzami, B.: Building high-performance team through effective job design for an early stage software start-up. In: Proceedings. 2005 IEEE International Engineering Management Conference, 2005, vol. 2, pp. 789–792. IEEE (2005)Google Scholar
  38. 38.
    Unterkalmsteiner, M., et al.: Software startups - a research agenda. e-Informatica Softw. Eng. J. 10(1), 1–28 (2016)Google Scholar
  39. 39.
    Yin, R.: Case Study Research: Design and Methods. Applied Social Research Methods. SAGE Publications, Thousand Oaks (2003)Google Scholar
  40. 40.
    Zettel, J., Maurer, F., Münch, J., Wong, L.: LIPE: a lightweight process for E-Business startup companies based on extreme programming. In: Product Focused Software Process Improvement, pp. 255–270 (2001)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Faculty of Computer ScienceFree University of Bozen-BolzanoBolzanoItaly
  2. 2.School of TechnologyPUCRSPorto AlegreBrazil

Personalised recommendations