Skip to main content
SpringerLink
Log in
SpringerLink
Log in

Software Product Quality in DevOps Contexts: A Systematic Literature Review

  • Conference paper
  • First Online:
Trends and Applications in Software Engineering (CIMPS 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1071))

Included in the following conference series:

Abstract

DevOps is a change in the organizational culture that aims to reduce the gap between development and operation teams, accelerating the software release process. However, little is known about the impact of this approach on software product quality. This study aims to analyze the influence of the application of DevOps on software product quality; therefore, a systematic literature review was conducted. Thirty-one articles related to DevOps and its influence on product quality were identified. The studies indicate a strong influence of some product quality characteristics, specifically: Reliability and Maintainability. Additionally, practices associated with DevOps, such as the minimum viable product, deployment automation, test automation, cloud computing and team cooperation, show a relationship with the improvement in software product quality, however, its adoption also brings new challenges to preserve security.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Järvinen, J., Huomo, T., Mikkonen, T., Tyrväinen, P.: From agile software development to mercury business. In: LNBIP, vol. 182 (2014)

    Google Scholar 

  2. Chen, L.: Continuous delivery: huge benefits, but challenges too. IEEE Softw. 32(2), 50–54 (2015)

    Article  Google Scholar 

  3. Rodríguez, P., et al.: Continuous deployment of software intensive products and services: a systematic mapping study. J. Syst. Softw. 123, 263–291 (2017)

    Article  Google Scholar 

  4. Callanan, M., Spillane, A.: DevOps: making it easy to do the right thing. IEEE Softw. 33(3), 53–59 (2016)

    Article  Google Scholar 

  5. Ebert, C., Gallardo, G., Hernantes, J., Serrano, N.: DevOps. IEEE Softw. 33(3), 94–100 (2016)

    Article  Google Scholar 

  6. Virmani, M.: Understanding DevOps & bridging the gap from continuous integration to continuous delivery. In: 5th International Conference on Innovative Computing Technology, INTECH 2015, pp. 78–82 (2015)

    Google Scholar 

  7. ISO: International Standard ISO/IEC 25010 - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models. ISO, Geneva (2011)

    Google Scholar 

  8. Debois, P.: Agile Infrastructure and Operations (2008)

    Google Scholar 

  9. Rapaport, R.: A short history of DevOps (2014). http://www.ca.com/us/rewrite/articles/devops/a-short-history-of-devops.html. Accessed 28 Dec 2017

  10. Weerasiri, D., Barukh, M.C., Benatallah, B., Cao, J.: A model-driven framework for interoperable cloud resources management. In: Lecture Notes Compute Science (Including Subseries Lecture Notes Artificial Intelligence on Lecture Notes Bioinformatics). LNCS, vol. 9936, pp. 186–201 (2016)

    Google Scholar 

  11. Yoshii, J., Hanawa, D.: Technology to automate system operations for flexibly responding to dynamically changing cloud environments. Fujitsu Sci. Tech. J. 51(2), 81–85 (2015)

    Google Scholar 

  12. Díaz, O., Muñoz, M.: Implementation of a DevSecOps + Risk management in a data center of a mexican organization [Implementación de un enfoque DevSecOps + Risk Management en un Centro de Datos de una organización Mexicana]. RISTI – Rev. Iber. Sist. e Tecnol. Inf. 26, 43–53 (2018)

    Google Scholar 

  13. Díaz, O., Muñoz, M.: Reinforcing DevOps approach with security and risk management: an experience of implementing it in a data center of a mexican organization. In: 2017 6th International Conference on Software Process Improvement (CIMPS), pp. 1–7 (2017)

    Google Scholar 

  14. Basiri, A., et al.: Chaos engineering. IEEE Softw. 33(3), 35–41 (2016)

    Article  Google Scholar 

  15. Feitelson, D., Frachtenberg, E., Beck, K.: Development and deployment at Facebook. IEEE Internet Comput. 17(4), 8–17 (2013)

    Article  Google Scholar 

  16. ISO/IEC: ISO/IEC 25000: 2014 - Systems and software engineering – Systems and software Quality Requirements and Evaluation (SQuaRE) – Guide to SQuaRE (2014)

    Google Scholar 

  17. Roche, J.: Adopting DevOps practices in quality assurance. Commun. ACM 56(11), 38–43 (2013)

    Article  Google Scholar 

  18. Erich, F., Amrit, C., Daneva, M.: Report: DevOps Literature Review, October 2014

    Google Scholar 

  19. Velasquez, N.F., Kim, G., Kersten, N., Humble, J.: 2014 State of DevOps report. Puppetlabs (2014)

    Google Scholar 

  20. Forsgren, N., Kim, G., Kersten, N., Humble, J.: 2015 State of DevOps report. Puppetlabs (2015)

    Google Scholar 

  21. Brown, A., Forsgren, N., Humble, J., Kersten, N., Kim, G.: 2016 state of DevOps. Puppetlabs (2016)

    Google Scholar 

  22. Forsgren, N., Humble, J., Kim, G., Brown, A., Kersten, N.: 2017 state of DevOps report. Puppetlabs (2017)

    Google Scholar 

  23. Velasquez, N.F., Kim, G., Kersten, N., Humble, J.: State of DevOps report 2018 (2018)

    Google Scholar 

  24. Kitchenham, B., Charters, S.: Guidelines for performing systematic literature reviews in software engineering version 2.3, vol. 45, no. 4ve (2007)

    Google Scholar 

  25. Genero, M., Cruz, J.A., Piattini, M.G.: Métodos de investigación en ingeniería del software (2014)

    Google Scholar 

  26. Solingen, R., Berghout, E.: The goal/question/metric method: a practical guide for quality improvement of software development (1999)

    Google Scholar 

  27. Santos, C.M.D.C, Pimenta, C.A.D.M, Nobre, M.R.C.: A estratégia PICO para a construção da pergunta de pesquisa e busca de evidências. Rev. Lat. Am. Enfermagem 15(3), 508–511 (2007). https://doi.org/10.1590/S0104-11692007000300023

  28. Elberzhager, F., Münch, J., Nha, V.T.N.: A systematic mapping study on the combination of static and dynamic quality assurance techniques. Inf. Softw. Technol. 54(1), 1–15 (2012)

    Article  Google Scholar 

  29. Cukier, D.: DevOps patterns to scale web applications using cloud services. In: Proceedings of the 2013 Companion Publication for Conference on Systems, Programming; Applications: Software for Humanity, vol. 38, pp. 143–152 (2013)

    Google Scholar 

  30. De Bayser, M., Azevedo, L.G., Cerqueira, R.: ResearchOps: the case for DevOps in scientific applications. In: Proceedings of the 2015 IFIP/IEEE International Symposium on Integrated Network Management, IM 2015, pp. 1398–1404 (2015)

    Google Scholar 

  31. Schneider, T.: Achieving cloud scalability with microservices and DevOps in the connected car domain. In: CEUR Workshop Proceedings, vol. 1559, pp. 138–141 (2016)

    Google Scholar 

  32. Gotimer, G., Stiehm, T.: DevOps advantages for testing: increasing quality through continuous delivery. CrossTalk Mag. 29(3), 13–18 (2016)

    Google Scholar 

  33. Jones, S., Noppen, J., Lettice, F.: Management challenges for DevOps adoption within UK SMEs. In: Proceedings of the 2nd International Workshop on Quality-Aware DevOps, pp. 7–11 (2016)

    Google Scholar 

  34. Nybom, K., Smeds, J., Porres, I.: On the impact of mixing responsibilities between Devs and Ops. In: Lecture Notes on Business Information Processing, vol. 251, pp. 131–143 (2016)

    Google Scholar 

  35. Lwakatare, L.E., et al.: Towards DevOps in the embedded systems domain: why is it so hard? In: Proceedings of the Annual Hawaii International Conference on System Sciences, pp. 5437–5446, March 2016

    Google Scholar 

  36. Ardulov, Y., Shchemelinin, D.: DevOps: reliability, monitoring and management with service asset and configuration management. In: imPACt 2017 - Internet, Mobile, Performance and Capacity, Cloud and Technology, November 2017

    Google Scholar 

  37. Elberzhager, F., Arif, T., Naab, M., Süß, I., Koban, S.: From agile development to DevOps: going towards faster releases at high quality - experiences from an industrial context. In: Lecture Notes on Business Information Processing, vol. 269, pp. 33–44 (2017)

    Google Scholar 

  38. Meirosu, C., John, W., Opsenica, M., Mecklin, T., Degirmenci, F., Dinsing, T.: Fueling the evolution. Ericsson Rev. (English Ed.), vol. 95, no. 2, pp. 70–81 (2017)

    Google Scholar 

  39. Perez-Palacin, D., Ridene, Y., Merseguer, J.: Quality assessment in DevOps: automated analysis of a tax fraud detection system. In: Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering Companion, pp. 133–138 (2017)

    Google Scholar 

  40. Laukkanen, E., Paasivaara, M., Itkonen, J., Lassenius, C., Arvonen, T.: Towards continuous delivery by reducing the feature freeze period: a case study. In: Proceedings of the 39th International Conference on Software Engineering: Software Engineering in Practice Track, pp. 23–32 (2017)

    Google Scholar 

  41. Fazal-Baqaie, M., Güldali, B., Oberthür, S.: Towards DevOps in multi-provider projects. In: CEUR Workshop Proceedings, vol. 1806, pp. 18–21 (2017)

    Google Scholar 

  42. Li, H., Chen, T.-H. P, Hassan, A.E., Nasser, M., Flora, P.: Adopting autonomic computing capabilities in existing large-scale systems: an industrial experience report. In: Proceedings of International Conference on Software Engineering, pp. 1–10 (2018)

    Google Scholar 

  43. Herger, L.M., Bodarky, M., Fonseca, C.A.: Breaking down the barriers for moving an enterprise to cloud. In: IEEE International Conference on Cloud Computing, CLOUD, pp. 572–576, July 2018

    Google Scholar 

  44. Zheng, E., Gates-Idem, P., Lavin, M.: Building a Virtually air-gapped secure environment in aws: with principles of DevOps security program and secure software delivery. In: Proceedings of the 5th Annual Symposium and Bootcamp on Hot Topics in the Science of Security, pp. 11:1–11:8 (2018)

    Google Scholar 

  45. Cleveland, S.B., Dooley, R., Perry, D., Stubbs, J., Fonner, J.M., Jacobs, G.A.: Building science gateway infrastructure in the middle of the pacific and beyond: experiences using the agave deployer and agave platform to build science gateways. In: ACM International Conference Proceeding Series (2018)

    Google Scholar 

  46. Rahman, A.: Characteristics of Defective infrastructure as code scripts in DevOps. In: Proceedings of the 40th International Conference on Software Engineering: Companion Proceedings, pp. 476–479 (2018)

    Google Scholar 

  47. Sampedro, Z., Holt, A., Hauser, T.: Continuous Integration and Delivery for HPC: using singularity and jenkins. In: Proceedings of the Practice and Experience on Advanced Research Computing, pp. 6:1–6:6 (2018)

    Google Scholar 

  48. Senapathi, M., Buchan, J., Osman, H.: DevOps capabilities, practices, and challenges: insights from a case study. In: ACM International Conference Proceeding Series, vol. Part F1377 (2018)

    Google Scholar 

  49. Düllmann, T.F., Paule, C., van Hoorn, A.: Exploiting DevOps practices for dependable and secure continuous delivery pipelines. In: Proceedings of the 4th International Workshop on Rapid Continuous Software Engineering, pp. 27–30 (2018)

    Google Scholar 

  50. Guija, D., Siddiqui, M.S.: Identity and Access control for micro-services based 5G NFV platforms. In: Proceedings of the 13th International Conference on Availability, Reliability and Security, pp. 46:1–46:10 (2018)

    Google Scholar 

  51. Ivanov, V., Smolander, K.: Implementation of a DevOps pipeline for serverless applications. In: Lecture Notes in Computer Science (Including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (2018)

    Google Scholar 

  52. Lewerentz, M., et al.: Implementing DevOps practices at the control and data acquisition system of an experimental fusion device. Fusion Eng. Des. 146, Part A, 40–45 (2019). https://doi.org/10.1016/j.fusengdes.2018.11.022. ISSN 0920-3796

  53. Lautenschlager, F., Ciolkowski, M.: Making runtime data useful for incident diagnosis: an experience report. In: Lecture Notes in Computer Science (Including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), LNCS, vol. 11271, pp. 422–430 (2018)

    Google Scholar 

  54. Jacobsen, D.M., Kleinman, R., Longley, H.: Managing a Cray supercomputer as a git branch. Concurr. Comput. Pract. Experience 31, e5092 (2019). https://doi.org/10.1002/cpe.5092

    Google Scholar 

  55. Bruza, M., Reith, M.: Teaming with silicon valley to enable multi-domain command and control. In: 13th International Conference on Cyber Warfare and Security, ICCWS 2018, pp. 663–667 (2018)

    Google Scholar 

  56. Albuquerque, A.B., Cruz, V.L.: Implementing DevOps in legacy systems. Adv. Intell. Syst. Comput. 860, 143–161 (2019)

    Google Scholar 

  57. Savor, T., Douglas, M., Gentili, M., Williams, L., Beck, K., Stumm, M.: Continuous deployment at Facebook and OANDA. In: 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C), pp. 21–30 (2016)

    Google Scholar 

Download references

Acknowledgments

This work is framed within the CalProdSw project supported by the Department of Engineering and the Grupo de Investigación y Desarrollo de Ingeniería de Software (GIDIS) from the Pontificia Universidad Católica del Perú.

Author information

Authors and Affiliations

  1. Escuela de Posgrado, Pontificia Universidad Católica del Perú, Lima, Peru

    Daniel Céspedes

  2. Facultad de Ingeniería y Tecnología Informática, Universidad de Belgrano, Buenos Aires, Argentina

    Paula Angeleri

  3. Departamento de Ingeniería, Pontificia Universidad Católica del Perú, Lima, Peru

    Karin Melendez & Abraham Dávila

Authors
  1. Daniel Céspedes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paula Angeleri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Karin Melendez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Abraham Dávila
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abraham Dávila .

Editor information

Editors and Affiliations

  1. Unidad Zacatecas, Centro de Investigación en Matemáticas, Zacatecas, Zacatecas, Mexico

    Jezreel Mejia

  2. Unidad Zacatecas, Centro de Investigación en Matemáticas A.C, Zacatecas, Zacatecas, Mexico

    Mirna Muñoz

  3. Departamento de Engenharia Informática, Universidade de Coimbra, Coimbra, Portugal

    Álvaro Rocha

  4. Escuela Técnica Superior de Ingenieros Informáticos, Universidad Politécnica de Madrid, Madrid, Spain

    Jose A. Calvo-Manzano

Appendix

Appendix

https://drive.google.com/open?id=11yehxYJJ0VfJrgkbe0k2T_pYIrq0gZbp

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Céspedes, D., Angeleri, P., Melendez, K., Dávila, A. (2020). Software Product Quality in DevOps Contexts: A Systematic Literature Review. In: Mejia, J., Muñoz, M., Rocha, Á., A. Calvo-Manzano, J. (eds) Trends and Applications in Software Engineering. CIMPS 2019. Advances in Intelligent Systems and Computing, vol 1071. Springer, Cham. https://doi.org/10.1007/978-3-030-33547-2_5

Download citation

Publish with us

Policies and ethics

Search

Navigation