Skip to main content
SpringerLink
Log in

On the Removal of Feature Toggles

A Study of Python Projects and Practitioners Motivations

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Feature Toggling is a technique to control the execution of features in a software project. For example, practitioners using feature toggles can experiment with new features in a production environment by exposing them to a subset of users. Some of these toggles require additional maintainability efforts and are expected to be removed, whereas others are meant to remain for a long time. However, to date, very little is known about the removal of feature toggles, which is why we focus on this topic in our paper. We conduct an empirical study that focuses on the removal of feature toggles. We use source code analysis techniques to analyze 12 Python open source projects and surveyed 61 software practitioners to provide deeper insights on the topic. Our study shows that 75% of the toggle components in the studied Python projects are removed within 49 weeks after introduction. However, eventually practitioners remove feature toggles to follow the life cycle of a feature when it becomes stable in production. We also find that not all long-term feature toggles are designed to live that long and not all feature toggles are removed from the source code, opening the possibilities to unwanted risks. Our study broadens the understanding of feature toggles by identifying reasons for their survival in practice and aims to help practitioners make better decisions regarding the way they manage and remove feature toggles.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Listing 1
Listing 2
Listing 3
Listing 4
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. Toggles are also called flags, bits, flippers, switches or gates. In this paper, we use the term toggles.

  2. A replication package is available on https://github.com/elhoyos/toggles_package

  3. https://gitlab.com/juan.hoyosr/extractor

  4. Through our personal correspondence with Mozilla Engineers.

  5. Contacted via the slack channel of OpenedX.

  6. Contacted via personal correspondence.

  7. Via personal correspondence with a Software Engineer of the Center for Open Science.

  8. Via our personal correspondence with the project’s Engineering Manager at Mozilla.

References

  • Center for Open Science (2013) https://github.com/CenterForOpenScience/osf.io. Accessed 2019-01-09

  • Django Packages : Feature Flipping (2018) https://djangopackages.org/grids/g/feature-flip/. Accessed 2018-11-05

  • Django Waffle — django-waffle 0.14.0 documentation (2018) https://waffle.readthedocs.io/en/stable/. Accessed 2019-01-11

  • edx-platform (2011) https://github.com/edx/edx-platform. Accessed 2019-01-09

  • Libraries.io - The Open Source Discovery Service (2015) https://libraries.io/. Accessed 2019-01-09

  • The Django template language — Django documentation — Django (2019) https://docs.djangoproject.com/en/2.1/ref/templates/language/. Accessed 2019-06-16

  • The Web framework for perfectionists with deadlines — Django (2019) https://www.djangoproject.com/. Accessed 2019-03-05

  • PEP 318 – Decorators for Functions and Methods — Python.org (2003) https://www.python.org/dev/peps/pep-0318/. Accessed 2019-01-23

  • Adams B, Bellomo S, Bird C, Marshall-Keim T, Khomh F, Moir K (2015) The practice and future of release engineering: a roundtable with three release engineers. IEEE Softw 32(2):42–49

    Article  Google Scholar 

  • Adams B, McIntosh S (2016) Modern release engineering in a nutshell – why researchers should care. In: Leaders of tomorrow: future of software engineering, proceedings of the 23rd IEEE international conference on software analysis, evolution, and reengineering (SANER), Osaka, Japan, pp 78–90

  • Asthagiri N (2018) OEP-17: Feature toggles. https://open-edx-proposals.readthedocs.io/en/latest/oep-0017-bp-feature-toggles.html. Accessed 2020-01-10

  • Bosworth A (2012) Building and testing at Facebook. https://www.facebook.com/notes/facebook-engineering/building-and-testing-at-facebook/10151004157328920https://www.facebook.com/notes/facebook-engineering/building-and-testing-at-facebook/10151004157328920. Accessed 2019-03-25

  • Claps GG, Berntsson Svensson R, Aurum AA (2015) On the journey to continuous deployment: Technical and social challenges along the way. Inf Softw Technol 57(1):21–31

    Article  Google Scholar 

  • Cohen J (1960) A coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37–46

    Article  Google Scholar 

  • Corbin J, Strauss A et al (2008) Basics of qualitative research: techniques and procedures for developing grounded theory

  • Feitelson DG, Frachtenberg E, Beck KL (2013) Development and deployment at facebook. IEEE Internet Comput 17(4):8–17

    Article  Google Scholar 

  • GitHub (2011) GitHub API v3 — GitHub Developer Guide. https://developer.github.com/v3/. Accessed 2019-01-09

  • GitHub (2011) Search — GitHub Developer Guide. https://developer.github.com/v3/search. Accessed 2019-01-03

  • GitHub (2013) Searching code - User Documentation. https://help.github.com/articles/searching-code. Accessed 2019-01-03

  • Harry B (2012) Announcing Continuous Deployment to Azure with Team Foundation Service — Brian Harrys blog. https://bit.ly/2MvKEkT. Accessed 2019-01-22

  • Hodgson P (2016) Feature Toggles. https://martinfowler.com/articles/feature-toggles.html. Accessed 2017-08-30

  • Humble J, Farley D (2010) Continuous delivery: reliable software releases through build, test and deployment

  • Kalliamvakou E, Gousios G, Blincoe K, Singer L, German DM, Damian D (2014) The promises and perils of mining github. In: Proceedings of the 11th working conference on mining software repositories, MSR 2014, ACM, pp 92–101

  • Kaplan EL, Meier P (1958) Nonparametric estimation from incomplete observations. J Am Stat Assoc 53(282):457–481

    Article  MathSciNet  Google Scholar 

  • Kästner C (2019) Feature flags vs configuration options — same difference? https://www.cs.cmu.edu/ckaestne/featureflags/. (Accessed on 03/30/2020)

  • LaunchDarkly (2015) Feature flags - feature flags, toggles, controls. http://featureflags.io/feature-flags/. Accessed 2018-11-03

  • Mäntylä MV, Adams B, Khomh F, Engström E, Petersen K (2015) On rapid releases and software testing: a case study and a semi-systematic literature review. Emp Softw Eng 20(5):1384–1425

    Article  Google Scholar 

  • Neely S, Stolt S (2013) Continuous delivery? Easy! Just change everything (well, maybe it is not that easy). In: Proceeding - AGILE 2013, pp 121–128

  • Osherove R (2016) Feature toggles – enterprise devOps. http://enterprisedevops.org/feature-toggle-frameworks-list/. Accessed 2018-12-11

  • Rahman AAU, Helms E, Williams L, Parnin C (2015) Synthesizing continuous deployment practices used in software development. 2015 Agile Conference, 1–10

  • Rahman MT, Querel L-P, Rigby PC, Adams B (2016) Feature toggles: practitioner practices and a case study. In: Proceedings of the 13th international conference on mining software repositories, MSR ’16. ACM, New York, pp 201–211

  • Rahman MT, Rigby PC, Shihab E (2018) The modular and feature toggle architectures of Google Chrome. Emp Softw Eng, 1–28

  • Ray B, Posnett D, Filkov V, Devanbu P (2014) A large scale study of programming languages and code quality in github. In: Proceedings of the 22nd ACM SIGSOFT international symposium on foundations of software engineering - FSE 2014. ACM Press, New York, pp 155–165

  • Sayagh M, Kerzazi N, Adams B, Petrillo F (2018) Software configuration engineering in practice: interviews, survey, and systematic literature review. IEEE Trans Softw Eng

  • Schermann G, Cito J, Leitner P, Zdun U, Gall H (2016) An empirical study on principles and practices of continuous delivery and deployment. Peer J Preprints 4:e1889v1

  • Securities and E. Commission (2013) Administrative and cease-and-desist proceedings against knight capital americas LLC. https://www.sec.gov/litigation/admin/2013/34-70694.pdf. Accessed 2019-01-19

  • Smith E, Loftin R, Murphy-Hill E, Bird C, Zimmermann T (2013) Improving developer participation rates in surveys. In: 2013 6Th international workshop on cooperative and human aspects of software engineering (CHASE). IEEE, pp 89–92

  • Yin RK (2009) Case study research: design and methods (applied social research methods). London and Singapore: Sage

  • Zapata D (2014) Going from 3 week to daily releases at netflix. USENIX Association, Philadelphia

    Google Scholar 

Download references

Acknowledgments

We would like to thank the practitioners who devoted their time and effort to respond to our online survey, and to all the practitioners of Mozilla, Université catholique de Louvain, Open edX, Tennessee Data Commons and the Center for Open Science, that responded back to our call and contributed their valuable experience using feature toggles in their projects.

Author information

Authors and Affiliations

  1. Universidad Nacional de Colombia, Medellín, Colombia

    Juan Hoyos & Albeiro Espinosa Bedoya

  2. Concordia University, Montreal, QC, Canada

    Rabe Abdalkareem, Suhaib Mujahid & Emad Shihab

  3. Queen’s University, Kingston, ON, Canada

    Rabe Abdalkareem

Authors
  1. Juan Hoyos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rabe Abdalkareem
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Suhaib Mujahid
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Emad Shihab
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Albeiro Espinosa Bedoya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juan Hoyos.

Additional information

Communicated by: Sarah Nadi

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hoyos, J., Abdalkareem, R., Mujahid, S. et al. On the Removal of Feature Toggles. Empir Software Eng 26, 15 (2021). https://doi.org/10.1007/s10664-020-09902-y

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10664-020-09902-y

Keywords

Search

Navigation