Skip to main content
SpringerLink
Log in

More for Less: Automated Experimentation in Software-Intensive Systems

  • Conference paper
  • First Online:
Product-Focused Software Process Improvement (PROFES 2017)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10611))

Abstract

Companies developing autonomous and software-intensive systems show an increasing need to adopt experimentation and data-driven strategies in their development process. With the growing complexity of the systems, companies are increasing their data analytic and experimentation teams to support data-driven development. However, organizations cannot increase in size at the same pace as the system complexity grows. Experimentation teams could run a larger number of experiments by letting the system itself to coordinate its own experiments, instead of the humans. This process is called automated experimentation. However, currently, no tools or frameworks address the challenge of running automated experiments.

This paper discusses, through a set of architectural design decisions, the development of an architecture framework that supports automated continuous experiments. The contribution of this paper is twofold. First, it presents, through a set of architectural design decisions, an architecture framework for automated experimentation. Second, it evaluates the architecture framework experimentally in the context of a human-robot interaction proxemics distance problem. This automated experimentation framework aims to deliver more value from the experiments while using fewer R&D resources.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Notes

  1. 1.

    www.turtlebot.com.

  2. 2.

    www.ros.org.

References

  1. Fabijan, A., Olsson, H.H., Bosch, J.: Time to say ‘good bye’: feature lifecycle. In: 2016 42th Euromicro Conference on Software Engineering and Advanced Applications (SEAA), pp. 9–16 (2016)

    Google Scholar 

  2. Olsson, H.H., Bosch, J.: From opinions to data-driven software R&D: a multi-case study on how to close the ‘open loop’ problem. In: Proceedings of 40th EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA), SEAA 2014, pp. 9–16 (2014)

    Google Scholar 

  3. Kohavi, R., Longbotham, R., Sommerfield, D., Henne, R.M.: Controlled experiments on the web: survey and practical guide. Data Mining Knowl. Discov. 18(1), 140–181 (2009)

    Article  MathSciNet  Google Scholar 

  4. Fabijan, A., Dmitriev, P., Olsson, H.H., Bosch, J.: The evolution of continuous experimentation in software product development. In: Proceedings of the 39th International Conference on Software Engineering, ICSE 2017 (2017)

    Google Scholar 

  5. Kohavi, R., Deng, A., Frasca, B., Walker, T., Xu, Y., Pohlmann, N.: Online controlled experiments at large scale. In: Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD 2013, p. 1168 (2013)

    Google Scholar 

  6. Li, L., Chu, W., Langford, J., Schapire, R.E.: A contextual-bandit approach to personalized news article recommendation. WWW 2010, 10 (2010)

    Google Scholar 

  7. Bosch, J., Olsson, H.H.: Data-driven continuous evolution of smart systems. In: Proceedings of the 11th International Workshop on Software Engineering for Adaptive and Self-Managing Systems - SEAMS 2016, pp. 28–34 (2016)

    Google Scholar 

  8. Mattos, D.I., Bosch, J., Olsson, H.H.: Your system gets better every day you use it: towards automated continuous experimentation. In: 2017 43th Euromicro Conference on Software Engineering and Advanced Applications (SEAA) (2017)

    Google Scholar 

  9. Kohavi, R., Deng, A., Frasca, B., Walker, T.: Online controlled experiments at large scale. In: Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, KDD 2013, pp. 1–9 (2013)

    Google Scholar 

  10. Jansen, A., Bosch, J.: Software architecture as a set of architectural design decisions. In: 5th Working IEEE/IFIP Conference on Software Architecture (WICSA 2005), vol. 2005, pp. 109–120 (2005)

    Google Scholar 

  11. Bakshy, E., Eckles, D., Bernstein, M.S.: Designing and deploying online field experiments. In: Proceedings of 23rd International Conference on World wide web - WWW 2014, pp. 283–292, September 2014

    Google Scholar 

  12. Tang, D., Agarwal, A., O’Brien, D., Meyer, M.: Overlapping experiment infrastructure. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD 2010, p. 17 (2010)

    Google Scholar 

  13. Hevner, A.R., March, S.T., Park, J., Ram, S.: Design science in information systems research. MIS Q. 28(1), 75–105 (2004)

    Article  Google Scholar 

  14. Oskoei, M.A., Walters, M.L., Dautenhahn, K.: An autonomous proxemic system for a mobile companion robot. In: Proceedings of Second International Symposium on New Frontier Human-Robot Interaction, pp. 9–15, April 2010

    Google Scholar 

Download references

Acknowledgements

This work was partially supported by the Wallenberg Autonomous Systems and Software Program (WASP).

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Chalmers University of Technology, Hörselgången 11, 412 96, Gothenburg, Sweden

    David Issa Mattos & Jan Bosch

  2. Department of Computer Science and Media Technology, Malmö University, Östra Varvsgatan 11, 205 06, Malmö, Sweden

    Helena Holmström Olsson

Authors
  1. David Issa Mattos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Bosch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Helena Holmström Olsson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Issa Mattos .

Editor information

Editors and Affiliations

  1. University of Innsbruck, Innsbruck, Austria

    Michael Felderer

  2. Technical University Munich, Garching, Germany

    Daniel Méndez Fernández

  3. Brunel University London, Uxbridge, United Kingdom

    Burak Turhan

  4. Pontifical Catholic University of Rio de, Rio de Janeiro, Brazil

    Marcos Kalinowski

  5. University College London, London, United Kingdom

    Federica Sarro

  6. Vienna University of Technology, Vienna, Austria

    Dietmar Winkler

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Mattos, D.I., Bosch, J., Holmström Olsson, H. (2017). More for Less: Automated Experimentation in Software-Intensive Systems. In: Felderer, M., Méndez Fernández, D., Turhan, B., Kalinowski, M., Sarro, F., Winkler, D. (eds) Product-Focused Software Process Improvement. PROFES 2017. Lecture Notes in Computer Science(), vol 10611. Springer, Cham. https://doi.org/10.1007/978-3-319-69926-4_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-69926-4_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-69925-7

  • Online ISBN: 978-3-319-69926-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation