Skip to main content
SpringerLink
Log in

Efficient and Flexible Test Automation in Production Systems Engineering

  • Chapter
  • First Online:
Book cover Security and Quality in Cyber-Physical Systems Engineering

Abstract

Context and background: In Production Systems Engineering (PSE), software and systems testing are success-critical along the production automation life cycle to identify defects early and efficiently. Although test automation concepts enable continuous integration and tests during engineering and maintenance, tool chains are often hardwired, less flexible, and inefficient. Thus, there is a need for more flexible tool chains to support verification and validation of control code variants. Objective: In this book chapter, we (a) describe a flexible Test Automation Framework (TAF) to enable continuous integration and tests and (b) provide an adapted maintenance process to enable efficient verification and validation of control code variants. Method: We build on best practices from Software Engineering and Software Testing to establish a flexible TAF based on Behavior-Driven Testing. We use the Abstract Syntax Tree (AST) as foundation for human-based verification and validation. We developed an initial prototype derived from industry partners and used an Industry 4.0 Testbed for evaluation. Results and conclusion: First results of the prototype implementation with selected testing tools showed the capability of the TAF concept for supporting flexible configurations of testing tool chains. The AST concept can support the human-based verification and validation of control code variants.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Aurum, A., Petersson, H., & Wohlin, C. (2002). State-of-the-art: Software inspections after 25 years. Journal of Software Testing, Verification and Reliability, 12(3), 133–154.

    Article  Google Scholar 

  • Beck, K. (2003). Test-driven development: By example. Boston, MA: Addison-Wesley.

    Google Scholar 

  • Biffl, S., Lüder, A., & Winkler, D. (2016). Multi-disciplinary engineering for Industrie 4.0: Semantic challenges, needs, and capabilities. In S. Biffl & M. Sabou (Eds.), Semantic web for intelligent engineering applications. New York: Springer.

    Google Scholar 

  • Brambilla, M., Cabot, J., & Wimmer, M. (2012). Model-driven software engineering in practice. San Rafael, CA: Morgan & Claypool.

    Book  Google Scholar 

  • Broy, M. (2006). The ‘grand challenge’ in informatics: Engineering software-intensive systems. IEEE Computer, 39(10), 72–80.

    Article  Google Scholar 

  • Duvall, P. M., Matyas, S., & Glover, A. (2007). Continuous integration: Improving software quality and reducing risk. London: Pearson.

    Google Scholar 

  • Erdogmus, H., Morisio, M., & Torchiano, M. (2005). On the effectiveness of the test-first approach to programming. IEEE Transactions on Software Engineering, 31(3), 226–237.

    Article  Google Scholar 

  • Gilb, T., Graham, D., & Finzi, S. (1993). Software inspection. Boston, MA: Addison Wesley.

    Google Scholar 

  • Häser, F., Felderer, M., & Breu, R. (2016). Is business domain language support beneficial for creating test case specifications: A controlled experiment. Information and Software Technology, 79, 52–62.

    Article  Google Scholar 

  • ISO 29119-2. (2013). Software and systems engineering – Software testing – Part 2: Test processes, ISO/IEC/IEEE291192.

    Google Scholar 

  • ISO 29119-5. (2016). Software and systems engineering – Software testing – Part 5: Keyword-driven testing, ISO/IEC/IEEE 29119-5.

    Google Scholar 

  • Jones, J. (2003). Abstract syntax tree implementation idioms. In Proceedings of the 10th Conference on Pattern Languages of Programs (PLOP) (pp. 1–10). North Carolina: The Hillside Group.

    Google Scholar 

  • Laitenberger, O., & DeBaud, J. M. (2000). An encompassing life cycle centric survey of software inspection. Journal of Systems and Software, 50(1), 5–31.

    Article  Google Scholar 

  • Laukkanen, E., Itkonen, J., & Lassenius, C. (2017). Problems, causes and solutions when adopting continuous delivery—A systematic literature review. Information and Software Technology, 82, 55–79.

    Article  Google Scholar 

  • Lindstrom, L., & Jeffries, R. (2003). Extreme programming and agile software development methodologies. In IS management handbook (8th ed.). Abingdon: Taylor & Francis.

    Google Scholar 

  • Mårtensson, T., Hammarstrom, P., & Bosch, J. (2017). Continuous integration is not about build systems. In Proceedings of the 43rd Euromicro Conference on Software Engineering and Advanced Applications (SEAA). IEEE.

    Google Scholar 

  • Mårtensson, T., Ståhl, D., & Bosch, J. (2018). Enable more frequent integration of software in industry projects. Journal of Systems and Software, 142, 223–236.

    Article  Google Scholar 

  • Micallef, M., & Colombo, C. (2015). Lessons learnt from using DSLs for automated software testing. In Proceedings of 8th International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE.

    Google Scholar 

  • Moser, T., Mordinyi, R., & Winkler, D. (2012). Extending mechatronic objects for automation systems engineering in heterogeneous engineering environments. In Proceedings of the 17th International Conference on Emerging Technologies & Factory Automation (ETFA). IEEE.

    Google Scholar 

  • Musil, J., Musil, A., Weyns, D., & Biffl, S. (2015). An architecture framework for collective intelligence systems. In Proceedings of the 12th Working IEEE/IFIP Conference on Software Architecture. ACM.

    Google Scholar 

  • Roche, J. (2013). Adopting DevOps practices in quality assurance. Communications of the ACM, 56(11), 38–43.

    Article  Google Scholar 

  • Rosen, R., von Wichert, G., Lo, G., & Bettenhausen, K. D. (2015). About the importance of autonomy and digital twins for the future of manufacturing. IFAC-PapersOnLine, 48(3), 567–572.

    Article  Google Scholar 

  • Sabou, M., Winkler, D., Penzerstadler, P., & Biffl, S. (2018). Verifying conceptual domain models with human computation: A casestudy in software engineering. In Proceedings of the 6th AAAI conference on human computation and crowdsourcing (HCOMP). Zurich: AAAI.

    Google Scholar 

  • Schatten, A., Biffl, S., Demolsky, M., Gostischa-Fanta, E., Östereicher, T., & Winkler, D. (2010). Best Practice Software-Engineering: Eine praxiserprobte Zusammenstellung von komponentenorientierten Konzepten, Methoden und Werkzeugen. Springer.

    Google Scholar 

  • Schwaber, K., & Beedle, M. (2002). Agile software development with Scrum. London: Pearson.

    MATH  Google Scholar 

  • Shahin, M., Babar, M. A., & Zhu, L. (2017). Continuous integration, delivery and deployment: A systematic review on approaches, tools, challenges and practices. IEEE Access, 5, 3909–3943.

    Article  Google Scholar 

  • Soeken, M., Wille, R., & Drechsler, R. (2012). Assisted behavior driven development using natural language processing. In Objects, models, components, patterns, tools (pp. 269–287). Berlin: Springer.

    Chapter  Google Scholar 

  • Solis, C., & Wang, X. (2011). A study of the characteristics of behaviour driven development. In Proceedings of the 37th EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA). IEEE.

    Google Scholar 

  • Spillner, A., Linz, T., & Schaefer, H. (2014). Software testing foundations: A study guide for the certified tester exam. San Rafael, CA: Rocky Nook.

    Google Scholar 

  • Ståhl, D., & Bosch, J. (2014). Modeling continuous integration practice differences in industry software development. Journal of Systems and Software, 87, 48–59.

    Article  Google Scholar 

  • Sunindyo, W. D., Moser, T., Winkler, D., & Biffl, S. (2010). Foundations for event-based process analysis in heterogeneous software engineering environments. In Proceedings of the 36th EUROMICRO conference on software engineering and advanced applications (SEAA). IEEE.

    Google Scholar 

  • Van Bennekum, A., Cockburn, A., Cunningham, W., Fowler, M., Grenning, J., Highsmith, J., Hunt, A., Jeffries, R., Beck, K., & Beedle, M. (2001). Manifesto for agile software development. Alicante: Universidad de Alicante.

    Google Scholar 

  • Vogel-Heuser, B., Kegel, G., Bender, K., & Wucherer, K. (2009). Global information architecture for industrial automation. In Automatisierungstechnische Praxis (atp). Munich: Oldenbourg-Verlag.

    Google Scholar 

  • Vogel-Heuser, B., Fay, A., Schaefer, I., & Tichy, M. (2015). Evolution of software in automated production systems: Challenges and research directions. Journal of Systems and Software, 110, 54–84.

    Article  Google Scholar 

  • Whitehead, J. (2007). Collaboration in software engineering: A roadmap. In Future of software engineering (FOSE). IEEE.

    Google Scholar 

  • Winkler, D., Wimmer, M., Berardinelli, L., & Biffl, S. (2017a). Towards model quality assurance for multi-disciplinary engineering.Needs, challenges, and solution concepts in an AutomationMLcontext. In S. Biffl, A. Lüder, & D. Gerhard (Eds.), Multi-disciplinary engineering for cyber-physical production systems, chapter 16 (pp. 433–457). Cham: Springer.

    Chapter  Google Scholar 

  • Winkler, D., Sabou, M., Petrovic, S., Biffl, S., Kalinowski, M., & Carneiro, G. (2017b). Improving model inspection with crowdsourcing. In Proceedings of the 4th International Workshop on Crowdsourcing in Software Engineering. Buenos Aires: ACM/IEEE International Conference on Software Engineering (ICSE).

    Google Scholar 

  • Winkler, D., Sabou, M., & Biffl, S. (2017c). Improving quality assurance in multidisciplinary engineering environments with semantic technologies. In Quality control and assurance - an ancient Greek term re-mastered. London: Intec Publishing.

    Google Scholar 

  • Winkler, D., Meixner, K., & Biffl, S. (2018a). Towards flexible and automated testing in production systems engineering projects. In Proceedings of the 23rd International Conference on Emerging Technologies and Factory Automation (ETFA). IEEE.

    Google Scholar 

  • Winkler, D., Sabou, M., Petrovic, S., Biffl, S., & Kalinowski, M. (2018b). Investigating a distributed and scalable model review process. Centro Latinoamericano de Estudios en Informatica (CLEI), 21(1).

    Google Scholar 

  • Zhu, Y.-M. (2016). Software reading techniques: Twenty techniques for more effective software review and inspection. New York: Apress.

    Book  Google Scholar 

Download references

Acknowledgments

The financial support by the Christian Doppler Research Association, the Austrian Federal Ministry for Digital and Economic Affairs, and the National Foundation for Research, Technology and Development is gratefully acknowledged. The research done by Petr Novak has been supported by the DAMiAS project funded by the Technology Agency of the Czech Republic.

Author information

Authors and Affiliations

  1. Christian Doppler Laboratory for Security and Quality Improvement in the Production System Lifecycle (CDL-SQI), Institute of Information Systems Engineering, Technische Universität Wien, Vienna, Austria

    Dietmar Winkler & Kristof Meixner

  2. Czech Technical University, Prague, Czech Republic

    Petr Novak

Authors
  1. Dietmar Winkler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kristof Meixner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Petr Novak
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dietmar Winkler .

Editor information

Editors and Affiliations

  1. Institute of Information Systems Engineering, Technische Universität Wien, Vienna, Austria

    Stefan Biffl

  2. Institute of Information Systems Engineering, Technische Universität Wien, Vienna, Austria

    Matthias Eckhart

  3. Institute of Ergonomics, Manufacturing Systems and Automation, Otto von Guericke University Magdeburg, Magdeburg, Germany

    Arndt Lüder

  4. Institute of Information Systems Engineering, Technische Universität Wien, Vienna, Austria

    Edgar Weippl

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Winkler, D., Meixner, K., Novak, P. (2019). Efficient and Flexible Test Automation in Production Systems Engineering. In: Biffl, S., Eckhart, M., Lüder, A., Weippl, E. (eds) Security and Quality in Cyber-Physical Systems Engineering. Springer, Cham. https://doi.org/10.1007/978-3-030-25312-7_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-25312-7_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-25311-0

  • Online ISBN: 978-3-030-25312-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation