Skip to main content
SpringerLink
Log in

Time-Aware Test Case Execution Scheduling for Cyber-Physical Systems

  • Conference paper
  • First Online:
Principles and Practice of Constraint Programming (CP 2017)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10416))

Abstract

Testing cyber-physical systems involves the execution of test cases on target-machines equipped with the latest release of a software control system. When testing industrial robots, it is common that the target machines need to share some common resources, e.g., costly hardware devices, and so there is a need to schedule test case execution on the target machines, accounting for these shared resources. With a large number of such tests executed on a regular basis, this scheduling becomes difficult to manage manually. In fact, with manual test execution planning and scheduling, some robots may remain unoccupied for long periods of time and some test cases may not be executed.

This paper introduces TC-Sched, a time-aware method for automated test case execution scheduling. TC-Sched uses Constraint Programming to schedule tests to run on multiple machines constrained by the tests’ access to shared resources, such as measurement or networking devices. The CP model is written in SICStus Prolog and uses the Cumulatives global constraint. Given a set of test cases, a set of machines, and a set of shared resources, TC-Sched produces an execution schedule where each test is executed once with minimal time between when a source code change is committed and the test results are reported to the developer. Experiments reveal that TC-Sched can schedule 500 test cases over 100 machines in less than 4 min for 99.5% of the instances. In addition, TC-Sched largely outperforms simpler methods based on a greedy algorithm and is suitable for deployment on industrial robot testing.

A. Gotlieb and H. Spieker—These authors are supported by the ResearchCouncil of Norway (RCN) through the research-based innovation center Certus, under the SFI programme.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    CPS can simply be seen as communicating embedded software systems.

  2. 2.

    OTS was part of the Industrial Modelling Competition at CP 2015.

  3. 3.

    In [5] an additional third argument to Cumulatives, \(Op \in \{\le , \ge \}\) is defined. We omit it throughout our work and always set \(Op =\,\le \).

  4. 4.

    Throughout the paper, lower-case characters are used to represent constants and upper-case characters are used to represent variables.

  5. 5.

    The general cumulative scheduling problem is known to be NP-hard [2].

  6. 6.

    All generated instances are available in CSPLib, a library of test problems for constraint solvers [25].

References

  1. Aggoun, A., Beldiceanu, N.: Extending CHIP in order to solve complex scheduling and placement problems. Math. Comput. Modell. 17(7), 57–73 (1993)

    Article  Google Scholar 

  2. Baptiste, P., Le Pape, C., Nuijten, W.: Constraint-Based Scheduling: Applying Constraint Programming to Scheduling Problems, vol. 39. Springer Science & Business Media, Berlin (2001)

    MATH  Google Scholar 

  3. Beck, J.C., Feng, T.K., Watson, J.P.: Combining constraint programming and local search for job-shop scheduling. INFORMS J. Comput. 23(1), 1–14 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  4. Behnke, D., Geiger, M.J.: Test instances for the flexible job shop scheduling problem with work centers. Technical report RR-12-01-01, Helmut-Schmidt University, Hamburg, Germany (2012)

    Google Scholar 

  5. Beldiceanu, N., Carlsson, M.: A new multi-resource cumulatives constraint with negative heights. In: Hentenryck, P. (ed.) CP 2002. LNCS, vol. 2470, pp. 63–79. Springer, Heidelberg (2002). doi:10.1007/3-540-46135-3_5

    Chapter  Google Scholar 

  6. Brandimarte, P.: Routing and scheduling in a flexible job shop by tabu search. Ann. Oper. Res. 41(3), 157–183 (1993)

    Article  MATH  Google Scholar 

  7. Brucker, P., Knust, S.: Complex Scheduling (GOR-Publications). Springer-Verlag New York Inc., Secaucus (2006)

    MATH  Google Scholar 

  8. Brucker, P., Drexl, A., Möhring, R., Neumann, K., Pesch, E.: Resource-constrained project scheduling: notation, classification, models, and methods. Eur. J. Oper. Res. 112(1), 3–41 (1999)

    Article  MATH  Google Scholar 

  9. de Campos, J., Arcuri, A., Fraser, G., de Abreu, R.: Continuous test generation: enhancing continuous integration with automated test generation. In: ASE 2014, Västerås, Sweden, pp. 55–66 (2014)

    Google Scholar 

  10. Carlsson, M., Ottosson, G., Carlson, B.: An open-ended finite domain constraint solver. In: Glaser, H., Hartel, P., Kuchen, H. (eds.) PLILP 1997. LNCS, vol. 1292, pp. 191–206. Springer, Heidelberg (1997). doi:10.1007/BFb0033845

    Chapter  Google Scholar 

  11. Carlsson, M., et al.: SICStus Prolog user’s manual, release 4. Technical Report, SICS - Swedish Institute of Computer Science (2007)

    Google Scholar 

  12. Do, H., Mirarab, S., Tahvildari, L., Rothermel, G.: The effects of time constraints on test case prioritization: a series of controlled experiments. IEEE Trans. Soft. Eng. 36(5), 593–617 (2010)

    Article  Google Scholar 

  13. Duvall, P.M., Matyas, S., Glover, A.: Continuous Integration: Improving Software Quality and Reducing Risk. Pearson Education, London (2007)

    Google Scholar 

  14. Elbaum, S., Rothermel, G., Penix, J.: Techniques for improving regression testing in continuous integration development environments. In: FSE 2014 (2014)

    Google Scholar 

  15. Fowler, M., Foemmel, M.: Continuous integration (2006). http://martinfowler.com/articles/continuousIntegration.html

  16. Gotlieb, A., Marijan, D.: Flower: optimal test suite reduction as a network maximum flow. In: ISSTA 2014, San José, CA, USA, pp. 171–180 (2014)

    Google Scholar 

  17. Hao, D., Zhang, L., Wu, X., Mei, H., Rothermel, G.: On-demand test suite reduction. In: ICSE 2012, pp. 738–748 (2012)

    Google Scholar 

  18. Hartmann, S., Briskorn, D.: A survey of variants and extensions of the resource-constrained project scheduling problem. Eur. J. Oper. Res. 207(1), 1–14 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  19. Hartmann, S., Kolisch, R.: Experimental evaluation of state-of-the-art heuristics for the resource-constrained project scheduling problem. Eur. J. Oper. Res. 127(2), 394–407 (2000)

    Article  MATH  Google Scholar 

  20. Herroelen, W., De Reyck, B., Demeulemeester, E.: Resource-constrained project scheduling: a survey of recent developments. Comput. Oper. Res. 25(4), 279–302 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  21. Kolisch, R., Hartmann, S.: Experimental investigation of heuristics for resource-constrained project scheduling: an update. Eur. J. Oper. Res. 174(1), 23–37 (2006)

    Article  MATH  Google Scholar 

  22. Kreter, S., Schutt, A., Stuckey, P.J.: Modeling and solving project scheduling with calendars. In: Pesant, G. (ed.) CP 2015. LNCS, vol. 9255, pp. 262–278. Springer, Cham (2015). doi:10.1007/978-3-319-23219-5_19

    Google Scholar 

  23. Lin, C., Tang, K., Kapfhammer, G.: Test suite reduction methods that decrease regression testing costs by identifying irreplaceable tests. Inf. Softw. Technol. 56, 1322–1344 (2014)

    Article  Google Scholar 

  24. Marijan, D., Gotlieb, A., Sen, S.: Test case prioritization for continuous regression testing: an industrial case study. In: ICSM 2013, Eindhoven, The Netherlands (2013)

    Google Scholar 

  25. Mossige, M.: CSPLib problem 073: test scheduling problem. http://www.csplib.org/Problems/prob073

  26. Mossige, M., Gotlieb, A., Meling, H.: Using CP in automatic test generation for ABB robotics’ paint control system. In: O’Sullivan, B. (ed.) CP 2014. LNCS, vol. 8656, pp. 25–41. Springer, Cham (2014). doi:10.1007/978-3-319-10428-7_6

    Google Scholar 

  27. Orso, A., Rothermel, G.: Software testing: a research travelogue (2000–2014). In: FOSE 2014, Hyderabad, India, pp. 117–132 (2014)

    Google Scholar 

  28. Orso, A., Shi, N., Harrold, M.J.: Scaling regression testing to large software systems. In: FSE 2014, pp. 241–251. ACM Press, Newport Beach (2004)

    Google Scholar 

  29. Schutt, A., Feydy, T., Stuckey, P.J., Wallace, M.G.: Why cumulative decomposition is not as bad as it sounds. In: Gent, I.P. (ed.) CP 2009. LNCS, vol. 5732, pp. 746–761. Springer, Heidelberg (2009). doi:10.1007/978-3-642-04244-7_58

    Chapter  Google Scholar 

  30. Schutt, A., Chu, G., Stuckey, P.J., Wallace, M.G.: Maximising the net present value for resource-constrained project scheduling. In: Beldiceanu, N., Jussien, N., Pinson, É. (eds.) CPAIOR 2012. LNCS, vol. 7298, pp. 362–378. Springer, Heidelberg (2012). doi:10.1007/978-3-642-29828-8_24

    Chapter  Google Scholar 

  31. Schutt, A., Feydy, T., Stuckey, P.J.: Scheduling optional tasks with explanation. In: Schulte, C. (ed.) CP 2013. LNCS, vol. 8124, pp. 628–644. Springer, Heidelberg (2013). doi:10.1007/978-3-642-40627-0_47

    Chapter  Google Scholar 

  32. Schutt, A., Feydy, T., Stuckey, P.J., Wallace, M.G.: Solving RCPSP/max by lazy clause generation. J. Sched. 16(3), 273–289 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  33. Siala, M., Artigues, C., Hebrard, E.: Two clause learning approaches for disjunctive scheduling. In: Pesant, G. (ed.) CP 2015. LNCS, vol. 9255, pp. 393–402. Springer, Cham (2015). doi:10.1007/978-3-319-23219-5_28

    Google Scholar 

  34. Simonis, H., O’Sullivan, B.: Search Strategies for rectangle packing. In: Stuckey, P.J. (ed.) CP 2008. LNCS, vol. 5202, pp. 52–66. Springer, Heidelberg (2008). doi:10.1007/978-3-540-85958-1_4

    Chapter  Google Scholar 

  35. Stolberg, S.: Enabling agile testing through continuous integration. In: AGILE 2009, pp. 369–374. IEEE (2009)

    Google Scholar 

  36. Szeredi, R., Schutt, A.: Modelling and solving multi-mode resource-constrained project scheduling. In: Rueher, M. (ed.) CP 2016. LNCS, vol. 9892, pp. 483–492. Springer, Cham (2016). doi:10.1007/978-3-319-44953-1_31

    Chapter  Google Scholar 

  37. Taillard, E.: Benchmarks for basic scheduling problems. Eur. J. Oper. Res. 64(2), 278–285 (1993)

    Article  MATH  Google Scholar 

  38. Walcott, K.R., Soffa, M.L., Kapfhammer, G.M., Roos, R.S.: Time-aware test suite prioritization. In: ISSTA 2006, Portland, Maine, USA, pp. 1–12 (2006)

    Google Scholar 

  39. Zhang, L., Hou, S., Guo, C., Xie, T., Mei, H.: Time-aware test-case prioritization using integer linear programming. In: ISSTA 2009, Chicago, IL, USA, pp. 213–224 (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ABB Robotics, Bryne, Norway

    Morten Mossige

  2. Simula Research Laboratory, Lysaker, Norway

    Arnaud Gotlieb & Helge Spieker

  3. University of Stavanger, Stavanger, Norway

    Morten Mossige & Hein Meling

  4. RISE SICS, Kista, Sweden

    Mats Carlsson

Authors
  1. Morten Mossige
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arnaud Gotlieb
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Helge Spieker
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hein Meling
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mats Carlsson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Helge Spieker .

Editor information

Editors and Affiliations

  1. University of Toronto, Toronto, Ontario, Canada

    J. Christopher Beck

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Mossige, M., Gotlieb, A., Spieker, H., Meling, H., Carlsson, M. (2017). Time-Aware Test Case Execution Scheduling for Cyber-Physical Systems. In: Beck, J. (eds) Principles and Practice of Constraint Programming. CP 2017. Lecture Notes in Computer Science(), vol 10416. Springer, Cham. https://doi.org/10.1007/978-3-319-66158-2_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-66158-2_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-66157-5

  • Online ISBN: 978-3-319-66158-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation