Abstract
Learning-based testing (LBT) is an emerging paradigm for fully automated requirements testing. This approach combines machine learning and model-checking techniques for test case generation and verdict construction. LBT is well suited to requirements testing of low-latency safety critical embedded systems, such as can be found in the automotive sector.
We evaluate the feasibility and effectiveness of applying LBT to two safety critical industrial automotive applications. We also benchmark our LBT tool against an existing industrial test tool that executes manually written test cases.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
By test latency we mean the average time to execute a single test case on the SUT.
- 2.
In the context of the joint Vinnova FFI project 2013-05608 VIRTUES.
- 3.
The reason here is that an SUT is by definition behaviorally correct w.r.t. a model that has been reverse engineered from its behavior. So there is nothing to test.
- 4.
This has limited our possibility to disclose all technical details for commercial reasons.
- 5.
Here, a model variable is an internal SUT signal that is not part of the API. Model variables may be defined in terms of other model variables, though recursive definitions are not allowed.
References
Mbtech. www.mbtech-group.com
Supporting processes, baseline 17. ISO 26262:(2011)–Part 8 Road vehicles—functional safety, International Organization for Standardization (2011). https://www.iso.org/obp/ui/#iso:std:iso:26262:-8:ed-1:v1:en
Alur, R.: Techniques for Automatic Verification of Real-time Systems. Ph.D. thesis, Stanford, CA, USA (1992). uMI Order No. GAX92-06729
Balcázar, J.L., Díaz, J., Gavaldà, R., Watanabe, O.: Algorithms for learning finite automata from queries: a unified view. In: Du, D.Z., Ko, K.I. (eds.) Advances in Algorithms, Languages, and Complexity, pp. 53–72. Springer, Boston (1997). doi:10.1007/978-1-4613-3394-4_2
Bellard, F.: QEMU, a fast and portable dynamic translator. In: Proceedings of the FREENIX Track: 2005 USENIX Annual Technical Conference, pp. 41–46 (2005)
Broy, M., Jonsson, B., Katoen, J.P., Leucker, M., Pretschner, A.: Model-Based Testing of Reactive Systems: Advanced Lectures. LNCS. Springer Inc., New York (2005)
Cimatti, A., Clarke, E., Giunchiglia, E., Giunchiglia, F., Pistore, M., Roveri, M., Sebastiani, R., Tacchella, A.: NuSMV 2: an opensource tool for symbolic model checking. In: Brinksma, E., Larsen, K.G. (eds.) CAV 2002. LNCS, vol. 2404, pp. 359–364. Springer, Heidelberg (2002). doi:10.1007/3-540-45657-0_29
DeMillo, R.A., Lipton, R.J., Sayward, F.G.: Hints on test data selection: help for the practicing programmer. Computer 11(4), 34–41 (1978). http://dx.doi.org/10.1109/C-M.1978.218136
Dwyer, M.B., Avrunin, G.S., Corbett, J.C.: Patterns in property specifications for finite-state verification. In: Proceedings of the 1999 International Conference on Software Engineering, pp. 411–420, May 1999
Emerson, E.A.: Temporal and modal logic. In: van Leeuwen, (ed.) Handbook of Theoretical Computer Science. Volume B: Formal Models and Sematics (B), pp. 995–1072. MIT Press, Cambridge (1990)
Filipovikj, P., Nyberg, M., Rodriguez-Navas, G.: Reassessing the pattern-based approach for formalizing requirements in the automotive domain. In: 2014 IEEE 22nd International Requirements Engineering Conference (RE), pp. 444–450, August 2014
Fisher, M.: An Introduction to Practical Formal Methods Using Temporal Logic, 1st edn. Wiley Publishing, Chichester (2011). http://cds.cern.ch/record/1518490
Hierons, R.M., Bogdanov, K., Bowen, J.P., Cleaveland, R., Derrick, J., Dick, J., Gheorghe, M., Harman, M., Kapoor, K., Krause, P., Lüttgen, G., Simons, A.J.H., Vilkomir, S., Woodward, M.R., Zedan, H.: Using formal specifications to support testing. ACM Comput. Surv. 41(2), 9:1–9:76 (2009). http://doi.acm.org/10.1145/1459352.1459354
De la Higuera, C.: Grammatical Inference: Learning Automata and Grammars. Cambridge University Press, Cambridge (2010)
Jorgensen, P.C.: Software testing (2008)
Khosrowjerdi, H., Meinke, K., Rasmusson, A.: Automated behavioral requirements testing for automotive ECU applications. Technical report, KTH Royal Institute of Technology (2016)
Konrad, S., Cheng, B.H.C.: Real-time specification patterns. In: Proceedings of the 27th International Conference on Software Engineering, ICSE 2005, pp. 372–381. ACM, New York (2005). http://doi.acm.org/10.1145/1062455.1062526
Meinke, K.: Automated black-box testing of functional correctness using function approximation. In: Proceedings of the 2004 ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2004, pp. 143–153. ACM, New York (2004). http://doi.acm.org/10.1145/1007512.1007532
Meinke, K., Niu, F.: A learning-based approach to unit testing of numerical software. In: Petrenko, A., Simão, A., Maldonado, J.C. (eds.) ICTSS 2010. LNCS, vol. 6435, pp. 221–235. Springer, Heidelberg (2010). doi:10.1007/978-3-642-16573-3_16
Meinke, K., Nycander, P.: Learning-based testing of distributed microservice architectures: correctness and fault injection. In: Bianculli, D., Calinescu, R., Rumpe, B. (eds.) SEFM 2015. LNCS, vol. 9509, pp. 3–10. Springer, Heidelberg (2015). doi:10.1007/978-3-662-49224-6_1
Meinke, K., Sindhu, M.A.: Incremental learning-based testing for reactive systems. In: Gogolla, M., Wolff, B. (eds.) TAP 2011. LNCS, vol. 6706, pp. 134–151. Springer, Heidelberg (2011). doi:10.1007/978-3-642-21768-5_11
Meinke, K., Sindhu, M.A.: LBTest: a learning-based testing tool for reactive systems. In: Proceedings of the 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation, ICST 2013, pp. 447–454. IEEE Computer Society, Washington (2013). http://dx.doi.org/10.1109/ICST.2013.62
Post, A., Menzel, I., Hoenicke, J., Podelski, A.: Automotive behavioral requirements expressed in a specification pattern system: a case study at BOSCH. Requirements Eng. 17(1), 19–33 (2012). http://dx.doi.org/10.1007/s00766-011-0145-9
Shahbaz, M., Shashidhar, K.C., Eschbach, R.: Specification inference using systematic reverse-engineering methodologies: an automotive industry application. IEEE Softw. 29(6), 62–69 (2012). http://dx.doi.org/10.1109/MS.2011.159
Walkinshaw, N., Bogdanov, K., Derrick, J., Paris, J.: Increasing functional coverage by inductive testing: a case study. In: Petrenko, A., Simão, A., Maldonado, J.C. (eds.) ICTSS 2010. LNCS, vol. 6435, pp. 126–141. Springer, Heidelberg (2010). doi:10.1007/978-3-642-16573-3_10
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Khosrowjerdi, H., Meinke, K., Rasmusson, A. (2017). Learning-Based Testing for Safety Critical Automotive Applications. In: Bozzano, M., Papadopoulos, Y. (eds) Model-Based Safety and Assessment. IMBSA 2017. Lecture Notes in Computer Science(), vol 10437. Springer, Cham. https://doi.org/10.1007/978-3-319-64119-5_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-64119-5_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-64118-8
Online ISBN: 978-3-319-64119-5
eBook Packages: Computer ScienceComputer Science (R0)