Abstract
Context: Most safety-critical systems have to fulfil compliance needs specified in safety standards. These needs can be difficult to understand from the text of the standards, and the use of conceptual models has been proposed as a solution. Goal: We aim to evaluate the understanding of safety compliance needs with models. Method: We have conducted an experiment to study the effectiveness, efficiency, and perceived benefits in understanding these needs, with text of safety standards and with UML object diagrams. Results: Sixteen Bachelor students participated in the experiment. Their average effectiveness in understanding compliance needs and their average efficiency were higher with models (17% and 15%, respectively). However, the difference is not statistically significant. The students found benefits in using models, but on average they are undecided about their ease of understanding. Conclusions: Although the results are not conclusive enough, they suggest that the use of models could improve the understanding of safety compliance needs.
This is a preview of subscription content, log in via an institution to check access.
References
Abdulkhaleq, A, Wagner, S.: A controlled experiment for the empirical evaluation of safety analysis techniques for safety-critical software. In: EASE 2015, pp. 16:1–16:10 (2015)
Abrahão, S., et al.: Assessing the effectiveness of sequence diagrams in the comprehension of functional requirements. IEEE Trans. Softw. Eng. 39(3), 327–342 (2013)
Ayora, C., et al.: Variability management in process families through change patterns. Inform. Softw. Tech. 74, 86–104 (2016)
Briand, L., et al.: Traceability and SysML design slices to support safety inspections: a controlled experiment. ACM Trans. Softw. Eng. Meth. 23(1), 9:1–9:43 (2014)
de la Vara, J.L., et al.: An industrial survey on safety evidence change impact analysis practice. IEEE Trans. Softw. Eng. 42(12), 1095–1117 (2016)
de la Vara, J.L., et al.: Do models improve the understanding of safety compliance needs? Insights from a pilot experiment. In: ESEM, pp. 32:1–32:6 (2016)
de la Vara, J.L., et al.: Model-based specification of safety compliance needs for critical systems: a holistic generic metamodel. Inform. Softw. Tech. 72, 16–30 (2016)
de la Vara, J.L., et al.: An analysis of safety evidence management with the structured assurance case metamodel. Comput. Stand. Interfaces 50, 179–198 (2017)
De Lucia, A., et al.: An experimental comparison of ER and UML class diagrams for data modelling. Empir. Softw. Eng. 15(5), 455–492 (2010)
ESA. Software engineering and standardisation (2006). http://www.esa.int/TEC/Software_engineering_and_standardisation/TECBUCUXBQE_0.html
Labunets, K., et al.: An experimental comparison of two risk-based security methods. In: ESEM, pp 163–172 (2013)
Nair, S., et al.: An extended systematic literature review on provision of evidence for safety certification. Inform. Softw. Tech. 56(7), 689–717 (2014)
Nair, S., et al.: Evidence management for compliance of critical systems with safety standards: a survey on the state of practice. Inform. Softw. Tech. 60, 1–15 (2015)
Panesar-Walawege, R.K., et al.: Supporting the verification of compliance to safety standards via model-driven engineering. Inform. Softw. Tech. 55(5), 836–864 (2013)
Razali, R., et al.: Experimental comparison of the comprehensibility of a UML-based formal specification versus a textual one. In: EASE (2007)
Salman, I., et al.: Are students representatives of professionals in software engineering experiments? In: ICSE (2015)
Sharafi, Z., et al.: An empirical study on the efficiency of graphical vs. textual representations in requirements comprehension. In: ICPC (2013)
Vegas, S., et al.: Crossover designs in software engineering experiments: benefits and perils. IEEE Trans. Softw. Eng. 42(2), 120–135 (2016)
Wohlin, C., et al.: Experimentation in Software Engineering, 2nd edn. Springer, Heidelberg (2012)
Acknowledgments
The research leading to this paper has received funding from the AMASS project (H2020-ECSEL grant agreement no 692474; Spain’s MINECO ref. PCIN-2015-262) and the AMoDDI project (Ref. 11130583). We also thank the subjects that participated in the experiment.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
de la Vara, J.L., Marín, B., Ayora, C., Giachetti, G. (2017). An Experimental Evaluation of the Understanding of Safety Compliance Needs with Models. In: Mayr, H., Guizzardi, G., Ma, H., Pastor, O. (eds) Conceptual Modeling. ER 2017. Lecture Notes in Computer Science(), vol 10650. Springer, Cham. https://doi.org/10.1007/978-3-319-69904-2_20
Download citation
DOI: https://doi.org/10.1007/978-3-319-69904-2_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-69903-5
Online ISBN: 978-3-319-69904-2
eBook Packages: Computer ScienceComputer Science (R0)