Abstract
Context and motivation: In the railway safety-critical domain requirements documents have to abide to strict quality criteria. Rule-based natural language processing (NLP) techniques have been developed to automatically identify quality defects in natural language requirements. However, the literature is lacking empirical studies on the application of these techniques in industrial settings. Question/problem: Our goal is to investigate to which extent NLP can be practically applied to detect defects in the requirements documents of a railway signalling manufacturer. Principal idea/results: To address this goal, we first identified a set of typical defects classes, and, for each class, an engineer of the company implemented a set of defect-detection patterns by means of the GATE tool for text processing. After a preliminary analysis, we applied the patterns to a large set of 1866 requirements previously annotated for defects. The output of the patterns was further inspected by two domain experts to check the false positive cases. Contribution: This is one of the first works in which defect detection NLP techniques are applied on a very large set of industrial requirements annotated by domain experts. We contribute with a comparison between traditional manual techniques used in industry for requirements analysis, and analysis performed with NLP. Our experience tells that several discrepancies can be observed between the two approaches. The analysis of the discrepancies offers hints to improve the capabilities of NLP techniques with company specific solutions, and suggests that also company practices need to be modified to effectively exploit NLP tools.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
- 2.
The dataset appears balanced since VE1 continued to select requirements until a balanced number of accepted and rejected requirements was obtained.
- 3.
The requirement was not rejected since it was clarified by other subsequent requirements. This violates the guideline (c) that require requirements to be stand-alone, but the defect was not considered crucial.
References
Alvarez, S.A.: An exact analytical relation among recall, precision, and classification accuracy in information retrieval. Technical report BCCS-02-01. Computer Science Department, Boston College (2002)
Arora, C., Sabetzadeh, M., Briand, L., Zimmer, F.: Automated checking of conformance to requirements templates using natural language processing. IEEE TSE 41(10), 944–968 (2015)
Berry, D., Gacitua, R., Sawyer, P., Tjong, S.F.: The case for dumb requirements engineering tools. In: Regnell, B., Damian, D. (eds.) REFSQ 2012. LNCS, vol. 7195, pp. 211–217. Springer, Heidelberg (2012). doi:10.1007/978-3-642-28714-5_18
Berry, D.M., Kamsties, E., Krieger, M.M.: From contract drafting to software specification: linguistic sources of ambiguity (2003)
Bonin, F., Dell’Orletta, F., Montemagni, S., Venturi, G.: A contrastive approach to multi-word extraction from domain-specific corpora. In: LREC 2010 (2010)
CENELEC: EN 50128: 2011: Railway applications - communication, signalling and processing systems - software for railway control and protection systems. Technical report (2011)
Chantree, F., Nuseibeh, B., Roeck, A.N.D., Willis, A.: Identifying nocuous ambiguities in natural language requirements. In: RE 2006, pp. 56–65 (2006)
Cunningham, H.: GATE, a general architecture for text engineering. Comput. Humanit. 36(2), 223–254 (2002)
Cutts, M.: The Plain English Guide. Oxford University Press, Oxford (1996)
Falessi, D., Cantone, G., Canfora, G.: Empirical principles and an industrial case study in retrieving equivalent requirements via natural language processing techniques. IEEE Trans. Softw. Eng. 39(1), 18–44 (2013)
Femmer, H., Fernández, D.M., Wagner, S., Eder, S.: Rapid quality assurance with requirements smells. J. Syst. Softw. 123, 190–213 (2017)
Femmer, H., Kučera, J., Vetrò, A.: On the impact of passive voice requirements on domain modelling. In: ESEM 2014, p. 21. ACM (2014)
Ferrari, A., Gnesi, S.: Using collective intelligence to detect pragmatic ambiguities. In: RE 2012, pp. 191–200 (2012)
Gleich, B., Creighton, O., Kof, L.: Ambiguity detection: towards a tool explaining ambiguity sources. In: Wieringa, R., Persson, A. (eds.) REFSQ 2010. LNCS, vol. 6182, pp. 218–232. Springer, Heidelberg (2010). doi:10.1007/978-3-642-14192-8_20
Gnesi, S., Lami, G., Trentanni, G.: An automatic tool for the analysis of natural language requirements. Comput. Syst. Sci. Eng. 20(1), 53–62 (2005)
Kang, N., van Mulligen, E.M., Kors, J.A.: Comparing and combining chunkers of biomedical text. J. Biomed. Inform. 44(2), 354–360 (2011)
Landis, J.R., Koch, G.G.: The measurement of observer agreement for categorical data. Biometrics 33(1), 159–174 (1977)
Manning, C.D.: Part-of-speech tagging from 97% to 100%: is it time for some linguistics? In: Gelbukh, A.F. (ed.) CICLing 2011. LNCS, vol. 6608, pp. 171–189. Springer, Heidelberg (2011). doi:10.1007/978-3-642-19400-9_14
Mich, L., Franch, M., Inverardi, P.N.: Market research for requirements analysis using linguistic tools. REJ 9(1), 40–56 (2004)
Tjong, S.F., Berry, D.M.: The design of SREE — a prototype potential ambiguity finder for requirements specifications and lessons learned. In: Doerr, J., Opdahl, A.L. (eds.) REFSQ 2013. LNCS, vol. 7830, pp. 80–95. Springer, Heidelberg (2013). doi:10.1007/978-3-642-37422-7_6
Yang, H., Roeck, A.N.D., Gervasi, V., Willis, A., Nuseibeh, B.: Analysing anaphoric ambiguity in natural language requirements. REJ 16(3), 163–189 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Rosadini, B. et al. (2017). Using NLP to Detect Requirements Defects: An Industrial Experience in the Railway Domain. In: Grünbacher, P., Perini, A. (eds) Requirements Engineering: Foundation for Software Quality. REFSQ 2017. Lecture Notes in Computer Science(), vol 10153. Springer, Cham. https://doi.org/10.1007/978-3-319-54045-0_24
Download citation
DOI: https://doi.org/10.1007/978-3-319-54045-0_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-54044-3
Online ISBN: 978-3-319-54045-0
eBook Packages: Computer ScienceComputer Science (R0)