Skip to main content
SpringerLink
Log in
Book cover

International SDL Forum

SDL 2007: SDL 2007: Design for Dependable Systems pp 54–68Cite as

TTCN-3 Quality Engineering: Using Learning Techniques to Evaluate Metric Sets

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNCCN,volume 4745))

Abstract

Software metrics are an essential means to assess software quality. For the assessment of software quality, typically sets of complementing metrics are used since individual metrics cover only isolated quality aspects rather than a quality characteristic as a whole. The choice of the metrics within such metric sets, however, is non-trivial. Metrics may intuitively appear to be complementing, but they often are in fact non-orthogonal, i.e. the information they provide may overlap to some extent. In the past, such redundant metrics have been identified, for example, by statistical correlation methods. This paper presents, based on machine learning, a novel approach to minimise sets of metrics by identifying and removing metrics which have little effect on the overall quality assessment. To demonstrate the application of this approach, results from an experiment are provided. In this experiment, a set of metrics that is used to assess the analysability of test suites that are specified using the Testing and Test Control Notation (TTCN-3) is investigated.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Fenton, N.E., Pfleeger, S.L.: Software Metrics. PWS Publishing, Boston (1997)

    Google Scholar 

  2. CMMI Product Team: CMMI for Development, Version 1.2. Technical Report CMU/SEI-2006-TR-008, Carnegie Mellon University, Software Engineering Institute (2006)

    Google Scholar 

  3. ISO/IEC: ISO/IEC Standard No. 15504: Information technology – Process Assessment; Parts 1–5. International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC), Geneva, Switzerland (2003-2006)

    Google Scholar 

  4. ETSI: ETSI Standard (ES) 201 873-1 V3.2.1 (2007-02): The Testing and Test Control Notation version 3; Part 1: TTCN-3 Core Language. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis, France, also published as ITU-T Recommendation Z.140 (February 2007)

    Google Scholar 

  5. Grabowski, J., Hogrefe, D., Réthy, G., Schieferdecker, I., Wiles, A., Willcock, C.: An introduction to the testing and test control notation (TTCN-3). Computer Networks 42(3), 375–403 (2003)

    Article  MATH  Google Scholar 

  6. Zeiss, B., Neukirchen, H., Grabowski, J., Evans, D., Baker, P.: Refactoring and Metrics for TTCN-3 Test Suites. In: Gotzhein, R., Reed, R. (eds.) SAM 2006. LNCS, vol. 4320, pp. 148–165. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  7. Zeiss, B., Vega, D., Schieferdecker, I., Neukirchen, H., Grabowski, J.: Applying the ISO 9126 Quality Model to Test Specifications – Exemplified for TTCN-3 Test Specifications. In: Bleek, W.G., Raasch, J., Züllighoven, H. (eds.) Software Engineering 2007, Bonn, Gesellschaft für Informatik. Lecture Notes in Informatics (LNI), vol. 105, pp. 231–242. Köllen Verlag (March 2007)

    Google Scholar 

  8. ISO/IEC: ISO/IEC Standard No. 9126: Software engineering – Product quality; Parts 1–4. International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC), Geneva, Switzerland (2001-2004)

    Google Scholar 

  9. Halstead, M.: Elements of Software Science. Elsevier, New York (1977)

    MATH  Google Scholar 

  10. McCabe, T.J.: A Complexity Measure. IEEE Transactions on Software Engineering 2(4), 308–320 (1976)

    Article  MathSciNet  Google Scholar 

  11. Watson, A.H., McCabe, T.J.: Structured Testing: A Testing Methodology Using the Cyclomatic Complexity Metricy. NIST Special Publication 500-235, National Institute of Standards and Technology, Computer Systems Laboratory, Gaithersburg, MD, United States of America (1996)

    Google Scholar 

  12. Fan, C.F., Yih, S.: Prescriptive metrics for software quality assurance. In: Proceedings of the First Asia-Pacific Software Engineering Conference, Tokyo, Japan, pp. 430–438. IEEE Computer Society Press, Los Alamitos (1994)

    Google Scholar 

  13. Basili, V.R., Weiss, D.M.: A Methodology for Collecting Valid Software Engineering Data. IEEE Transactions on Software Engineering 10(6), 728–738 (1984)

    Article  Google Scholar 

  14. Henry, S., Kafura, D., Harris, K.: On the Relationships Among Three Software Metrics. In: Proceedings of the 1981 ACM Workshop/Symposium on Measurement and Evaluation of Software Quality, pp. 81–88. ACM Press, New York (1981)

    Google Scholar 

  15. Valiant, L.: A theory of learnability. Communications of the ACM 27(11), 1134–1142 (1984)

    Article  MATH  Google Scholar 

  16. Valiant, L.: Deductive learning. Philosophical Transactions of the Royal Society London A 312, 441–446 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  17. Vapnik, V., Chervonenkis, A.Y.: On the uniform convergence of relative frequencies of events and their probabilities. Theory of Probabilty and its Applications 16(2), 264–280 (1971)

    Article  MATH  MathSciNet  Google Scholar 

  18. Blumer, A., Ehrenfeucht, A., Haussler, D., Warmuth, M.K.: Learnability and the Vapnik-Chervonenkis dimension. Journal of the ACM 36(4), 929–969 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  19. Vapnik, V.: The Nature of Statistical Lerning Theory. Springer, New York (1995)

    Google Scholar 

  20. Koltchinskii, V.I., Pachenko, D.: Rademacher processes and bounding the risk of learning function. High Dimensional Probability II, 443–459 (2000)

    Google Scholar 

  21. Kearns, M.J., Vazirani, U.V.: An Introduction to Computational Learning Theory. MIT Press, Cambridge (1994)

    Google Scholar 

  22. Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)

    Google Scholar 

  23. Massart, P.: Some applications of concentration inequalities to statistics. Annales de la Faculté des Sciences de Toulouse, 245–303 vol. spécial dédié à Michel Talagrand (2000)

    Google Scholar 

  24. ETSI: Technical Specification (TS) 102 027-3 V3.2.1 (2005-07): SIP ATS & PIXIT; Part 3: Abstract Test Suite (ATS) and partial Protocol Implementation eXtra Information for Testing (PIXIT). European Telecommunications Standards Institute (ETSI), Sophia-Antipolis, France (July 2005)

    Google Scholar 

  25. ETSI: Technical Specification (TS) 102 516 V1.1 (2006-04): IPv6 Core Protocol; Conformance Abstract Test Suite (ATS) and partial Protocol Implementation eXtra Information for Testing (PIXIT). European Telecommunications Standards Institute (ETSI), Sophia-Antipolis, France (April 2006)

    Google Scholar 

  26. Baker, P., Evans, D., Grabowski, J., Neukirchen, H., Zeiss, B.: TRex – The Refactoring and Metrics Tool for TTCN-3 Test Specifications. In: Proceedings of TAIC PART 2006 (Testing: Academic & Industrial Conference – Practice And Research Techniques), Cumberland Lodge, Windsor Great Park, UK, 29th–31st August 2006, IEEE Computer Society Press, Los Alamitos (2006)

    Google Scholar 

  27. TRex Team: TRex Website (2007), http://www.trex.informatik.uni-goettingen.de

Download references

Author information

Authors and Affiliations

  1. Institute for Informatics, University of Göttingen, Lotzestr. 16–18, 37083 Göttingen, Germany

    Edith Werner, Jens Grabowski, Helmut Neukirchen, Nils Röttger, Stephan Waack & Benjamin Zeiss

Authors
  1. Edith Werner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jens Grabowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Helmut Neukirchen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nils Röttger
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stephan Waack
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Benjamin Zeiss
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Emmanuel Gaudin Elie Najm Rick Reed

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Werner, E., Grabowski, J., Neukirchen, H., Röttger, N., Waack, S., Zeiss, B. (2007). TTCN-3 Quality Engineering: Using Learning Techniques to Evaluate Metric Sets. In: Gaudin, E., Najm, E., Reed, R. (eds) SDL 2007: Design for Dependable Systems. SDL 2007. Lecture Notes in Computer Science, vol 4745. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74984-4_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-74984-4_4

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-74983-7

  • Online ISBN: 978-3-540-74984-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation