Advertisement

Concepts of Simulation Model Testing, Verification and Validation

  • David J. Murray-Smith
Part of the Simulation Foundations, Methods and Applications book series (SFMA)

Abstract

The suitability of a simulation model, in terms of its specific intended application, is an issue of great importance. Errors and uncertainties are always present in any simulation model and model testing and evaluation are inseparable from all the other processes involved in the iterative procedures of model development. Attempting to prove that a given model is “correct” or “valid” is impossible and it must be recognised that all models have limitations. Confidence in a model should increase steadily as it is developed, until the overall performance is judged to be acceptable for the planned application. The user of the simulation model must then have a good understanding of the remaining deficiencies, the performance to be expected from the model and its limitations in terms of the range of conditions over which it can be applied. A distinction is made between the processes of “verification”, which are concerned with the consistency of the simulation with the underlying mathematical model, and “validation” which relates to the degree to which the simulation model is an accurate representation of the corresponding real system. The general principles of model evaluation and testing are discussed within this chapter, including issues that arise in the verification and validation of sub-models.

Keywords

Computational Fluid Dynamics Real System Validation Process Intended Application Flight Test 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Sargent RG (1979) Validation of simulation models. In: Highland HJ (ed) Proceedings of the 1979 Winter Simulation conference, vol 2. IEEE Press, Piscataway, pp 497–503Google Scholar
  2. 2.
    Ören TI (1981) Concepts and criteria to assess acceptability of simulation studies: a frame of reference. In: Adam N (ed) Simulation Modeling and Statistical Computing. Communications of the ACM 24(4):180–188Google Scholar
  3. 3.
    Balci O (1997) Principles of simulation model validation. Trans Soc Comput Simul Int 14(1):3–12MathSciNetGoogle Scholar
  4. 4.
    Balci O (1997) Verification, validation and accreditation of simulation models. In: Andradóttir S, Healy KJ, Withers DH et al (eds) Proceedings of the 1997 Winter Simulation conference. IEEE Computer Society, Washington, DC, pp 135–147Google Scholar
  5. 5.
    Balci O (2004) Quality assessment, verification and validation of modeling and simulation applications. In: Ingailis RG, Rossetti MD, Smith JS et al (eds) Proceedings of the 2004 Winter Simulation conference. IEEE Computer Society, Washington, DC, pp 122–129Google Scholar
  6. 6.
    Brade D (2000) Enhancing modeling and simulation accreditation by structuring verification and validation results. In: Joines JA, Barton R, Kang K et al (eds) Proceedings of the 2000 Winter Simulation conference. Society for Modelling and Computer Simulation International, La Jolla, pp 840–848CrossRefGoogle Scholar
  7. 7.
    Brade D (2003) A generalized process for the verification and validation of models and simulation results. Dissertation, Fakultät für Informatik, Universität der Bundeswehr München, GermanyGoogle Scholar
  8. 8.
    American Institute of Aeronautics and Astronautics (2002) Guide: guide to the verification and validation of computational fluid dynamics simulations (AIAA G-077-1998(2002)), AIAA, RestonGoogle Scholar
  9. 9.
    Hemez FM (2004) The myth of science-based predictive modelling. In: Proceedings foundations ’04 workshop for verification, validation and accreditation (VV&A) in the 21st century, Arizona State University, Tempe, Arizona, 13–15 October 2004. Report LA-UR-04-6829, Los Alamos National Laboratory, USAGoogle Scholar
  10. 10.
    Montgomery DC, Conard RG (1980) Comparison of simulation and flight-test data for missile systems. Simulation 34(2):63–72CrossRefGoogle Scholar
  11. 11.
    Oberkampf WL (2007) Predictive capabilities in computational science and engineering. Presented at OASCR applied mathematics PI meeting, Lawrence Livermore National Laboratory, 22–24 May 2007. http://science.energy.gov/~/media/ascr/pdf/workshops-conferences/mathtalks/Oberkampf.pdf.Accessed 5 June 2015
  12. 12.
    Rasmussen CE, Ghahramani Z (2001) Occam’s razor. In: Leen TK, Dietterich TG, Tresp V (eds) Advances in Neural Information Processing Systems 13, Papers from Neural Information Processing Systems (NIPS) 2000, Denver CO, MIT Press, Cambridge MA, pp 294–300Google Scholar
  13. 13.
    The Mitre Corporation (2014) Verification and validation of simulation models. In: Mitre systems engineering guide, pp 461–469. www.mitre.org/publications/technical-papers/the-mitre-systems-engineering-guide. Accessed 5 June 2015
  14. 14.
    Pasquier M, Duoba M, Rousseau A (2001) Validating simulation tools for vehicle system studies using advanced control and testing procedures. In: Proceedings 18th international electric vehicle symposium (EVS18), Berlin, GermanyGoogle Scholar
  15. 15.
    SCS Technical Committee on Model Credibility (1979) Terminology for model credibility. Simulation 32(3):103–104CrossRefGoogle Scholar
  16. 16.
    Murray-Smith DJ (1998) Methods for the external validation of continuous system simulation models: a review. Math Comput Model Dyn Syst 4(1):5–31CrossRefzbMATHGoogle Scholar
  17. 17.
    Eddy DM, Hollingworth W, Caro JJ et al (2012) Model transparency and validation. A report of the ISPOR_SMDM Modeling Good Research Practices Task Force-7. Med Decis Making 35(5):733–743CrossRefGoogle Scholar
  18. 18.
    Oberkampf WL, Roy C (2010) Verification and validation in scientific computing. Cambridge University Press, CambridgeCrossRefzbMATHGoogle Scholar
  19. 19.
    American Society of Mechanical Engineers (ASME) Committee V&V40 verification and validation in computational modeling of medical devices and associated sub-groups. https://cstools.asme.org/csconnect/CommitteePages.cfm?Committee=100108782
  20. 20.
    American Society of Mechanical Engineers (ASME) Committee on verification and validation in computational modeling and simulation and associated sub-committees (V&V 10, V&V20, V&V30 and V&V40). https://cstools.asme.org/csconnect/CommitteePages.cfm?Committee=100003367. Accessed 5 June 2015
  21. 21.
    Modeling and Simulation Coordination Office (M&S CO). http://www.acq.osd.mil/se/org_msco.html. Accessed 5 June 2015
  22. 22.
    Kuhn DR, Chandramouli R, Butler RW (2002) Cost effective use of formal methods in verification and validation. Invited paper, Workshop on foundations for modeling and simulation (M&S) verification and validation (V&V) in the 21st century (Foundations ’02 Workshop), US Department of Defense, Laurel, Maryland, 22–23 October 2002Google Scholar
  23. 23.
    Cook DA, Skinner JM (2005). How to perform credible verification, validation and accreditation for modelling and simulation, J Def Softw Eng 18(5):20–24Google Scholar
  24. 24.
    Eisenberg N, Federline M, Sagar B et al (1995) Model validation from a regulatory perspective, GEOVAL’94, validation through model testing, Proceedings NAE/SKI symposium, Paris, France, 11–14 October 1994. Nuclear Energy Agency, Organisation for Economic Co-operation and Development, Paris, pp 421–434Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • David J. Murray-Smith
    • 1
  1. 1.School of EngineeringUniversity of GlasgowGlasgowUK

Personalised recommendations