Abstract
In the context of simulation model development, the word “validation” is used to describe the process of establishing the extent to which a given model is an accurate representation of the corresponding real-world system. In practice, the emphasis should always be on “invalidation” since a model can never be proved to be, in any general sense, “correct”. This chapter provides an introduction to methods to be used for this process and includes quantitative methods, such as those based on system identification and parameter-estimation principles, barrier certificate methods, techniques based on concepts of model distortion and methods based on parameter sensitivity analysis. Methods of face validation, involving a more subjective approach and the opinion of experts, familiar with the real system, are also considered. That type of validation process is illustrated using an example of the development and testing of a complex simulation of a hydro-turbine generating system. Further sections of this chapter are devoted to approaches based on comparison with other models, the choice of data sets for model testing and validation, the validation of sub-models and generic models, issues relating to the validation of distributed parameter models and the validation of discrete-event and hybrid models. Before the final discussion section there is also a short review of issues arising in the acceptance or upgrading of models.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Oberkampf WL (2007) Predictive capabilities in computational science and engineering. Presented at OASCR Applied Mathematics PI Meeting, Lawrence Livermore National Laboratory, 22–24 May 2007. http://science.energy.gov/~/media/ascr/pdf/workshops-conferences/mathtalks/Oberkampf.pdf. Accessed 10 June 2015
Anderson J, Papachristodoulou A (2009) On validation and invalidation of biological models. BMC Bioinforma 10:132. doi:10.1186/1471-2105-10-132, http://www.biomedcentral.com/1471-2105/10/132. Accessed 10th June 2015
Gore R, Diallo S (2013) The need for usable formal methods in verification and validation. In: Pasupathy R, Kim S-H, Tolk A et al (eds) Proceedings of the 2013 winter simulation conference. IEEE, Washington, DC, pp 1257–1268. doi:10.1109/WSC.2013.6721513
Heitmeyer CL (2007) Formal methods for specifying, validating and verifying requirements. J Univ Comput Sci 13(5):607–618
Gustavsson I (1972) Comparison of different methods for identification of industrial processes. Automatica 8(2):127–142
Prajna S (2003) Barrier certificates for nonlinear model validation. In: Proceedings 42nd IEEE conference on decision and control 9–12 Dec 2003, vol 3. IEEE, Washington, DC. pp 2884–2889, doi:10.1109/CDC.2003.1273063
Tischler MB, Remple RK (2012) Aircraft and rotorcraft system identification, 2nd edn. AIAA, Reston, VA
Knudsen M (2006) Experimental modelling of dynamic systems: an educational approach. IEEE Trans Educ 49(1):29–38
Gong M, Murray-Smith DJ (1993) Model reduction by an extended complex curve-fitting approach. Trans Inst Meas Control 15(4):188–198
Balci O, Sargent R (1984) Validation of simulation models via simultaneous confidence intervals. Am J Math Manag Sci 4:375–406
McFarland J, Mahadevan S (2008) Multivariate significance tests and model calibration under uncertainty. Comput Methods Appl Mech Eng 197(29–32):2407–2479
Huynh DPB, Knezevic DJ, Patera AT (2012) Certified reduced basis model characterization: a frequentistic uncertainty framework. Comput Methods Appl Mech Eng 201:13–24
Rosenberg JR, Murray-Smith DJ, Rigas A (1982) An introduction to the application of system identification techniques to elements of the neuromuscular system. Trans Inst Meas Control 4(4):187–201
Butterfield MH, Thomas PJ (1986) Methods of quantitative validation for dynamic system models-part 1: theory. Trans Inst Meas Control 8:182–200
Cameron RG (1998) Model validation by the distortion method: linear state space systems control theory and applications. IEE Proc D 139(3):296–300
Thomas PJ (1999) Simulation of industrial processes for control engineers. Butterworth-Heinemann, Oxford
Cameron RG, Marcos RL, De Prada C (1998) Model validation of discrete transfer functions using the distortion method. Math Comput Model Dyn Syst 4(1):58–72
Gray GJ, Voon LK, Murray-Smith (1997) Application of the distortion method for model validation. In: Troch I, Breitenecker F (eds) Proceedings 2nd MATHMOS VIENNA IMACS symposium on mathematical modelling February 1997. Argesim, Vienna, pp 1033–1038
Butterfield MH, Thomas PJ (1987) Quantitative validation of dynamic models for use in fast reactor safety assessments. In: Proceedings international conference on science and technology of fast reactor safety, Guernsey (UK), 12–16 May 1986. British Nuclear Energy Society, London, pp 145–152
Butterfield MH, Thomas PJ (1986) Methods of quantitative validation for dynamic system models-part 2: applications. Trans Inst Meas Control 8:201–219
Butterfield MH (1990) A method of quantitative validation based on model distortion. Trans Inst Meas Control 12:167–173
Kleijnen JPC (1995) Verification and validation of simulation models. Eur J Oper Res 82:145–162
Chattee E, Saam NJ, Möhring M (2000) Sensitivity analysis in the social sciences: problems and prospects. In: Suleiman et al (eds) Tools and techniques for social science simulation, chapter 3. Physica Verlag, Heidelberg
Bryce GW, Foord TR, Murray-Smith DJ, Agnew P (1976) Hybrid simulation of water turbine governors. In: Crosbie RE, Hay JL (eds) Simulations councils proceedings series 6(1), Simulation Councils, La Jolla, pp 35–44
Bryce GW, Agnew PW, Foord TR et al (1977) On-site investigation of electrohydraulic governors for water turbines. Proc IEE 124(2):147–153
The Mitre Corporation (2014) Verification and validation of simulation models. In: Mitre systems engineering guide, pp 461–469. www.mitre.org/publications/technical-papers/the-mitre-systems-engineering-guide. Accessed 10 June 2015
Hemez FM (2004) The myth of science-based predictive modelling. In: Proceedings foundations’04 workshop for verification, validation and accreditation (VV&A) in the 21st century, Arizona State University, Tempe, Arizona, 13–15 October 2004. Report LA-UR-04-6829, Los Alamos National Laboratory, USA
Kocijan J, Girard A, Banko B et al (2005) Dynamic system identification with dynamic processes. Math Comput Model Dyn Syst 11(4):411–424
Tischler MB (1996) System identification for aircraft flight control development and validation. In: Tischler MB (ed) Advances in aircraft flight control. Taylor and Francis, London, pp 35–69
Tischler MB, Remple RK (2006) Aircraft and rotorcraft system identification. AIAA, Reston
Smith MI, Murray-Smith DJ, Hickman D (2007) Verification and validation issues in a generic model of electro-optic sensor systems. Def Model J Simul 4(1):17–17
Smith MI, Murray-Smith DJ, Hickman D (2007) Mathematical and computer modeling of electro-optic systems using a generic modeling approach. J Def Model Simul 4(1):3–16
Walmsley CW, McCurry MR, Clausen PD et al (2013) Beware the black box: investigating the sensitivity of FEA simulation to modelling factors in comparative biomechanics. PeerJ 1:e204, http://dx.doi.org/10.7717/peerj.204}. Accessed 10 June 2015
Rizzi A, Vos J (1998) Towards establishing credibility in computational fluid dynamics simulations. AIAA J 36(5):668–675
Thompson, DE (2005) Verification, validation and solution quality in computational physics: CFD methods applied to ice sheets, NASA/TM-2005-213453, NASA Technical Reports Server, 37 pp
Padfield GP, Du Val RW (1991) Application areas for rotorcraft system identification: simulation model validation. In: AGARD Lecture Series 178, Rotorcraft System Identification, 12.1-12.30, AGARD, Neuilly-sur-Seine, France
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Murray-Smith, D.J. (2015). Methods for the Invalidation/Validation of Simulation Models. In: Testing and Validation of Computer Simulation Models. Simulation Foundations, Methods and Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-15099-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-15099-4_7
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-15098-7
Online ISBN: 978-3-319-15099-4
eBook Packages: Computer ScienceComputer Science (R0)