Skip to main content
SpringerLink
Log in

Verification and Validation

  • Chapter
  • First Online:
Estimating Impact

Abstract

“All models are wrong; but some are useful.” This well-known adage by the statistician, George Box, contains several important insights (Box 1979) – first, that a model is an abstraction of reality, and second, that the parts of a model which are important for its intended purpose should be emphasized, while those which are nonessential can be deemphasized or even left out. Consider, for example, “model” airplanes. Display models, such as those designed to be hung from ceilings, are concerned with proportions vis a vis size, material composition, or flying capability. If they look right, they make for satisfied users. Models intended for flying, on the other hand, sacrifice proportions and composition in order to produce a device that “performs.” Since neither model reproduces reality, each is inherently “wrong”; however, each may meet its intended purpose quite well. The situation is similar with software models. A skilled modeler should focus on those aspects of reality that are germane to the issue of interest. If he or she succeeds, and the abstraction effectively addresses the factors relevant to the model’s purpose, then it is possible that the model will be “useful.”

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Balci, O. (1997) Verification, Validation and Accreditation of Simulation Models. In S. Andradottir, K. J. Healy, D. H. Withers, and B. L. Nelson (Eds.) Proceedings of the 1997 Winter Simulation Conference (pp. 135–141). Washington, DC.

    Google Scholar 

  • Balci, O. (2001) Verification, Validation and Testing of Models. In S. I. Gass, and C. M. Harris (Eds.) Encyclopedia of Operations Research and Management Science. Kluwer Academic Publishers, Boston, MA.

    Google Scholar 

  • Balci, O., et al. (2002) A Collaborative Evaluation Environment for Credibility Assessment of Modeling and Simulation Applications. In E. Yucesan, C. H. Chen, J. L. Snowdon, and J. M. Charnes (Eds.) Proceedings of the 2002 Winter Simulation Conference, (pp. 214–220). Washington, DC.

    Google Scholar 

  • Bauman, W. (1995) Ardennes Campaign Simulation (ARCAS), CAA-SR-95-8. US Army Concepts Analysis Agency, Bethesda, MD.

    Google Scholar 

  • Box, G. E. P. (1979) Robustness in the Strategy of Scientific Model Building. In R.L. Launer, and G.N. Wilkinson (Eds.) Robustness in Statistics. Academic Press, New York.

    Google Scholar 

  • Chaturvedi, A. R. (2003) SEAS-UV 2004: The Theoretical Foundation of a Comprehensive PMESII/DIME Agent-Based Synthetic Environment. Simulex, Inc.

    Google Scholar 

  • Clemence, R. C., et al. (2007) Verification, Validation, and Accreditation (VV&A). Evidence Based Research, Vienna, VA.

    Google Scholar 

  • United States Coast Guard (2006) Changes to Deepwater Plan Appear Sound, and Program Management Has Improved, but Continued Monitoring Is Warranted. GAO, Washington, DC.

    Google Scholar 

  • Conwell, C. L., et al. (2000) Capability Maturity Models Support of Modeling and Simulation Verification, Validation, and Accreditation. In J. A. Joines, R. R. Barton, K. Kang, and P. A. Fishwick (Eds.) Proceedings of the 2000 Winter Simulation Conference (pp. 819–828). Washington, DC.

    Google Scholar 

  • Davis, P. K. (1992) Chapter VI – A Framework for Verification, Validation, and Accreditation. In A. E. Ritchie (Ed.) Simulation Validation Workshop Proceedings (SIMVAL II). MORS, Alexandria, VA.

    Google Scholar 

  • Davis, D. F. (2006) Consolidated Report Consisting of three Research and Development Project Summary Reports. Contract #W15P7T-06-T-P238. Peace Operations Policy Program. George Mason University, Arlington, VA.

    Google Scholar 

  • Department of the Army, Pamphlet 5-11 (1999) Verification, Validation, and Accreditation of Army Models and Simulations. Washington, DC.

    Google Scholar 

  • DMSO VV&A website (as of 2007) http://vva.dmso.mil/.

  • DoD Directive 5000.59 (2007) DoD Modeling and Simulation (M&S) Management. http://www.dtic.mil/whs/directives/corres/pdf/500059p.pdf.

  • Giadrosich, D. (1992) Chapter IV-A – Validating Models and Simulations. In A. E. Ritchie (Ed.) Simulation Validation Workshop Proceedings (SIMVAL II). MORS, Alexandria, VA.

    Google Scholar 

  • Hartley III, D. S. (1975) An Examination of a Distribution of TAC CONTENDER Solutions. National Military Command System Support Center, Washington, DC.

    Google Scholar 

  • Hartley III, D. S., et al. (1989) Sensitivity Analysis of the Joint Theater Level Simulation I K/DSRD-70. Martin Marietta Energy Systems, Inc., Oak Ridge, TN.

    Google Scholar 

  • Hartley III, D. S., Quillinan, J. D. and Kruse, K. L. (1990) Verification and Validation of SIMNET-T, KJDSRD-117. Martin Marietta Energy Systems, Inc., Oak Ridge, TN.

    Google Scholar 

  • Hartley III, D. S. (1991) Confirming the Lanchestrian Linear-Logarithmic Model of Attrition, K/DSRD-263/R1. Martin Marietta Energy Systems, Inc., Oak Ridge, TN.

    Google Scholar 

  • Hartley III, D. S., Radford, C. and Snyder, C. E. (1991) Evaluation of the Advanced Battle Simulation in WARES 3-90, K/DSRD-597. Martin Marietta Energy Systems, Inc., Oak Ridge, TN.

    Google Scholar 

  • Hartley III, D. S., et al. (1994) An Independent Verification And Validation of The Future Theater Level Model Conceptual Model, K/DSRD-1637. Martin Marietta Energy Systems, Inc., Oak Ridge, TN.

    Book  Google Scholar 

  • Hartley III, D. S. (1996) Operations Other Than War: Requirements for Analysis Tools Research Report, K/DSRD-2098. Lockheed Martin Energy Systems, Inc., Oak Ridge, TN.

    Book  Google Scholar 

  • Hartley III, D. S. (1997) Verification & Validation in Military Simulations. In S. Andradóttir, K. J. Healy, D. H. Withers, and B. L. Nelson (Eds.) Proceedings of the 1997 Winter Simulation Conference (pp. 925–932). Washington, DC. http://www.informs-sim.org/wsc97papers/0925.PDF.

  • Hartley III, D. S. (2001) Predicting Combat Effects. Linthicum, MD: INFORMS.

    Google Scholar 

  • Hartley III, D. S. (2004) FAST for the Warfighter: Test Strategy & Plan (Revision 3) Dynamics Research Corporation, Vienna, VA.

    Google Scholar 

  • Hartley III, D. S. (2005a) MOOTW FAST Prototype Toolbox: FY04 Validation Strategy & Plan. Dynamics Research Corporation, Vienna, VA.

    Google Scholar 

  • Hartley III, D. S. (2005b). MOOTW FAST Prototype Toolbox: FY05 Validation Strategy & Plan. Dynamics Research Corporation, Vienna, VA.

    Google Scholar 

  • Hartley III, D. S., Holdsworth, D. and Farrell, C. (2006) OOTW FAST Prototype Toolbox: Analysis Process. Dynamics Research Corporation, Orlando, FL.

    Google Scholar 

  • Hayes, B. C. and Sands, J. I. (1997) Doing Windows: Non-Traditional Military Responses to Complex Emergencies. Command & Control Research Program, Washington, DC.

    Google Scholar 

  • Henderson, D. (1992) Chapter IV-B – The Multidimensional Space of Validation. In A. E. Ritchie (Ed.) Simulation Validation Workshop Proceedings (SIMVAL II). MORS, Alexandria, VA.

    Google Scholar 

  • IEEE 610-1991 (1991) IEEE Standard Computer Dictionary. A Compilation of IEEE Standard Computer Glossaries. http://ieeexplore.ieee.org/xpl/tocresult.jsp?isnumber=4683.

  • IEEE 1278.1-1995 (1996) IEEE Standard for Distributed Interactive Simulation – Application Protocols. http://ieeexplore.ieee.org/xpl/tocresult.jsp?isNumber=10849.

  • IEEE 1278.1A-1998 (1998) IEEE Standard for Distributed Interactive Simulation – Application Protocols. http://usl.sis.pitt.edu/wjj/otbsaf/IEEE1278.1a-1998.pdf.

  • IEEE 1516.1-2000 (2000a) IEEE Standard for Modeling and Simulation (M&S) High Level Architecture (HLA) Federate Interface Specification. http://ieeexplore.ieee.org/xpl/standardstoc.jsp?isnumber=19743.

  • IEEE Std 1516-2000 (2000b) IEEE Standard for Modeling and Simulation (M&S) High Level Architecture (HLA) – Framework and Rules. http://ieeexplore.ieee.org/xpl/standardstoc.jsp?isnumber=19334.

  • IEEE 1516.2-2000 (2001) IEEE Standard for Modeling and Simulation (M&S) High Level Architecture (HLA) – Object Model Template (OMT) specification. http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?tp=&isnumber=19791&arnumber=915738&punumber=7319.

  • Kott, A. (2008) Fiction vs. Reality: Validation of Decision-Support Models via Cognitive Impact. Keynote speech at MODSIM World 2008 Conference, Sep. 15–18, Virginia Beach, VA.

    Google Scholar 

  • Knepell, P. L. and Arangno, D. C. (1993) Simulation Validation: A Confidence Assessment Methodology. IEEE Computer Society Press, Los Alamitos, CA.

    Google Scholar 

  • Meadows, D. H., Meadows, D. L., Randers, J. and Behrens, W. (1972) The Limits to Growth. Signet, New York.

    Google Scholar 

  • McDonald, C. S. (2006) Verification and Validation Report for the Synthetic Environment for Analysis and Simulation (SEAS). The Johns Hopkins University – Applied Physics Laboratory, Laurel, MD.

    Google Scholar 

  • Metz, M. L. (2000) Joint Warfare System (JWARS) Verification and Validation Lessons Learned. In J. A. Joines, R. R. Barton, K. Kang, and P. A. Fishwick (Eds.) Proceedings of the 2000 Winter Simulation Conference (pp. 855–858). Washington, DC.

    Google Scholar 

  • Sargent, R. G. (2004) Validation and Verification of Simulation Models. In R. G. Ingalls, M. D. Rossetti, J. S. Smith, and B. A. Peters (Eds.) Proceedings of the 2004 Winter Simulation Conference. Washington, DC.

    Google Scholar 

  • Senko, R. M. (2005) Flexible Asymmetric Simulation Technologies (FAST) for the Warfighter: FY05 Final Verification Test Report. Dynamics Research Corporation, Vienna, VA.

    Google Scholar 

  • Sheldon, B. (2006) Memorandum for the Record: V&V Report for the Synthetic Environment for Analysis and Simulation (SEAS). 27 October 2006.

    Google Scholar 

  • Simulation Verification, Validation and Accreditation Guide (2005) Australian Defence Simulation Office, Department of Defence, Canberra, Australia.

    Google Scholar 

  • Staniec, C. and Hartley III, D. S. (1997) OOTW Analysis and Modeling Techniques (OOTWAMT) Workshop Proceedings. MORS, Alexandria, VA.

    Google Scholar 

  • Secretary of the Navy Instruction 5200.40 (1999) Verification, Validation, and Accreditation (VV&A) of Models and Simulations. Washington, DC.

    Google Scholar 

  • Vector Research (1981) Verification Analysis of VECTOR-2 with the 1973 Arab-Israeli War and Analysis of Related Military Balance Issues. VRI-NGIC-1 FR 81-1. VRI., Ann Arbor, MI.

    Google Scholar 

  • Wait, P. (2001) Project Protection Comes at a Price. Washington Technology. http://washingtontechnology.com/articles/2001/08/24/project-protection-comes-at-a-price.aspx.

  • Waltz (2008)

    Google Scholar 

  • Youngblood, S. (2004) DoDI 5000.61 and the VV&A RPG. Defense Modeling and Simulation Office, Washington, DC.

    Google Scholar 

  • Youngblood, S. (2005) VV&A Standards Initiatives. Briefing at the NDIA M&S Committee Meeting. Washington, DC. http://www.ndia.org/Divisions/Divisions/SystemsEngineering/Documents/Content/ContentGroups/Divisions1/Systems_Engineering/PDFs18/Modeling_Committee_PDFs/VVA%20Youngblood%20to%20NDIA%20MS%20Committee%20-%2007%20Feb%202007%20v2.pdf.

Download references

Author information

Authors and Affiliations

  1. George Mason University School of Public Policy, 4400 University Drive, Fairfax, VA, 22030, USA

    Dean Hartley

Authors
  1. Dean Hartley
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stuart Starr
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dean Hartley .

Editor information

Editors and Affiliations

  1. Army Research Laboratory, Powder Mill Road 2800, Adelphi, 20783-1197, Maryland, USA

    Alexander Kott

  2. SoSA Corporation, Fairfax Ridge Rd. 3877, Fairfax, 20030, Virginia, USA

    Gary Citrenbaum

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer US

About this chapter

Cite this chapter

Hartley, D., Starr, S. (2010). Verification and Validation. In: Kott, A., Citrenbaum, G. (eds) Estimating Impact. Springer, Boston, MA. https://doi.org/10.1007/978-1-4419-6235-5_11

Download citation

  • DOI: https://doi.org/10.1007/978-1-4419-6235-5_11

  • Published:

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4419-6234-8

  • Online ISBN: 978-1-4419-6235-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation