A Credibility Assessment Method for Training Simulations from the View of Training Effectiveness

  • Shenglin Lin
  • Wei Li
  • Shuai Niu
  • Ping MaEmail author
  • Ming Yang
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 946)


The complex training simulation process often comprises multiple ordered tasks, and the training effects need to be evaluated from single task to multiple combined tasks for credibility assessment. In order to assess the credibility of training simulations, a novel credibility assessment method is proposed from the view of training effectiveness. An integrated credibility assessment framework for training simulations is given primarily involving project process, technical process and information artifacts. Then the single-task training effectiveness evaluation (TEE) method based on paired t-test and the multi-tasks TEE method using cluster analysis are proposed respectively for credibility assessment of complex training simulation. Finally, an application case about TEE for flight training simulation is given to illustrate the proposed method.


Training simulator Credibility assessment Training effectiveness evaluation Consistency analysis 



This research is supported by the National Natural Science Foundation of China (Grant No. 61627810).


  1. 1.
    Lin, S., Li, W., Ma, P., Yang, M.: A new credibility assessment framework for training simulators. In: 8th International Conference on Computer Modeling and Simulation, pp. 182–186. Association for Computing Machinery, Australia (2017)Google Scholar
  2. 2.
    Lemmers, A., Roza, M., Voogd, J., van Heck, F.: V&V study of an F-16 familiarization training simulator. In: Fall SIW Proceeding, pp. 13F-SIW-016. SISO, USA (2013)Google Scholar
  3. 3.
    Department of Defense: Verification Validation and Accreditation (VV&A) Recommended Practice Guide. Technical report. US Department of Defense, USA (1996)Google Scholar
  4. 4.
    IEEE Computer Society: Verification, validation, and accreditation for distributed interactive simulation, IEEE Std 1278.4 – 1997(R2010). Technical report (2010)Google Scholar
  5. 5.
    IEEE Computer Society: An overlay to the high level architecture federation development and execution process, IEEE Std 1516. Technical report (2007)Google Scholar
  6. 6.
    SISO: GM-VV Vol. 3: Reference Manual, SISO-GUIDE-001.3-2013. Technical report (2013)Google Scholar
  7. 7.
    Roza, M., Lemmers, A., van Heck, F.: Verification and validation of distributed air operation mission training environment. In: AIAA Modeling and Simulation Technology Conference, Boston, USA (2013)Google Scholar
  8. 8.
    Voogd, J., Roza, M., van Lier, A.: The GM-VV tailored for a naval ship-handling training simulation. In: International Simulation Multi-Conference (2011)Google Scholar
  9. 9.
    Brade, D., Pohl, S., Youngblood, S.: Preliminary convention report: first combined convention on International VV&A standardization endeavors (CConVV&A), Munich, Germany, October, pp. 20–23 (2005)Google Scholar
  10. 10.
    Brade, D.: WEAG THALES JP11.20 2005. Final state of the REVVA methodology. Spring SIW 2005, San Diego (2005)Google Scholar
  11. 11.
    ITOP: General procedure for modeling and simulation verification and validation information exchange. Technical report (2004)Google Scholar
  12. 12.
    Lynn, M., Michelle, F.: A validation study of driving errors using a driving simulator. Transp. Res. Part F 29, 14–21 (2015)CrossRefGoogle Scholar
  13. 13.
    Geraint, L.H., Thomas, P.C., Pritam, S.: Validation of a pediatric single-port laparoscopic surgery simulator. J. Pediatr. Surg. 50, 1762–1766 (2015)CrossRefGoogle Scholar
  14. 14.
    Ganesh, S., Baichun, L., Amie, M.: Face validation of the virtual electrosurgery skill trainer. Surg. Endosc. 30, 730–738 (2016)CrossRefGoogle Scholar
  15. 15.
    VanderVliet, G.M., Wilkinson, C.H., Roscoe, M.F.: Verification, validation, and accreditation of a flight simulator: the JSHIP experience. In: AIAA Modeling and Simulation Technology Conference, pp. 1–11. AIAA, Canada (2001)Google Scholar
  16. 16.
    Qiao, J.F., Hu, J.: Validation of discrete event state of simulation model. J. Syst. Simul. 16(4), 794–799 (2004)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  • Shenglin Lin
    • 1
  • Wei Li
    • 1
  • Shuai Niu
    • 1
  • Ping Ma
    • 1
    Email author
  • Ming Yang
    • 1
  1. 1.Control and Simulation CenterHarbin Institute of TechnologyHarbinChina

Personalised recommendations