Abstract
Simulation-based training can be used to evaluate communication and teamwork competencies among interdisciplinary teams and promote healthcare quality and safety (Rosen MA et al., Sim Health Care 3:33–41, 2008; Guise J et al., Sim Healthcare. 3:217–223, 2008; Goffman D et al., Semin perinatol. 37:140–142, 2013). Simulation has been used as an instructional design method to teach medical students, residents, and subspecialty fellows (Medical simulation in medical education: results of an AAMC survey. /https://www.aamc.org/download/259760/data/medical simulation in medical education an aamc survey.pdfS; Deering S et al., Semin Perinatol. 37:143–14, 2013). It is also recognized as a method of evaluating professional competence for practicing physician (Sidi A et al., J Grad Med Educ 489–494, 2014; Chiu M et al., Can J Anesth. 63:1357–1363, 2016). Simulation-based training is most effective when it is designed to successfully capture the knowledge, skills, and clinical reasoning that underlie expected behavior within the framework of the competencies assessed. Accurate and objective evaluation of trainee performance depends on the assessment method and the assessment tools selected. In this chapter, we will review a commonly recommended framework for evaluating the quality of an assessment tool. We end the chapter with examples of checklists and global rating scales, two common assessment tools that can be used during simulation-based training to determine if the expected clinical and procedural performance outcomes for learners have been achieved.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Rosen MA, Salas E, Wilson KA, et al. Measuring team performance in simulation-based training; adopting best practices for healthcare. Sim Health Care. 2008;3:33–41.
Guise J, Deering SH, Kanki BG, et al. Validation of a tool to measure and promote clinical teamwork. Sim Healthcare. 2008;3:217–23.
Goffman D, Lee C, Bernstein PS. Simulation in maternal–fetal medicine: making a case for the need. Semin perinatol. 2013;37:140–2.
Medical simulation in medical education: results of an AAMC survey. https://www.aamc.org/download/259760/data/medical simulation in medical education an aamc survey.pdfS.
Deering S, Auguste T, Lockrow E. Obstetric simulation for medical student, resident, and fellow education. Semin Perinatol. 2013;37:143–14.
Sidi A, Gravenstein N, Lampotang S. Construct validity and generalizability of simulation-based objective structured clinical examination scenarios. J Grad Med Educ. 2014;6(3):489–94.
Chiu M, Tarshis J, Antoniou A. Simulation-based assessment of anesthesiology residents’ competence: development and implementation of the Canadian National Anesthesiology Simulation Curriculum (CanNASC). Can J Anesth. 2016;63:1357–63.
Webster- Reliability. https://www.google.com/search?q=reliability&oq=reliability&aqs=chrome..69i57j0l5.2916j0j8&sourceid=chrome&ie=UTF-8.
Scalese, RJ, Hatala R. Competency assessment. The comprehensive textbook of healthcare simulation. Chapter 11. Springer Science Media: New York 2013. 135–160.
Cook DA, Hatala R. Validation of educational assessments: a primer for simulation and beyond. Adv Simul. 2016;1:31.
McEvoy MD, Hand WR, Furse CM, et al. Validity and reliability assessment of detailed scoring checklists for use during perioperative emergency simulation training. Simul Healthc : J Soc Simul Healthc. 2014;9(5):295–303.
Webster- Validity. https://www.google.com/#q=definition+of+validity+as+a+noun.
Andreatta PB, Gruppen LD. Conceptualising and classifying validity evidence for simulation. Med Educ. 2009;43:128–03.
Andreatta PB, Marzano DA, Curran DS. Validity: what does it mean for competency based assessment in obstetrics and gynecology? Am J Obstet Gynecol. 2011;204:384 e1–6.
Downing S. Validity: on the meaningful interpretation of assessment data. Med Educ. 2003;37:830–7.
Messick S. Validity of psychological assessment: validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. Am Psychologist. 1995;50:741–9.
Cook DA, Zendeja B, Hamstra SJ, Hatala R, Brydges R. What counts as validity evidence? Examples and prevalence in a systemic review of simulation based assessment. Adv Health Sci Educ. 2014;19:233–50.
Messick S. Standards of validity and the validity of standards in performance assessment. Educ Measure Issues Prac. 1995;14:50–8.
American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Validity. Standards for educational and psychological testing. Washington, DC: American Educational Research Association. 2014. pp. 11–31.
Cook DA, Brydges R, Zendejas B, et al. Technology-enhanced simulation to assess health professionals: A systematic review of validity evidence, research methods, and reporting quality. Acad Med:872–83. https://doi.org/10.1097/ACM.0b013e31828ffdcf.
Ilgen JS, Ma IWY, Hatala R, Cook DA. A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment. Med Educ. 2015;49:161–73.
Norman GR, Van der Vleuten CP, De Graaff E. Pitfalls in the pursuit of objectivity: issues of validity, efficiency and acceptability. Med Educ. 1991;25:119–26.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Garrison, E.A., Pippen, J.L. (2019). Competency Assessment in Simulation-Based Training: Educational Framework and Optimal Strategies. In: Deering, S., Auguste, T., Goffman, D. (eds) Comprehensive Healthcare Simulation: Obstetrics and Gynecology. Comprehensive Healthcare Simulation. Springer, Cham. https://doi.org/10.1007/978-3-319-98995-2_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-98995-2_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-98994-5
Online ISBN: 978-3-319-98995-2
eBook Packages: MedicineMedicine (R0)