Skip to main content

Competency Assessment in Simulation-Based Training: Educational Framework and Optimal Strategies

  • Chapter
  • First Online:
Book cover Comprehensive Healthcare Simulation: Obstetrics and Gynecology

Part of the book series: Comprehensive Healthcare Simulation ((CHS))

  • 712 Accesses

Abstract

Simulation-based training can be used to evaluate communication and teamwork competencies among interdisciplinary teams and promote healthcare quality and safety (Rosen MA et al., Sim Health Care 3:33–41, 2008; Guise J et al., Sim Healthcare. 3:217–223, 2008; Goffman D et al., Semin perinatol. 37:140–142, 2013). Simulation has been used as an instructional design method to teach medical students, residents, and subspecialty fellows (Medical simulation in medical education: results of an AAMC survey. /https://www.aamc.org/download/259760/data/medical simulation in medical education an aamc survey.pdfS; Deering S et al., Semin Perinatol. 37:143–14, 2013). It is also recognized as a method of evaluating professional competence for practicing physician (Sidi A et al., J Grad Med Educ 489–494, 2014; Chiu M et al., Can J Anesth. 63:1357–1363, 2016). Simulation-based training is most effective when it is designed to successfully capture the knowledge, skills, and clinical reasoning that underlie expected behavior within the framework of the competencies assessed. Accurate and objective evaluation of trainee performance depends on the assessment method and the assessment tools selected. In this chapter, we will review a commonly recommended framework for evaluating the quality of an assessment tool. We end the chapter with examples of checklists and global rating scales, two common assessment tools that can be used during simulation-based training to determine if the expected clinical and procedural performance outcomes for learners have been achieved.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Rosen MA, Salas E, Wilson KA, et al. Measuring team performance in simulation-based training; adopting best practices for healthcare. Sim Health Care. 2008;3:33–41.

    Article  Google Scholar 

  2. Guise J, Deering SH, Kanki BG, et al. Validation of a tool to measure and promote clinical teamwork. Sim Healthcare. 2008;3:217–23.

    Article  Google Scholar 

  3. Goffman D, Lee C, Bernstein PS. Simulation in maternal–fetal medicine: making a case for the need. Semin perinatol. 2013;37:140–2.

    Article  Google Scholar 

  4. Medical simulation in medical education: results of an AAMC survey. https://www.aamc.org/download/259760/data/medical simulation in medical education an aamc survey.pdfS.

  5. Deering S, Auguste T, Lockrow E. Obstetric simulation for medical student, resident, and fellow education. Semin Perinatol. 2013;37:143–14.

    Article  Google Scholar 

  6. Sidi A, Gravenstein N, Lampotang S. Construct validity and generalizability of simulation-based objective structured clinical examination scenarios. J Grad Med Educ. 2014;6(3):489–94.

    Article  Google Scholar 

  7. Chiu M, Tarshis J, Antoniou A. Simulation-based assessment of anesthesiology residents’ competence: development and implementation of the Canadian National Anesthesiology Simulation Curriculum (CanNASC). Can J Anesth. 2016;63:1357–63.

    Article  Google Scholar 

  8. Webster- Reliability. https://www.google.com/search?q=reliability&oq=reliability&aqs=chrome..69i57j0l5.2916j0j8&sourceid=chrome&ie=UTF-8.

  9. Scalese, RJ, Hatala R. Competency assessment. The comprehensive textbook of healthcare simulation. Chapter 11. Springer Science Media: New York 2013. 135–160.

    Google Scholar 

  10. Cook DA, Hatala R. Validation of educational assessments: a primer for simulation and beyond. Adv Simul. 2016;1:31.

    Article  Google Scholar 

  11. McEvoy MD, Hand WR, Furse CM, et al. Validity and reliability assessment of detailed scoring checklists for use during perioperative emergency simulation training. Simul Healthc : J Soc Simul Healthc. 2014;9(5):295–303.

    Article  Google Scholar 

  12. Webster- Validity. https://www.google.com/#q=definition+of+validity+as+a+noun.

  13. Andreatta PB, Gruppen LD. Conceptualising and classifying validity evidence for simulation. Med Educ. 2009;43:128–03.

    Google Scholar 

  14. Andreatta PB, Marzano DA, Curran DS. Validity: what does it mean for competency based assessment in obstetrics and gynecology? Am J Obstet Gynecol. 2011;204:384 e1–6.

    Article  Google Scholar 

  15. Downing S. Validity: on the meaningful interpretation of assessment data. Med Educ. 2003;37:830–7.

    Article  Google Scholar 

  16. Messick S. Validity of psychological assessment: validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. Am Psychologist. 1995;50:741–9.

    Article  Google Scholar 

  17. Cook DA, Zendeja B, Hamstra SJ, Hatala R, Brydges R. What counts as validity evidence? Examples and prevalence in a systemic review of simulation based assessment. Adv Health Sci Educ. 2014;19:233–50.

    Article  Google Scholar 

  18. Messick S. Standards of validity and the validity of standards in performance assessment. Educ Measure Issues Prac. 1995;14:50–8.

    Google Scholar 

  19. American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Validity. Standards for educational and psychological testing. Washington, DC: American Educational Research Association. 2014. pp. 11–31.

    Google Scholar 

  20. Cook DA, Brydges R, Zendejas B, et al. Technology-enhanced simulation to assess health professionals: A systematic review of validity evidence, research methods, and reporting quality. Acad Med:872–83. https://doi.org/10.1097/ACM.0b013e31828ffdcf.

    Article  Google Scholar 

  21. Ilgen JS, Ma IWY, Hatala R, Cook DA. A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment. Med Educ. 2015;49:161–73.

    Article  Google Scholar 

  22. Norman GR, Van der Vleuten CP, De Graaff E. Pitfalls in the pursuit of objectivity: issues of validity, efficiency and acceptability. Med Educ. 1991;25:119–26.

    Article  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Etoi A. Garrison .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer International Publishing AG, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Garrison, E.A., Pippen, J.L. (2019). Competency Assessment in Simulation-Based Training: Educational Framework and Optimal Strategies. In: Deering, S., Auguste, T., Goffman, D. (eds) Comprehensive Healthcare Simulation: Obstetrics and Gynecology. Comprehensive Healthcare Simulation. Springer, Cham. https://doi.org/10.1007/978-3-319-98995-2_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-98995-2_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-98994-5

  • Online ISBN: 978-3-319-98995-2

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics