Advertisement

Medical Science Educator

, Volume 29, Issue 1, pp 291–298 | Cite as

Developing Comprehensive Strategies to Evaluate Medical School Curricula

  • Sally A. SantenEmail author
  • Moshe Feldman
  • Sara Weir
  • Courtney Blondino
  • Meagan Rawls
  • Susan DiGiovanni
Monograph

Abstract

Evaluation of medical school curriculum is important to document outcomes, effectiveness of learning, engagement in quality improvement, and to meet accreditation compliance. This monograph provides a roadmap and resource for medical schools to meaningfully evaluate their curriculum based on specific metrics. The method of evaluation includes an examination of Kirkpatrick’s levels of outcomes including reactions, learning, behavior, and impact. It is important that student outcomes are mapped in relation to curricular objectives. There are specific outcomes that may be utilized to determine if the curriculum has met the institution’s goals. The first is comparison to national metrics (United States Medical Licensing Examinations and American Association of Medical Colleges Graduation Questionnaire). Second, medical schools collect internal program metrics, which include specific student performance metrics, such as number of students graduating, attrition, and matching to specialty. Further, schools may examine student performance and surveys in the preclerkship and clinical phases (e.g., grades, failing courses, survey responses about the curriculum), including qualitative responses on surveys or focus groups. As the learning environment is critical to learning, a deep dive to understand the environment and mistreatment may be important for program evaluation. This may be performed by specifically examining the Graduation Questionnaire, internal surveys, and mistreatment reporting. Finally, there are numerous attitudinal instruments that may help medical schools understand their students’ development at one point or over time. These include measurements of stress, wellness, burnout, lifelong learning, and attitudes toward patient safety. Together, examining the composite of outcomes helps to understand and improve the medical school curriculum.

Keywords

Medical students Evaluation Assessment 

Notes

Funding Information

Dr. Santen receives funding for the evaluation work related to the Accelerating Change in Medical Education Grant from the American Medical Association.

Compliance with Ethical Standards

Conflict of Interest

The author declares that they have no conflict of interest.

References

  1. 1.
    Stoddard HA, Brownfield ED, Churchward G, Eley JW. Interweaving curriculum committees: a new structure to facilitate oversight and sustain innovation. Acad Med. 2016;91(1):48–53.  https://doi.org/10.1097/ACM.0000000000000852.Google Scholar
  2. 2.
    Standards, Publications, & Notification Forms | LCME. http://lcme.org/publications/#Standards. Accessed April 2, 2018.
  3. 3.
    Moore DEJ, Green JS, Gallis HA. Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. J Contin Educ Health Prof Winter. 2009;29(1):1–15.  https://doi.org/10.1002/chp.20001.Google Scholar
  4. 4.
    Kirkpatrick DL, Kirkpatrick JD. Evaluating training programs: the four levels. 3rd ed. San Francisco: Berrett-Koehler; 2006.Google Scholar
  5. 5.
    Yardley S, Dornan T. Kirkpatrick’s levels and education ‘evidence’: Kirkpatrick’s levels in education. Med Educ. 2012;46(1):97–106.  https://doi.org/10.1111/j.1365-2923.2011.04076.x.Google Scholar
  6. 6.
    Bowe CM, Armstrong E. Assessment for systems learning: a holistic assessment framework to support decision making across the medical education continuum. Acad Med. 2017;92(5):585–92.  https://doi.org/10.1097/ACM.0000000000001321.Google Scholar
  7. 7.
    Hofmann DA, Jacobs R, Baratta JE. Dynamic criteria and the measurement of change. J Appl Psychol. 1993;78(2):194–204.  https://doi.org/10.1037/0021-9010.78.2.194.Google Scholar
  8. 8.
    McCoubrie P. Metrics in medical education. Ulster Med J. 2010;79(2):52–6.Google Scholar
  9. 9.
    Graduation Questionnaire (GQ) - data and analysis - AAMC. https://www.aamc.org/data/gq/. Accessed April 2, 2018.
  10. 10.
    NCIEC_focus_group_manual_07–20112.pdf. http://www.interpretereducation.org/wp-content/uploads/2015/02/NCIEC_focus_group_manual_07-20112.pdf. Accessed April 20, 2018.
  11. 11.
    Mavis B, Sousa A, Lipscomb W, Rappley MD. Learning about medical student mistreatment from responses to the medical school graduation questionnaire. Acad Med. 2014;89(5):705–11.  https://doi.org/10.1097/ACM.0000000000000199.Google Scholar
  12. 12.
    Marshall RE. Measuring the medical school learning environment. J Med Educ. 1978;53(2):98–104.Google Scholar
  13. 13.
    Rosenbaum ME, Schwabbauer M, Kreiter C, Ferguson KJ. Medical students’ perceptions of emerging learning communities at one medical school. Acad Med. 2007;82(5):508–15.  https://doi.org/10.1097/ACM.0b013e31803eae29.Google Scholar
  14. 14.
  15. 15.
    Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983;24(4):385–96.  https://doi.org/10.2307/2136404.Google Scholar
  16. 16.
    Gross JJ, John OP. Individual differences in two emotion regulation processes: implications for affect, relationships, and well-being. J Pers Soc Psychol. 2003;85(2):348–62.  https://doi.org/10.1037/0022-3514.85.2.348.Google Scholar
  17. 17.
    Dyrbye LN, Szydlo DW, Downing SM, Sloan JA, Shanafelt TD. Development and preliminary psychometric properties of a well-being index for medical students. BMC Med Educ 2010;10:8.  https://doi.org/10.1186/1472-6920-10-8.
  18. 18.
    Maslach C, Jackson SE. The measurement of experienced burnout. J Occup Behav. 1981;2(2):99–113.Google Scholar
  19. 19.
    Smith BW, Dalen J, Wiggins K, Tooley E, Christopher P, Bernard J. The brief resilience scale: assessing the ability to bounce back. Int J Behav Med. 2008;15(3):194–200.Google Scholar
  20. 20.
    Connor KM, Davidson JRT. Development of a new resilience scale: the Connor-Davidson Resilience Scale (CD-RISC). Depress Anxiety 1091–4269. 2003;18(2):76–82.Google Scholar
  21. 21.
    Carruthers S, Lawton R, Sandars J, Howe A, Perry M. Attitudes to patient safety amongst medical students and tutors: developing a reliable and valid measure. Med Teach. 2009;31(8):e370–6.  https://doi.org/10.1080/01421590802650142.Google Scholar
  22. 22.
    Wetzel AP, Dow AW, Mazmanian PE. Patient safety attitudes and behaviors of graduating medical students. Eval Health Prof. 2012;35(2):221–38.  https://doi.org/10.1177/0163278711414560.Google Scholar
  23. 23.
    Hojat M, Veloski JJM, Gonnella JS. Measurement and correlates of physicians’ lifelong learning. Acad Med. 2009;84(8):1066–74.  https://doi.org/10.1097/ACM.0b013e3181acf25f.Google Scholar
  24. 24.
    Wetzel AP, Mazmanian PE, Hojat M, Kreutzer KO, Carrico RJ, Carr C, et al. Measuring medical students’ orientation toward lifelong learning: a psychometric evaluation. Acad Med. 2010;85(10):S41–4.  https://doi.org/10.1097/ACM.0b013e3181ed1ae9.Google Scholar
  25. 25.
    Hojat M, Mangione S, Nasca TJ, Cohen MJM, Gonnella JS, Erdmann JB, et al. The Jefferson Scale of Physician Empathy: development and preliminary psychometric data. Educ Psychol Meas. 2001;61(2):349–65.  https://doi.org/10.1177/00131640121971158.Google Scholar
  26. 26.
    Geller G, Tambor ES, Chase GA, Holtzman NA. Measuring physicians’ tolerance for ambiguity and its relationship to their reported practices regarding genetic testing. Med Care. 1993;31(11):989–1001.Google Scholar
  27. 27.
    McCrae RR, Costa PT. A contemplated revision of the NEO Five-Factor Inventory. Personal Individ Differ. 2004;36(3):587–96.  https://doi.org/10.1016/S0191-8869(03)00118-1.Google Scholar
  28. 28.
    Dyrbye LN, Massie FS, Eacker A, Harper W, Power D, Durning SJ, et al. Relationship between burnout and professional conduct and attitudes among US medical students. JAMA. 2010;304(11):1173–80.  https://doi.org/10.1001/jama.2010.1318.Google Scholar
  29. 29.
    Topp CW, Østergaard SD, Søndergaard S, Bech P. The WHO-5 Well-Being Index: a systematic review of the literature. Psychother Psychosom. 2015;84(3):167–76.  https://doi.org/10.1159/000376585.Google Scholar
  30. 30.
    Dyrbye LNM, Schwartz A, Downing SM, Szydlo DW, Sloan JA, Shanafelt TD. Efficacy of a brief screening tool to identify medical students in distress. Acad Med. 2011;86(7):907–14.  https://doi.org/10.1097/ACM.0b013e31821da615.Google Scholar
  31. 31.
    Spitzer RL, Williams JB, Kroenke K, Linzer M, de Gruy FV 3rd, Hahn SR, et al. Utility of a new procedure for diagnosing mental disorders in primary care. The PRIME-MD 1000 study. JAMA. 1994;272(22):1749–56.Google Scholar
  32. 32.
    Ware JE, GlaxoSmithKline. How to score and interpret single-item health status measures: a manual for users of the of the SF-8 health survey : (with a supplement on the SF-6 health survey). Lincoln, RI; Boston, MA: QualityMetric, Inc. ; Health Assessment Lab; 2001.Google Scholar
  33. 33.
    Johns MW. A new method for measuring daytime sleepiness: the Epworth sleepiness scale. Sleep. 1991;14(6):540–5.  https://doi.org/10.1093/sleep/14.6.540.Google Scholar
  34. 34.
    Nolan RL, Haeckel SH. The role of technology in an information age: transforming symbols into action. 1994. https://www.hbs.edu/faculty/Pages/item.aspx?num=1203. Accessed April 20, 2018.
  35. 35.
    Casebeer A. Application of SWOT analysis. Brit J Hosp Med. 1993;49(6):430–1.Google Scholar

Copyright information

© International Association of Medical Science Educators 2018

Authors and Affiliations

  1. 1.Office of Assessment, Evaluation and ScholarshipVirginia Commonwealth School of MedicineRichmondUSA
  2. 2.Department of Emergency MedicineVirginia Commonwealth School of MedicineRichmondUSA
  3. 3.Office of Medical Student EducationUniversity of Michigan Medical SchoolAnn ArborUSA
  4. 4.Division of Epidemiology, Department of Family Medicine and Population HealthVirginia Commonwealth School of MedicineRichmondUSA
  5. 5.Medical Student Education & Student AffairsVirginia Commonwealth School of MedicineRichmondUSA

Personalised recommendations