Investigating the validity of web-enabled mechanistic case diagramming scores to assess students’ integration of foundational and clinical sciences
- 44 Downloads
As medical schools have changed their curricula to address foundational and clinical sciences in a more integrated fashion, teaching methods such as concept mapping have been incorporated in small group learning settings. Methods that can assess students’ ability to apply such integrated knowledge are not as developed, however. The purpose of this project was to assess the validity of scores on a focused version of concept maps called mechanistic case diagrams (MCDs), which are hypothesized to enhance existing tools for assessing integrated knowledge that supports clinical reasoning. The data were from the medical school graduating class of 2018 (N = 136 students). In 2014–2015 we implemented a total of 16 case diagrams in case analysis groups within the Mechanisms of Health and Disease (MOHD) strand of the pre-clinical curriculum. These cases were based on topics being taught during the lectures and small group sessions for MOHD. We created an overall score across all 16 cases for each student. We then correlated these scores with performance in the preclinical curriculum [as assessed by overall performance in MOHD integrated foundational basic science courses and overall performance in the Clinical and Professional Skills (CAPS) courses], and standardized licensing exam scores [United States Medical Licensing Exam (USMLE)] Step 1 (following core clerkships) and Step 2 Clinical Knowledge (at the beginning of the fourth year of medical school). MCD scores correlated with students’ overall basic science scores (r = .46, p = .0002) and their overall performance in Clinical and Professional Skills courses (r = .49, p < .0001). In addition, they correlated significantly with standardized exam measures, including USMLE Step 1 (r = .33, p ≤ .0001), and USMLE Step 2 CK (r = .39, p < .0001). These results provide preliminary validity evidence that MCDs may be useful in identifying students who have difficulty in integrating foundational and clinical sciences.
KeywordsBasic science education Integration Learner assessment Mechanistic case diagrams Validity
This project was funded by the Stemmler Fund of the National Board of Medical Examiners. Development of the software was funded by grants from the University of Iowa’s Innovations in Teaching with Technology Fund and from the Office of Consultation and Research in Medical Education’s Educational Development Fund. Approval for this project was obtained from the University of Iowa Carver College of Medicine Umbrella IRB on June 21, 2016 (Project ID # 201509).
Compliance with ethical standards
Conflict of interest
All authors declare that they have no conflict of interest.
- Association of American Medical Colleges. (2018). Curriculum change in US medical schools: Types of change in 2016–2017. AAMC Curriculum Inventory, 2016–2017. Retrieved August 30, 2018, from https://www.aamc.org/initiatives/cir/427196/27.html.
- Azer, S. A. (2005). Facilitation of students’ discussion in problem-based learning tutorials to create mechanisms: The use of five key questions. Annals of the Academy of Medicine, Singapore,34, 492–498.Google Scholar
- Dee, F. R. (2013). Mechanisms of disease/pathway diagramming exercises facilitate small group case analysis. Paper presented at the International Association of Medical Science Educators meeting, St. Andrews, Scotland.Google Scholar
- Dee, F. R., Wessels, S., Bolton, L., Kreiter, C. D., Schmidt, T., Rubenstein, P., & Leaven T. (2011). Improving basic medical science education through case-based pathway diagramming exercises. Presented at the Central Group for Educational Affairs of the AAMC at Omaha, Nebraska.Google Scholar
- Denton, G. D., Durning, S. J., Wimmer, A. P., Pangaro, L. N., & Hemmer, P. A. (2004). Is a faculty developed pretest equivalent to pre-third year GPA or USMLE Step 1 as a predictor of third-year Internal Medicine clerkship outcomes? Teaching and Learning in Medicine,16(4), 329–332.CrossRefGoogle Scholar
- Fischer, K., Sullivan, A., Krupat, E., & Schwartzstein R. M. (2018). Assessing the effectiveness of using mechanistic concept maps in case-based collaborative learning. Academic Medicine, September 11, 2018 - Volume Published Ahead of Print.Google Scholar
- Kane, M. T. (2006). In R. I. Brennan (Ed.), Educational measurement (4th ed., pp. 17–64). Westport, CT: Praeger.Google Scholar
- Novak, J. D., & Cañas, A. J. (2008). The theory underlying concept maps and how to construct and use them. Technical report IHMC Cmap Tools 2006-01 Rev 01-2008. Florida Institute for Human and Machine Cognition; 2008. Retrieved February 7, 2018, from http://www.webcitation.org/6Ovjy5ZZd.