Abstract
Background
The Fundamentals of Endoscopic Surgery (FES) program became required for American Board of Surgery certification as part of the Flexible Endoscopy Curriculum (FEC) for residents graduating in 2018. This study expands prior psychometric investigation of the FES skills test.
Methods
We analyzed de-identified first-attempt skills test scores and self-reported demographic characteristics of 2023 general surgery residents who were required to pass FES.
Results
The overall pass rate was 83%. “Loop Reduction” was the most difficult sub-task. Subtasks related to one another only modestly (Spearman’s ρ ranging from 0.11 to 0.42; coefficient α = .55). Both upper and lower endoscopic procedural experience had modest positive association with scores (ρ = 0.14 and 0.15) and passing. Examinees who tested on the GI Mentor Express simulator had lower total scores and a lower pass rate than those tested on the GI Mentor II (pass rates = 73% vs. 85%). Removing an Express-specific scoring rule that had been applied eliminated these differences.
Gender, glove size, and height were closely related. Women scored lower than men (408- vs. 489-point averages) and had a lower first-attempt pass rate (71% vs. 92%). Glove size correlated positively with score (ρ = 0.31) and pass rate. Finally, height correlated positively with score (r = 0.27) and pass rate. Statistically controlling for glove size and height did not eliminate gender differences, with men still having 3.2 times greater odds of passing.
Conclusions
FES skills test scores show both consistencies with the assessment’s validity argument and several remarkable findings. Subtasks reflect distinct skills, so passing standards should perhaps be set for each subtask. The Express simulator-specific scoring penalty should be removed. Differences seen by gender are concerning. We argue those differences do not reflect measurement bias, but rather highlight equity concerns in surgical technology, training, and practice.
Graphic abstract
Similar content being viewed by others
References
Hazey JW, Marks JM, Mellinger JD, Trus TL, Chand B, Delaney CP et al (2014) Why Fundamentals of Endoscopic Surgery (FES)? Surg Endosc 28(3):701–703
Hunter JE, Hunter RF (1984) Validity and utility of alternative predictors of job performance. Psychol Bull 96(1):72–98
Schmidt FL, Hunter JE (1998) The validity and utility of selection methods in personnel psychology: practical and theoretical implications of 85 years of research findings. Psychol Bull 124(2):262–274
Vassiliou MC, Dunkin BJ, Fried GM, Mellinger JD, Trus T, Kaneva P et al (2014) Fundamentals of endoscopic surgery: creation and validation of the hands-on test. Surg Endosc 28(3):704–711
Mueller CL, Kaneva P, Fried GM, Feldman LS, Vassiliou MC (2014) Colonoscopy performance correlates with scores on the FESTM manual skills test. Surg Endosc 28(11):3081–3085
Lineberry M (2019) Validity and quality. In: Yudkowsky R, Park YS, Downing SM (eds) Assessment in health professions education, 2nd edn. Routledge, New York, pp 17–32
Sackett PR, Zedeck S, Fogli L (1988) Relations between measures of typical and maximum job performance. J Appl Psychol 73(3):482–486
Stark S, Chernyshenko OS, Chan K-Y, Lee WC, Drasgow F (2001) Effects of the testing situation on item responding: cause for concern. J Appl Psychol 86(5):943–953
Lineberry M, Ritter EM (2017) Psychometric properties of the Fundamentals of Endoscopic Surgery (FES) skills examination. Surg Endosc 31(12):5219–5227
Mueller CL, Kaneva P, Fried GM, Mellinger JD, Marks JM, Dunkin BJ et al (2016) Validity evidence for a new portable, lower-cost platform for the fundamentals of endoscopic surgery skills test. Surg Endosc 30(3):1107–1112
Stark S, Chernyshenko OS, Drasgow F (2004) Examining the effects of differential item functioning and differential test functioning on selection decisions: when are statistically significant effects practically important? J Appl Psychol 89(3):497–508
Zhuang Z, Bradtmiller B (2005) Head-and-face anthropometric survey of U.S. respirator users. J Occup Environ Hyg 2(11):567–576
Bose D, Segui-Gomez M, Crandall JR (2011) Vulnerability of female drivers involved in motor vehicle crashes: an analysis of US population at risk. Am J Public Health 101(12):2368–2373
Munafo J, Diedrick M, Stoffregen TA (2017) The virtual reality head-mounted display Oculus Rift induces motion sickness and is sexist in its effects. Exp Brain Res 235(3):889–901
Criado PC (2019) Invisible women: data bias in a world designed for men. Abrams Press, New York
Shergill AK, McQuaid KR, Rempel D (2009) Ergonomics and GI endoscopy. Gastrointest Endosc 70(1):145–153
Cohen DL, Naik JR, Tamariz LJ, Madanick RD (2008) The perception of gastroenterology fellows towards the relationship between hand size and endoscopic training. Dig Dis Sci 53(7):1902–1909
Austin K, Schoenberger H, Sesto M, Gaumnitz E, Teo Broman A, Saha S (2019) Musculoskeletal injuries are commonly reported among gastroenterology trainees: results of a national survey. Dig Dis Sci 64(6):1439–1447
Shergill AK, McQuaid KR (2019) Ergonomic endoscopy: an oxymoron or realistic goal? Gastrointest Endosc 90(6):966–970
Yeo HL, Dolan PT, Mao J, Sosa JA (2020) Association of demographic and program factors with American Board of Surgery qualifying and certifying examinations pass rates. JAMA Surg 155(1):22–30
Meyerson SL, Sternbach JM, Zwischenberger JB, Bender EM (2017) The effect of gender on resident autonomy in the operating room. J Surg Educ 74(6):e111–e118
Meyerson SL, Odell DD, Zwischenberger JB, Schuller M, Williams RG, Bohnen JD et al (2019) The effect of gender on operative autonomy in general surgery residents. Surgery 166(5):738–743
Hoops H, Heston A, Dewey E, Spight D, Brasel K, Kiraly L (2019) Resident autonomy in the operating room: does gender matter? Am J Surg 217(2):301–305
Cooney CM, Aravind P, Scott Hultman C, Weber RA, Brooke S, Cooney DS et al (2019) Gender bias in plastic surgery resident assessment: a survey of 8,149 cases. Plast Reconstr Surg Glob Open 7(5S):8–9
Hyde JS (2005) The gender similarities hypothesis. Am Psychol 60(6):581–592
Hyde JS (2016) Sex and cognition: gender and cognitive functions. Curr Opin Neurobiol 38:53–56
Kass SJ, Ahlers RH, Dugger M (1998) Eliminating gender differences through practice in an applied visual spatial task. Hum Perform 11(4):337–349
Prinz A, Bolz M, Findl O (2005) Advantage of three dimensional animated teaching over traditional surgical videos for teaching ophthalmic surgery: a randomised study. Br J Ophthalmol 89(11):1495–1499
Ali A, Subhi Y, Ringsted C, Konge L (2015) Gender differences in the acquisition of surgical skills: a systematic review. Surg Endosc 29(11):3065–3073
Ritter EM, Lineberry M, Hashimoto DA, Gee D, Guzzetta AA, Scott DJ et al (2018) Simulation-based mastery learning significantly reduces gender differences on the Fundamentals of Endoscopic Surgery performance exam. Surg Endosc 32(12):5006–5011
Spencer SJ, Steele CM, Quinn DM (1999) Stereotype threat and women’s math performance. J Exp Soc Psychol 35(1):4–28
Nguyen H-HD, Ryan AM (2008) Does stereotype threat affect test performance of minorities and women? A meta-analysis of experimental evidence. J Appl Psychol 93(6):1314–1334
Theobald J, Gaglani S, Haynes MR (2015) The association between confidence and accuracy among users of a mobile web platform for medical education. Ann Intern Med 162(5):395–396
Flyckt RL, White EE, Goodman LR, Mohr C, Dutta S, Zanotti KM (2017) The use of laparoscopy simulation to explore gender differences in resident surgical confidence. Obstet Gyn Int 2017:e1945801. https://www.hindawi.com/journals/ogi/2017/1945801/ (Cited 8 June 2020)
Fonseca AL, Reddy V, Yoo PS, Gusberg RJ, Longo WE (2016) Senior surgical resident confidence in performing flexible endoscopy: what can we do differently? J Surg Educ 73(2):311–316
Cook DA, Lineberry M (2016) Consequences validity evidence: evaluating the impact of educational assessments. Acad Med 91(6):785–795
Lineberry M (2019) Assessment affecting learning. In: Yudkowsky R, Park YS, Downing SM (eds) Assessment in health professions education, 2nd edn. Routledge, New York, pp 257–271
Austin JT, Villanova P (1992) The criterion problem: 1917–1992. J Appl Psychol 77(6):836–874
Acknowledgements
We would like to gratefully acknowledge Jessica Mischna, Sarah Colon, Christelle Menetrier, and Vanessa Pryor from the SAGES Fundamentals Program for their help in providing the de-identified data.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Disclosures
Sara Hennessy, M.D. is a Member of the FES Committee in SAGES. E. Matthew Ritter, M.D. serves on the Board of Governors of SAGES and is a Member of the FES Committee. Dr. Ritter receives royalties and research support from the Henry M. Jackson Foundation for the Advancement of Military Medicine related to the development and sale of a flexible endoscopy simulator that is in no way related to FES. Matthew Lineberry Ph.D. and Yoon Soo Park Ph.D. have no conflicts of interest or financial ties to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Lineberry, M., Park, Y.S., Hennessy, S.A. et al. The Fundamentals of Endoscopic Surgery (FES) skills test: factors associated with first-attempt scores and pass rate. Surg Endosc 34, 3633–3643 (2020). https://doi.org/10.1007/s00464-020-07690-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00464-020-07690-6