Defining Success and Establishing Program Goals

The resident recruitment process is meant to select candidates who will ultimately become competent, practicing surgeons. Success can be defined as simply as that—being able to finish residency and pass boards. However, success as a surgery resident is not binary. There are multiple other factors that contribute to resident success and how we judge it, such as research, leadership, teaching, international medical work and progression to fellowship if desired. Each program must determine what attributes are most important to the program in order to establish and make known the expectations for its residents.

At the most basic level, a successful resident is one who completes residency. The attrition rate for general surgery residency hovers around 20%. Though there are various reasons for leaving a program, the majority leave for lifestyle reasons [1, 2]. One explanation is that residents have inadequate exposure to surgery as students. In a survey evaluation of general surgery interns entering training in the 2007–2008 academic year, Abelson et al. concluded “categorical general surgery interns with more realistic expectations of surgery residency and life as an attending surgeon are more likely to complete training” [3]. This does not necessarily mean longer time spent on surgery rotations, but the quality of the experience [4]. A different explanation is the “surgeon personality.” Students pursuing surgery and surgery residents consistently score higher on conscientiousness and extraversion. This has been demonstrated in multiple studies using a variety of personality tests [5,6,7]. Students pursuing surgery differ somewhat from residents in regards to openness to ideas, indicating some assimilation to peers during the training process. Using a different assessment, the World of Work Inventory Online, faculty and high performing residents had similar profiles, in contrast to low performing residents [8]. Burkhart et al. suggested grit, defined as “perseverance and passion for long-term goals,” could be used as a surrogate for risk of attrition by screening applicants with an eight question survey [9]. This was found to remain significant when used to assess residents [10]. Furthermore, in a single institution study evaluating grit scores and the big five personality traits together compared to information available in the resident application suggested that non-low performing residents “possess unique, measurable personality traits and that the commonly used evaluation methodology does not identify them” [11]. It appears that using personality assessments to potentially stratify residents at risk for attrition may be a beneficial tool.

Being able to pass the boards is the next basic tenet of surgery resident success. Information in surgery candidate applications has been evaluated to determine if there is a correlation with resident performance on further standardized testing, namely American Board of Surgery (ABS) In-Training Examination (ABSITE) score and the ABS qualifying and certifying examinations. In summary, these studies compared at least some of the following components: United States Medical Licensing Examination (USMLE) step I or II scores, ABSITE scores, passing the ABS qualifying and certifying exam on the first attempt, and demographics. The conclusions were that there is significant correlation with performance on the ABSITE and passing general surgery boards [12, 13]. Additionally, there is a positive association between USMLE step II and to a lesser extent step I, to ABSITE performance [12,13,14,15,16]. Therefore, it is reasonable to use the USMLE step scores to screen for residents who may have difficulty with passing the ABS qualifying or certifying exam. This is important to program directors as the Accreditation Council for Graduate Medical Education (ACGME) mandates minimum passing requirements to maintain program certification [17]. However, none of these tests were designed to be utilized for this purpose. For that reason and multiple others, the USMLE step I score will soon be reported as pass/fail. This will make the screening process of resident selection more challenging, as stated eloquently by Dr. Rozenshtein on behalf of the Association of Program Directors in Radiology in her recent paper [18]. Gardner et al. suggested using an alternative assessment. In a multi-institutional study, a questionnaire was sent to candidates electronically prior to offering an interview. The tool took an average of 35 min to complete and was comprised of a situational judgement test and questions about “perceived ideal training program attributes.” Though the effectiveness of the test for programs or candidates was not evaluated, 97% of applicants completed the test, suggesting additional testing is a reasonable screening option [19].

In addition to objective measurements, clinical performance is another aspect of resident success, and possibly the most challenging to predict when assessing candidates [20]. In an 18 year review of residents’ candidate information, ABSITE scores and clinical performance at a university program, Alterman et al. found that the interview score, USMLE step I score and reporting being a high achiever in athletics or art predicted academic and clinical success [21]. An older study was unable to find any specific predictors of success in admission folders, but did note that participation in team sports was a characteristic attributable only to highest resident performers [22]. Tolan et al. found overall competence was related to female gender, AOA status and number of honors received in medical school, while USMLE scores were predictive only of medical knowledge [23]. The interview process allows for the evaluation of subjective or non-cognitive applicant characteristics. In their literature review evaluating the residency interview, Stephenson-Famy et al. remarked “ideally, an applicant should be a good “fit” for the program, with a high likelihood of success and a low likelihood of problems.” Additionally, they laid out interview qualities to improve reliability, including clear expectations of the resident, standardized interview questions and limiting single interviewers [24]. However, it is known that interviews can be biased and are inherently subjective [24,25,26]. When individuals, often program directors or department chairs, make adjustments to the rank list, which is created based on an objectified scoring, it may jeopardize the selection of successful residents [27].

There have been a few novel strategies to attempt to improve successful resident recruitment. In 2012, Farkas et al. had evaluated using a surgery-specific written exam on the day of interviews. As expected, there was significant correlation with candidates ABSITE scores as they progressed with residency, with higher correlation than either USMLE test [28]. However, the test was administered on the interview day, so it was still not used as a screening tool but was available when determining the rank list. At the University of Pennsylvania, a novel selection strategy was created that utilized standardized screening, a preinterview essay, decreased faculty participants with scripted interviews, and minimized adjusting of the rank list after committee agreement. The result was decreased resident attrition; the authors concluded this was due to improved identification of “candidates most likely to succeed” [29]. Other possible processes that may lead to selecting successful residents include incorporating a skills test into the process in addition to other preinterview assessments [20].

Overall, it is essential that a program prioritize the importance of different candidate attributes. Though there is some conformation in the top few characteristics [30], “matching applicant objectives with program opportunities is of critical importance to many of the challenges faced by general surgery residency programs” [31]. For example, programs with nearly 100% fellowship participation after residency would not likely benefit by targeting the same demographic of applicants as a program with a history of graduating residents bound to practice in rural areas. Hence programs should know and attempt to match the program’s strengths to applicant career goals. That being said, “…while it is essential to recruit residents who are likely to meet existing standards, it is even better to identify and recruit those who will exceed these standards as they will benefit the program in many ways… and this may perpetuate the recruitment of high quality residents year after year” [20]

Candidates’ Priorities

In order to attract successful residents, programs must also understand the needs of the applicants. In an attempt to elucidate these priorities, there have been multiple studies utilizing surveys of fourth year medical students during the application process. Phitayakorn et al. evaluated surgery applicants to competitive academic centers over nearly a decade and found that the most important factors were the program’s ability to prepare residents for future training or position, resident esprit de corps, faculty availability and involvement in teaching, depth and breadth of faculty and variety of patients and clinical resources. Lifestyle factors, such as child care or call rooms, were ranked least important [32]. In another survey-based study comparing students pursuing surgery and current surgical junior residents in both the USA and Japan, there were similar findings—educational factors were the most important in selecting a particular program. These authors found the highest rated factors were educational program, clinical experience, faculty quality and in some subgroups, perception of resident happiness [33]. These more recent studies correlate with prior studies which had found resident satisfaction, diversity or volume of training experience and geography had been important factors prior to the ACGME duty hour requirement implementation [34]. Contrarily, there are studies that surveyed different populations of surgery applicants that found resident happiness/quality of life or collegiality to be more important to respondents than educational objectives [35,36,37]. This emphasizes understanding the strengths of a specific residency program and targeting candidates that “fit.”

The interview process was upturned during the last cycle due to the COVID-19 pandemic, and the lasting ramifications are yet to be appreciated. However, one thing is clear—social media (SM) and online presence has a new status in the process. The importance of program websites for applicants has been confirmed across specialties in prior literature. In fact, many residents use the website information not only to determine where to apply, but also in their interview and ranking decisions [38, 39]. Nonetheless, in studies that evaluated general surgery or specialty program websites, the majority were found to be incomplete or outdated, have non-functional links and be overall inadequate [31, 39, 40]. In a review of programs’ website specifically looking at elements of diversity, 89% missed the mark for inclusivity, with university programs faring better than private institutions [41]. Though program websites remain the most utilized electronic resource, more recently, SM influence appears to have increased. McHugh et al. published about the role of Facebook in 2014, with nearly half of the anesthesia residency candidates for their institution using it during the interview process [42]. Due to the creation of newer social media platforms and the restrictions enacted due to the COVID19 pandemic, this was a hot topic at the 2021 Association of Program Directors in Surgery (APDS) meeting. An abstract presented by Bludevich from the University of Massachusetts evaluated the increase in social media accounts and posts by surgery departments and surgery residency programs. An extensive amount of literature on the use and impact of SM is expected to materialize over the next year.

The importance of SM has already been evaluated in plastic surgery programs. In one study from 2021 that surveyed plastic surgery resident applicants, 60% of responders reported using social media. Though 72% of applicants said a program’s poor SM presence did not make them lose interest in a program, for those candidates who did lose interest based on SM, the factors that were reported were infrequent posts, outdated content, and lack of engagement [43]. In 2020, Azoury et al. found that Facebook is still the most popular platform, followed by Instagram and Twitter. The use of these platforms varies from personal to professional. However, “training program social media use is rising, with Instagram and Twitter presence growing at exponential rates” [44]. In a study evaluating plastic surgery applicants to Harvard, the authors found that 97% of applicants used SM for information gathering, but only twenty percent of respondents’ program rank list was influenced by a program’s SM. Additionally, the majority of respondents trusted the SM information as accurate only most of the time [39]. What content applicants find most important on SM is still controversial, with some studies finding respondents looking for resident life information and others seeking posts about educational or clinical information [39, 40, 44].

Traditionally, most programs hold a preinterview dinner followed by interviews the following day. In a survey of a single institution’s fourth year medical students, Schlitzkus et al. found that 86% of applicants found the preinterview session informative about the program [45]. This is likely due to being able to appreciate the subjective factors that applicants consider important when selecting a program, such as resident life, “fit,” and environment [35,36,37, 45]. As the pandemic halted in-person interviews, programs adapted in various virtual ways—hosting open house sessions, preinterview group interactions and interviews. Literature is just now emerging evaluating the financial differences to applicants and institutions. We have yet to see which of these practices will continue or be abandoned as we return to tried and tested processes. The programmatic implications of both virtual interviews and distanced medical school learning for nearly a year will likely not be appreciated for years, though altered attrition rate, specialty preference, and lack of comfort with procedures due to decreased exposure are possible consequences [46].

Conclusion

Beyond adequate acquisition of knowledge, the term “successful resident” has different implications based on residency program initiatives. There is ample literature about different aspects of the process, but onus is on the program to determine its assets, expectations and ambitions. Recognizing and highlighting individual program strengths will then allow applicants to match their needs to the program in more synergistic ways. It is necessary to make this information easily available to residency candidates. Currently, websites are the most utilized resource, but SM has increasing importance. Candidate assessment in addition to traditional application information may aid in adapting the screening and interview process. Undoubtedly, the COVID-19 pandemic has increased the complexity of the resident selection process in ways not yet completely delineated.