Non-technical skills: a review of training and evaluation in urology



With non-technical skills (NTS) deficits being recognised as a major cause for error in surgery, there is an increasing interest in their training and evaluation. A growing number of training courses are emerging and some NTS curricula have also been created. Many different training methods are described in the literature but there is still uncertainty with regards to their optimum combination within a curriculum.


A literature review of the electronic database Medline was performed. All articles published before December 2018 were screened by abstract and included if deemed relevant by the author. The included articles’ reference lists were also screened for further relevant studies.


Simulation training is accepted as the most effective way to train NTS. Within simulation training, it is shown that the ‘igloo’ full immersion/distributed simulation environment is appropriate for teaching NTS in urological scenarios where a designated operating room or space is not available. The use of multiple settings, for example wards and clinics as well as the operating room, is advantageous, as is training in an interprofessional team. Classroom teaching also plays a role in NTS training as an adjunct to simulation, with evidence that it improves some parameters of NTS. All levels, including qualified surgeons, benefit from NTS training; however, adaptation to both trainee level and specialty is important. Although less time consuming, training juniors and seniors together mainly benefits juniors, and training NTS at the same time as technical skills detracts from the quality of teaching. Debriefing is an important part of training and should be well structured; there are many debriefing models in existence, allowing for choice of method based on examiner preference and participant demographic. Furthermore, examiners should be well briefed in their task and trained in NTS assessment.


To move forward, studies should combine tried and tested learning techniques into a curriculum covering all training levels, which should then be validated and followed up long term to ensure a positive impact on patient safety.


Non-technical skills (NTS) are the cognitive and social abilities that complement a clinicians’ technical ability and generally comprise decision making, leadership, team work, and situational awareness (Table 1) [1, 2]. They are often grouped into three categories: social skills (leadership, communication and teamwork), cognitive skills (decision making and situational awareness), and personal resource factors. The latter includes how individuals cope with stressors and fatigue, known to negatively impact NTS and technical performance (2].

Table 1 Non-technical skills (NTS)

In the last decade, NTS have emerged as a vital area for improvement within healthcare: they are a major cause for error, causing 86% of adverse events in open surgery [3, 4]. A UK-based evaluation of skills survey found that only 41% of urological trainees felt that their NTS training was sufficient for their first day of practice compared to 78% of specialists, with an even lower 25% of trainees believing their current NTS training was sufficient [5].

Clearly, NTS training must improve further and there is a need for standardised and validated training curricula, especially in newer fields such as robotic surgery [1, 3, 5]. This review aims to collate the current evidence base on NTS training, assessment, curricula and debriefing methods to aid and recommend current practice and future developments in urological training.


A literature review of the electronic database Medline was performed. All articles published before December 2018 were screened by abstract and included if deemed relevant by the author. The included articles’ reference lists were also screened for further relevant studies.

Search terms included: non-technical skills AND training AND urology AND English[lang]; non-technical skills OR NTS AND simulation AND English[lang]; Debrief AND Simulation AND English[lang]. This was to find both articles specifically related to current NTS training in urology as well as further articles from other disciplines from which urological training can learn and improve.

Non-technical skills training

Cognitive training

Cognitive training is an established training method in areas such as aviation, the military, and sports [6]. Studies have shown that mental- or motor imagery (MI) can lead to lower stress levels and better error detection rates with technical skills (TS) training [6,7,8]. MI is a low-cost approach which involves mental rehearsal of a task with the aid of a script. A number of studies have suggested that MI has a positive effect on NTS, but no concrete benefit has been observed in the literature [6, 9].

Classroom/didactic teaching

Classroom teaching is useful for introducing NTS concepts, self-reflection and changing the attitudes of trainees [10]. The effect of a single didactic NTS seminar on a group of medical students was investigated with an examined scenario and compared to a control group. The NTS group significantly improved their teamwork and situational awareness, as well as “handling errors”. However, both groups saw a reduction in stress and better decision making. Medical endpoints and patient outcome did not differ significantly between groups [11]. Thus, although didactic teaching has a role in NTS training, exemplified by this study, difficult cognitive skills such as decision making and planning require further training using other methods [11, 12]. Other beneficial classroom-based techniques include observing famous incidents and video analysis of self-performance [12].


Simulation is the most effective way to train NTS [10, 13]. It is critical for teaching communication skills and shows superiority to didactic lectures for self-perceived improvement [14]. Ounounou et al. [13] divide simulation-based NTS training into three main categories: full immersion/distributed simulation (FIDS), high fidelity OR simulation (HFORS) and crisis resource management (CRM).

Full immersion/distributed simulation

FIDS consists of a 360° inflatable and mobile “igloo” shell filled with operative equipment to create a realistic environment [15]. Brewin et al. [16] investigated the combined teaching of TS (TURP) and NTS within this environment and demonstrated face, content and construct validity. The course received positive qualitative feedback and the NTS of the experienced urologists were significantly better than those of trainees, upon a NoTECHS scale (see Assessment methods), but did not demonstrate transfer of skills to OR. The results of this study were further supported by Brunckhorst et al. [17], where novices who were taught ureteroscopy and NTS within an FIDS environment outperformed those taught using didactic methods, with the former group gaining significantly higher NoTSS scores. The “igloo” is, therefore, a viable alternative for NTS courses where an OR is not available.

High fidelity simulation

HFORS uses a simulated OR for training to assess non-technical surgical skills. Abdelshehid et al. [18] conducted a prospective cohort study in which urology residents undertook a laparoscopic partial nephrectomy simulation-based team-training scenario, using validated simulator models. They found that the level of urology training did significantly affect non-technical performance using the NoTSS score, thus showing construct validity. Lee et al. [19] conducted a study similar but alongside anaesthetic trainees. 94% of participating trainees thought that the session was useful and should be included in training, demonstrating face validity. Although HFORS proves to be beneficial, long-term follow-up showing transfer validity is consistently lacking in the literature.

Crisis resource management and ward-based scenarios

Training must address the surgeon’s wider role, including outside the OR, in clinics and wards, since different skills are required for different settings [20]. Many studies exclusively use the OR setting for simulation but there are some examples of simulated ward rounds, such as the UK-based urology boot camp [3], an intense 1-week training curriculum and course for newly starting urology trainees. Simulated wards and clinics have also been employed as part of a larger programme in the urology-focussed course by Khan et al. [21], using the ‘SimMan’ model, and for general NTS training with multiprofessional groups. These courses were all well received [21, 22]. Future courses could stand to gain from incorporating these ideas into a mixed curriculum comprising both OR and ward scenarios.

CRM training, which addresses NTS in emergency scenarios, has been used successfully in aviation and acute care and is associated with a reduction in errors. Truta et al. [23] created a single-day CRM course with both didactic and simulation training for interprofessional EM teams. It resulted in significant NTS improvement two months following the course in all aspects of NTS for all levels of training. Yee et al. [24] also evaluated one similar CRM intervention and showed improvement at 1 and 2 months in all NTS categories. However, other studies do not show this retention [23]. CRM training is incorporated into a wider urology curriculum by Khan et al. [21], which shows feasibility and acceptability. However, this study focussed on technical performance and lacks quantitative data on NTS. Although CRM may be better suited to emergency teams, it should be considered for urological training pending further studies.

Team training

Team training rather than individual training has been found useful by trainees, particularly for communication skills [25]. Interprofessional team working needs to be addressed to improve patient safety, involving the range of professions that compose an OR team benefits simulations by improving communication and decreasing anxiety between groups [22, 26, 27]. This effect is most relevant to junior trainees, but seniors should also be aware of the issues faced by the less senior staff in their teams [22].

Training for robotic surgery

Robotic surgery is a multidisciplinary, complex environment and, thus, requires further development of NTS alongside TS training [1, 4]. New challenges include the new technology, the surgeon being physically displaced from their team, a change in the surgeons’ role and additional staff members to communicate with. The team must adapt their interactions and maintain excellent situational awareness and communication, for example staying aware of patient status and any equipment failure [1, 4, 12, 28]. Other NTS areas to consider in robotic training include cognitive skills: technically demanding robotic procedures require a high level of decision making and planning. The assistant surgeon also has a higher responsibility as they are at the patients’ side without the lead surgeon [1, 12, 28]. VR simulation can be used for robotic NTS training to further skills [1]. Individual and team reactions to system errors can be simulated, repeated and assessed. The latest generation trainers (dV-Trainer™ and Robotix mentor™) have an option to train the assistant alongside the surgeon, which allows for the development and assessment of teamwork in hazard scenarios and troubleshooting skills [4]. Considering current literature, it is recommended that NTS and team training be learnt in simulation training, with or without VR aid, that can replicate common and emergency scenarios in robotic surgery. This should support structured assessments within the robotic surgical curricula [1].


The structured integration of NTS training into the curriculum is important to improve delivery in medical education, ensuring competence and improved patient outcomes [20]. However, this faces many issues, for example the working hour restrictions, limited personnel and considerable costs [29]. HFORS is particularly costly and can be restricted to bigger centres with better resources [30, 31]. To combat this, innovations such as the FIDS “igloo” and a mobile education unit (MEU) have been created to transport simulation to new or more rural places [30] and standardise teaching using the same mobile setting. The ‘igloo’ has been shown to have face, content and construct validity and is recommended for NTS training. The MEU was also well received by clinicians [5, 10, 13, 30]. It is also disputed whether higher fidelity, costly models result in greatly improved learning outcomes: one study shows no significant difference between the groups who trained with a low fidelity model (costing €14) compared to a high-fidelity model (costing €2600) [25, 32]. Studies have also been run in the nursing profession and although higher fidelity models gave a benefit with TS training, the evidence for NTS is less robust and lacks any long-term significance [33].

In the Dutch urological practical skill curriculum, theoretically optimum methods for learning were not always logistically viable ([29]. For example, frequent short training sessions (e.g. 1 h per week) are known to be better than longer, less frequent sessions (e.g. one afternoon per month), but are more difficult to organise. Similarly, consolidation of skills cannot exclusively be achieved in short courses, but they are much easier to organise [29, 34]. Non-compliance and under prepared supervisors are also a problem, especially when they may not have clear instructions or emphasis on the importance of preparation [29]. On this basis, significant planning, well in advance, is necessary if a curriculum is to be successful.

Retention of knowledge

Transfer validity and skill retention are particularly difficult to investigate since following up participants is logistically challenging, resulting in many studies lacking these important data. Studies which do follow up participants show NTS retention for at least 2 months in current training programmes, in which some claiming trainees are still benefitting at 6 months, but others note that there is no significant difference between the NoTSS scores of surgeons who have previously undertaken NTS training versus those who have not. These studies often employ a second simulation session to record improvement and do not look at real performance in the OR [23, 26, 35]. The longer-term effects of any courses are yet to be shown, but repeat training is necessary to maintain skills after they are learnt initially and make sure they translate to practice [35]. This poses two unanswered questions: how frequent should NTS training be and how should ‘refresher’ courses be structured [23, 26]?

Further considerations

All training methods could benefit from using real urological case discussions (e.g. previous difficult cases from experience) to explore the decision processes and learn from mistakes and successes [20, 30]. Complex unexpected patient death scenarios have been successfully tested in high-fidelity simulation, and case studies have been employed in other studies such as the S-TEAMS course [20, 26]. NTS can also be consolidated in the workplace using informal methods and by debriefing thoroughly after critical incidents: it is shown that debriefing in the OR benefits NTS development. This could stand in place of or aid refresher courses to make sure the skills learnt continue to be employed in practice [12].

Furthermore, during career transitions doctors gain greater decision-making autonomy, making it a time of increased stress resulting in higher patient mortality, evidenced by the transitioning handover periods of doctors [13, 36]. Junior staff perform poorly when they have higher cognitive workloads; therefore, training of NTS at junior level is important to reduce errors in these times of high stress [13, 20].

Evaluation and assessment

Most rating scales for NTS have been thoroughly validated for use in surgical simulation [10]. The most popular and comprehensively validated are NoTSS (Non-Technical Skills for Surgeons), NOTECHS (NOn-TECHnical Skills) and OTAS (Observational Teamwork Assessment for Surgery) [10, 37]. For specialties such as urology, more specialised assessment scales can be necessary: so far the NoTSUS (Non-technical Skills for Urological Surgeons) scale has been developed, as well as the ICARS for robotic surgery. These are both based on NoTSS and have been validated for use in training [1, 28].

Wood et al. [38] evaluated NTS training tools for both individual surgeons and teams and concluded that NoTSS was the best scale when evaluating an individual, but the NOTECHS was optimum for assessing teams. This highlights the need to remember the context of the simulation when deciding on an assessment tool [4, 38]. Tools to assess individual NTS components were also discussed in the review, such as the ‘RATE’ tool for situational awareness training; these tools are useful if there is a need to train only one aspect of NTS, possibly useful for specialties such as robotic surgery due to their specific challenges [38].

To aid assessment, a ‘talk-aloud’ protocol can be adopted where participants narrate their considerations and decisions throughout the simulation [39]. Assessors are also able to take NoTSS as an independent course for training in scoring participants [35]. Both these elements can make assessment fairer and help standardise it.


It is widely accepted that the main opportunity for long-lasting and deep learning of NTS takes place during debriefing after simulation [40]. A structured debriefing session with a skilled facilitator is thought to be vital for the acquisition of NTS, as it encourages self-reflection [10, 35]. Debriefing after real events is also beneficial but should be structured differently [41]. The specific elements of debriefing required for improvement in NTS are not clearly defined in healthcare, and the specific skills needed for effective debriefing after simulation for NTS training can be unclear when both TS and NTS training are reviewed together. Most current studies on debriefing are limited in application due to bias and lack of generalisability; however, advice on a more systematic and standardised approach to the debrief session is needed to ensure all participants in NTS courses benefit consistently from training [40, 42].

There are three distinct debriefing categories (Table 2): facilitator-guided post-event debriefing, self-guided post-event debriefing, and facilitator-guided within-event debriefing, e.g. freeze frames [43]. Generally, debriefing should be as long as the simulation itself, and there should be a briefing to let participants know what to expect [40]. Frameworks for debriefing conversations can be found summarised in the review by Sawyer et al. [43] but more research is needed to determine any concrete benefit from using one model over another: it is likely that any model can be effective, with the act of debriefing being the important part. It is also necessary to account for the simulations’ context and what the debriefer feels most confident with when deciding on the model [43].

Table 2 Debriefing styles

Having a script or aid can help the debriefer to deliver a higher quality session by improving team leader performance; in a study by Jaye et al., a standardised structure (‘The Diamond’) was put forward to refocus debriefing on NTS rather than TS, as well as make sure the experiences between participants are equal [40, 44]. ‘The Diamond’ is a two-sided prompt sheet: the first side contains the scaffolding, with a series of specifically constructed questions for each phase of the debrief; the second lays out the theory behind the questions and the process. With this the variation in what is expected from debriefing can be addressed and standardised [44]. Assessment scales like NoTSS are also useful aids, either as an adjunct to discussion or self-reflection, giving the participant objective feedback to reflect on [38].

Debriefing styles and techniques

It is thought that having a skilled debriefer is important to the concept, but recent studies show that this might not be the case, provided that there is a form of educational process post-simulation [40]. Self-debriefing (e.g. reviewing a video of oneself with an assessment scale like NoTSS) has been shown to be effective in learning CRM, and it was found to be similarly effective to traditional instructor debriefing in other simulations [31]. This is important, as it can be expensive to train and hire senior individuals to these roles which impact the feasibility of simulation sessions and curricula [40]. Self-assessment has also been shown to create more goals for learning, which are then more likely to be carried out by the individual [45]. Self-debriefing is more cost effective than instructor debriefing, however, with a higher budget, instructor debriefing is preferred [31]. For more experienced learners, there is less reliance on an instructor and self-assessment is more cost effective, as well as allowing participants to control the pace of debriefing and the opportunity for review of self-perceived weaknesses [43, 46].

Often video recording is used in debriefing; however, recent studies have failed to find any additional benefit from using this technique [40, 47]. It may even distract participants from focusing on the learning objectives, but current research is not robust enough to discount it [46]. Eye tracking technology is an innovative method to provide participants with feedback. It significantly improves certain practices compared to verbal feedback, but other behaviours, such as decision making, are unaffected [48]. Alternatives to traditional instructor debriefing include use of self-video review, multimedia debriefing, and within-team debriefing [31]. Re-do stations have also been found useful by students as learning and debriefing experience, and those who had a staff member present found it more useful [49].

Freeze frames are a form of facilitator-guided within-event debriefing [13]. This is described as ‘stop action’ debriefing—participants can be stopped when an error occurs and receive corrective feedback before trying again. This promotes deliberate practice. However, within-event feedback alone has been found to be inferior to post-event feedback; thus, post-event debriefing should always accompany this method [43].

A final consideration is whether debriefing should be just for the individual or within their team. In a study by Martin et al. [30], half of each group were active participants in a case while the other half observed, and both participants and observers were invited to take part in the debriefing sessions. Participants found this to be a safe and positive educational experience [30]. In another study, CRM scenarios were followed by an instructor facilitated debriefing, with both teams involved in each other’s debriefing [23]. Both group and individual methods are able to adequately facilitate learning; thus, the method used should be decided based on logistical considerations and preference of the examiner.


To introduce NTS concepts, teaching should be classroom based; once this is completed and there is proficiency in basic technical skills, an appropriate form of simulation can be undertaken [12, 13]. There is evidence of this concept in the UK ‘core trainee boot camp’, where the idea of NTS and NoTSS assessment scale was introduced to early surgical trainees in the classroom, and in courses like ‘S-TEAMS’, where immediate post-graduate doctors were excluded since they would not have proficiency in technical skills [26, 36]. To date, the literature surrounding all forms of training has focused on surgical trainees and medical students rather than qualified surgeons, but new specialists can also become exhausted and burnt out due to poor NTS training and, in one Australian study, a decline in NTS was found post-fellowship for senior experienced surgeons [13, 35]. Considering that NTS do not always correlate with experience, there is evidently room for NTS training to be incorporated at higher stages of medical training [13, 25, 35]. It is useful to bear in mind that whilst training juniors and seniors together may be less time consuming and benefit juniors, it has a detrimental effect on the quality of teaching for seniors [29].

To create an NTS curriculum in urology, training should be specific to the level of seniority and subspecialty whilst remaining logistically viable [50]. To date, research has tended to focus on the validity and feasibility of specific aspects of NTS training, with high-quality studies evaluating transfer of NTS still lacking [35], which should be addressed in future studies. Simulation and CRM training improve all NTS categories at all levels; however, more research is also needed to find the relative benefit of different simulation settings (ward vs OR) for different subspecialties. Didactic teaching improves teamwork and situational awareness and should be used as an adjunct to simulation. Training bodies should investigate the NTS component deficits in the target audience (e.g. situational awareness in robotics) and adapt teaching accordingly (Table 3). Debriefing should not be overlooked: a combination of methods described may be used depending on the scenario and resources available. Any future training should thoroughly document and test their debriefing methods to improve the volume of literature on the topic. NTS teaching must be repeated and adapted throughout surgical training, with an effort to monitor and teach skills in the real OR: there should be research into creating resources given on training courses to achieve this.

Table 3 Summary of training modalities


NTS training has come a long way in the past decade, with many more courses offered for trainees and better validated training tools. A standardised surgical NTS curriculum is still lacking, facing logistical challenges alongside the issues of determining optimum training methods and testing validity. Simulation training is the most effective way to train NTS, with FIDS and lower fidelity models an acceptable alternative to HFORS. Didactic teaching improves some components of NTS and is a good adjunct to simulation, with a classroom-based element giving space for participant briefing and case-based discussion. Training TS alongside NTS removes the focus from the NTS aspect and is detrimental to learning; however, this may be avoided with adequate briefing for both participants and examiners. Debriefing is one of the most important elements of training, allowing for longer-term learning. There are many ways to debrief; the optimum debriefing method for any given training session is one which is well planned and adapted to the specific situation, bearing in mind the experience of the trainees and the trainers. Assessment scales give participants objective feedback on their performance and are useful when used by NTS trained examiners.

NTS training is appropriate for all levels, though training juniors alongside seniors mainly benefits the former; adaptation to level of seniority and specialty is beneficial. Future research will likely focus on the amalgamation and implementation of the well-described training methods into extensive, validated curricula aimed at all stages of a medical career.



Crisis resource management


Cognitive task analysis


Dutch urological practical skills (curriculum)


Mental/motor imagery


Non-technical skills for surgeons (assessment scale)


Non-technical skills


Operating room


Technical skills


Virtual reality


  1. 1.

    Collins JW, Dell’Oglio P, Hung AJ, Brook NR (2018) The importance of technical and non-technical skills in robotic surgery training. Eur Urol Focus 4(5):674–676

    PubMed  Article  Google Scholar 

  2. 2.

    Brunckhorst O, Khan MS, Dasgupta P, Ahmed K (2017) Nontechnical skill training and the use of scenarios in modern surgical education. Curr Opin Urol 27(4):330–336

    PubMed  Article  Google Scholar 

  3. 3.

    Somasundram K, Spence H, Colquhoun AJ, McIlhenny C, Biyani CS, Jain S (2018) Simulation in urology to train non-technical skills in ward rounds. BJU Int 122(4):705–712

    PubMed  Article  Google Scholar 

  4. 4.

    Sridhar AN, Briggs TP, Kelly JD, Nathan S (2017) Training in Robotic Surgery-an Overview. Curr Urol Rep 18(8):58

    PubMed  PubMed Central  Article  Google Scholar 

  5. 5.

    Aydin A, Ahmed K, Shafi AM, Khan MS, Dasgupta P (2016) The role of simulation in urological training—a quantitative study of practice and opinions. Surgeon 14(6):301–307

    PubMed  Article  Google Scholar 

  6. 6.

    Wallace L, Raison N, Ghumman F, Moran A, Dasgupta P, Ahmed K (2017) Cognitive training: how can it be adapted for surgical education? Surgeon 15(4):231–239

    PubMed  Article  Google Scholar 

  7. 7.

    Arora S, Aggarwal R, Moran A, Sirimanna P, Crochet P, Darzi A et al (2011) Mental practice: effective stress management training for novice surgeons. J Am Coll Surg 212(2):225–233

    PubMed  Article  Google Scholar 

  8. 8.

    Kohls-Gatzoulis JA, Regehr G, Hutchison C (2004) Teaching cognitive skills improves learning in surgical skills courses: a blinded, prospective, randomized study. Can J Surg 47(4):277–283

    PubMed  PubMed Central  Google Scholar 

  9. 9.

    Raison N, Ahmed K, Abe T, Brunckhorst O, Novara G, Buffi N et al (2018) Cognitive training for technical and non-technical skills in robotic surgery: a randomised controlled trial. BJU Int 122(6):1075–1081

    PubMed  Article  Google Scholar 

  10. 10.

    van der Poel H, Brinkman W, van Cleynenbreugel B, Kallidonis P, Stolzenburg JU, Liatsikos E et al (2016) Training in minimally invasive surgery in urology: European Association of Urology/International Consultation of Urological Diseases consultation. BJU Int 117(3):515–530

    PubMed  Article  Google Scholar 

  11. 11.

    Hagemann V, Herbstreit F, Kehren C, Chittamadathil J, Wolfertz S, Dirkmann D et al (2017) Does teaching non-technical skills to medical students improve those skills and simulated patient outcome? Int J Med Educ 8:101–113

    PubMed  PubMed Central  Article  Google Scholar 

  12. 12.

    Brunckhorst O, Khan MS, Dasgupta P, Ahmed K (2015) Effective non-technical skills are imperative to robot-assisted surgery. BJU Int 116(6):842–844

    PubMed  Article  Google Scholar 

  13. 13.

    Ounounou E, Aydin A, Brunckhorst O, Khan MS, Dasgupta P, Ahmed K (2018) Nontechnical skills in surgery: a systematic review of current training modalities. J Surg Educ 7:14–24

    Google Scholar 

  14. 14.

    Karkowsky CE, Landsberger EJ, Bernstein PS, Dayal A, Goffman D, Madden RC et al (2016) Breaking Bad news in obstetrics: a randomized trial of simulation followed by debriefing or lecture. J Matern Fetal Neonatal Med 29(22):3717–3723

    PubMed  Article  Google Scholar 

  15. 15.

    Kassab E, Tun JK, Arora S, King D, Ahmed K, Miskovic D et al (2011) “Blowing up the barriers” in surgical training: exploring and validating the concept of distributed simulation. Ann Surg 254(6):1059–1065

    PubMed  Article  Google Scholar 

  16. 16.

    Brewin J, Tang J, Dasgupta P, Khan MS, Ahmed K, Bello F et al (2015) Full immersion simulation: validation of a distributed simulation environment for technical and non-technical skills training in Urology. BJU Int 116(1):156–162

    PubMed  Article  Google Scholar 

  17. 17.

    Brunckhorst O, Shahid S, Aydin A, McIlhenny C, Khan S, Raza SJ et al (2015) Simulation-based ureteroscopy skills training curriculum with integration of technical and non-technical skills: a randomised controlled trial. Surg Endosc 29(9):2728–2735

    PubMed  Article  Google Scholar 

  18. 18.

    Abdelshehid CS, Quach S, Nelson C, Graversen J, Lusch A, Zarraga J et al (2013) High-fidelity simulation-based team training in urology: evaluation of technical and nontechnical skills of urology residents during laparoscopic partial nephrectomy. J Surg Educ 70(5):588–595

    PubMed  Article  Google Scholar 

  19. 19.

    Lee JY, Mucksavage P, Canales C, McDougall EM, Lin S (2012) High fidelity simulation based team training in urology: a preliminary interdisciplinary study of technical and nontechnical skills in laparoscopic complications management. J Urol 187(4):1385–1391

    PubMed  Article  Google Scholar 

  20. 20.

    Rashid P, Gianduzzo TR (2016) Urology technical and non-technical skills development: the emerging role of simulation. BJU Int. 117(Suppl 4):9–16

    PubMed  Article  Google Scholar 

  21. 21.

    Shamim Khan M, Ahmed K, Gavazzi A, Gohil R, Thomas L, Poulsen J et al (2013) Development and implementation of centralized simulation training: evaluation of feasibility, acceptability and construct validity. BJU Int 111(3):518–523

    PubMed  Article  Google Scholar 

  22. 22.

    Gordon M, Fell CW, Box H, Farrell M, Stewart A (2017) Learning health ‘safety’ within non-technical skills interprofessional simulation education: a qualitative study. Med Educ Online 22(1):1272838

    PubMed  PubMed Central  Article  Google Scholar 

  23. 23.

    Truta TS, Boeriu CM, Copotoiu SM, Petrisor M, Turucz E, Vatau D et al (2018) Improving nontechnical skills of an interprofessional emergency medical team through a one day crisis resource management training. Med (Baltim) 97(32):e11828

    Article  Google Scholar 

  24. 24.

    Yee B, Naik VN, Joo HS, Savoldelli GL, Chung DY, Houston PL et al (2005) Nontechnical skills in anesthesia crisis management with repeated exposure to simulation-based education. Anesthesiology 103(2):241–248

    PubMed  Article  Google Scholar 

  25. 25.

    Preece R (2015) The current role of simulation in urological training. Cent Eur J Urol 68(2):207–211

    Article  Google Scholar 

  26. 26.

    Stewart-Parker E, Galloway R, Vig S (2017) S-TEAMS: a truly multiprofessional course focusing on nontechnical skills to improve patient safety in the operating theater. J Surg Educ 74(1):137–144

    PubMed  Article  Google Scholar 

  27. 27.

    Robertson JM, Dias RD, Yule S, Smink DS (2017) Operating room team training with simulation: a systematic review. J Laparoendosc Adv Surg Tech A. 27(5):475–480

    PubMed  Article  Google Scholar 

  28. 28.

    Raison N, Wood T, Brunckhorst O, Abe T, Ross T, Challacombe B et al (2017) Development and validation of a tool for non-technical skills evaluation in robotic surgery-the ICARS system. Surg Endosc 31(12):5403–5410

    PubMed  Article  Google Scholar 

  29. 29.

    de Vries AH, Schout BM, van Merrienboer JJ, Pelger RC, Koldewijn EL, Muijtjens AM et al (2017) High educational impact of a national simulation-based urological curriculum including technical and non-technical skills. Surg Endosc 31(2):928–936

    PubMed  Article  Google Scholar 

  30. 30.

    Martin D, Bekiaris B, Hansen G (2017) Mobile emergency simulation training for rural health providers. Rural Remote Health 17(3):4057

    PubMed  Google Scholar 

  31. 31.

    Isaranuwatchai W, Alam F, Hoch J, Boet S (2017) A cost-effectiveness analysis of self-debriefing versus instructor debriefing for simulated crises in perioperative medicine in Canada. J Educ Eval Health Prof. 13:44

    PubMed  Article  Google Scholar 

  32. 32.

    Lovegrove CE, Abe T, Aydin A, Veneziano D, Sarica K, Khan MS et al (2017) Simulation training in upper tract endourology: myth or reality? Minerva Urol Nefrol 69(6):579–588

    PubMed  Google Scholar 

  33. 33.

    Sherwood RJ, Francis G (2018) The effect of mannequin fidelity on the achievement of learning outcomes for nursing, midwifery and allied healthcare practitioners: systematic review and meta-analysis. Nurse Educ Today 69:81–94

    PubMed  Article  Google Scholar 

  34. 34.

    Soria F, Morcillo E, Sanz JL, Budia A, Serrano A, Sanchez-Margallo FM (2014) Description and validation of realistic and structured endourology training model. Am J Clin Exp Urol. 2(3):258–265

    PubMed  PubMed Central  Google Scholar 

  35. 35.

    Gostlow H, Marlow N, Thomas MJ, Hewett PJ, Kiermeier A, Babidge W et al (2017) Non-technical skills of surgical trainees and experienced surgeons. Br J Surg 104(6):777–785

    CAS  PubMed  Article  Google Scholar 

  36. 36.

    Bamford R, Langdon L, Rodd CA, Eastaugh-Waring S, Coulston JE (2018) Core trainee boot camp, a method for improving technical and non-technical skills of novice surgical trainees. A before and after study. Int J Surg. 57:60–65

    CAS  PubMed  Article  Google Scholar 

  37. 37.

    Yule S, Gupta A, Gazarian D, Geraghty A, Smink DS, Beard J et al (2018) Construct and criterion validity testing of the Non-Technical Skills for Surgeons (NOTSS) behaviour assessment tool using videos of simulated operations. Br J Surg 105(6):719–727

    CAS  PubMed  Article  Google Scholar 

  38. 38.

    Wood TC, Raison N, Haldar S, Brunckhorst O, McIlhenny C, Dasgupta P et al (2017) Training tools for nontechnical skills for surgeons—a systematic review. J Surg Educ. 74(4):548–578

    PubMed  Article  Google Scholar 

  39. 39.

    de Vries AH, Muijtjens AMM, van Genugten HGJ, Hendrikx AJM, Koldewijn EL, Schout BMA et al (2018) Development and validation of the TOCO-TURBT tool: a summative assessment tool that measures surgical competency in transurethral resection of bladder tumour. Surg Endosc 32:4923–4931

    PubMed  Article  Google Scholar 

  40. 40.

    Garden AL, Le Fevre DM, Waddington HL, Weller JM (2015) Debriefing after simulation-based non-technical skill training in healthcare: a systematic review of effective practice. Anaesth Intensive Care 43(3):300–308

    CAS  PubMed  Article  Google Scholar 

  41. 41.

    Halamek LP (2016) Simulation and debriefing in neonatology 2016: mission incomplete. Semin Perinatol 40(7):489–493

    PubMed  Article  Google Scholar 

  42. 42.

    McKechnie A (2016) The importance and practice of debrief in medical simulation. Br J Hosp Med (Lond) 77(3):184–187

    Article  Google Scholar 

  43. 43.

    Sawyer T, Eppich W, Brett-Fleegler M, Grant V, Cheng A (2016) More than one way to debrief: a critical review of healthcare simulation debriefing methods. Simul Healthc 11(3):209–217

    PubMed  Article  Google Scholar 

  44. 44.

    Jaye P, Thomas L, Reedy G (2015) ‘The Diamond’: a structure for simulation debrief. Clin Teach 12(3):171–175

    PubMed  PubMed Central  Article  Google Scholar 

  45. 45.

    Aghera A, Emery M, Bounds R, Bush C, Stansfield RB, Gillett B et al (2018) A randomized trial of SMART goal enhanced debriefing after simulation to promote educational actions. West J Emerg Med 19(1):112–120

    PubMed  Article  Google Scholar 

  46. 46.

    Levett-Jones T, Lapkin S (2014) A systematic review of the effectiveness of simulation debriefing in health professional education. Nurse Educ Today 34(6):e58–e63

    PubMed  Article  Google Scholar 

  47. 47.

    Grant JS, Dawkins D, Molhook L, Keltner NL, Vance DE (2014) Comparing the effectiveness of video-assisted oral debriefing and oral debriefing alone on behaviors by undergraduate nursing students during high-fidelity simulation. Nurse Educ Pract 14(5):479–484

    PubMed  Article  Google Scholar 

  48. 48.

    Henneman EA, Cunningham H, Fisher DL, Plotkin K, Nathanson BH, Roche JP et al (2014) Eye tracking as a debriefing mechanism in the simulated setting improves patient safety practices. Dimens Crit Care Nurs. 33(3):129–135

    PubMed  Article  Google Scholar 

  49. 49.

    Sivertsen N, McNeill L (2016) RE-do stations after high-fidelity simulation debrief in nursing education. Aust Nurs Midwifery J 24(4):34

    PubMed  Google Scholar 

  50. 50.

    Aydin A, Raison N, Khan MS, Dasgupta P, Ahmed K (2016) Simulation-based training and assessment in urological surgery. Nat Rev Urol 13(9):503–519

    PubMed  Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Kamran Ahmed.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Griffin, C., Aydın, A., Brunckhorst, O. et al. Non-technical skills: a review of training and evaluation in urology. World J Urol 38, 1653–1661 (2020).

Download citation


  • Non-technical skills
  • Training
  • Assessment
  • Debriefing
  • Urology