Metrics for the Measurement of Skill

  • Anthony G. Gallagher
  • Gerald C. O’Sullivan
  • Gerald C. O’Sullivan
Part of the Improving Medical Outcome - Zero Tolerance book series (IMOZT)


The ‘Bristol Case’ (Senate of Surgery 1998) and the “To Err is Human” (Kohn et al. 2000) report have revealed a major deficiency in the area of surgical education, training, and assessment. Analysis revealed that there is no uniform or consistent training in surgical skills, either at a local or national level.


Laparoscopic Cholecystectomy Virtual Reality Operational Definition Task Analysis Surgical Skill 


  1. Adair JG. The Hawthorne effect: a reconsideration of the methodological artifact. J Appl Psychol. 1984;69(2):334-345.CrossRefGoogle Scholar
  2. Ahlberg G, Enochsson L, Gallagher AG, et al. Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg. 2007;193(6):797-804.PubMedCrossRefGoogle Scholar
  3. Byrne D. Interpersonal attraction and attitude similarity. J Abnorm Soc Psychol. 1961;62(3):713-715.PubMedCrossRefGoogle Scholar
  4. Catania A. Learning. Englewood Cliffs: Prentice-Hall; 1984.Google Scholar
  5. Coleman J, Nduka CC, Darzi A. Virtual reality and laparoscopic surgery. Br J Surg. 1994;81(12):1709-1711.PubMedCrossRefGoogle Scholar
  6. Darzi A, Smith S, Taffinder N. Assessing operative skill. BMJ. 1999;318(7188):887.PubMedCrossRefGoogle Scholar
  7. Francis NK, Hanna GB, Cuschieri A. Reliability of the Dundee Endoscopic Psychomotor Tester (DEPT) for dominant hand performance. Surg Endosc. 2001;15(7):673-676.PubMedCrossRefGoogle Scholar
  8. Francis NK, Hanna GB, Cuschieri A. The performance of master surgeons on the Advanced Dundee Endoscopic Psychomotor Tester: contrast validity study. Arch Surg. 2002;137(7):841.PubMedCrossRefGoogle Scholar
  9. Gallagher AG, Satava RM. Virtual reality as a metric for the assessment of laparoscopic psychomotor skills. Learning curves and reliability measures. Surg Endosc. 2002;16(12):1746-1752.PubMedCrossRefGoogle Scholar
  10. Gallagher AG, McClure N, McGuigan J, Crothers I, Browning J. Virtual reality training in laparoscopic surgery: a preliminary assessment of minimally invasive surgical trainer virtual reality (MIST VR). Endoscopy. 1999;31(4):310-313.PubMedCrossRefGoogle Scholar
  11. Gallagher AG, Hughes C, Reinhardt-Rutland AH, McGuigan J, McClure N. A case-control comparison of traditional and virtual-reality training in laparoscopic psychomotor performance. Minim Invasive Ther Allied Technol. 2000;9(5):347-352.CrossRefGoogle Scholar
  12. Gallagher AG, Richie K, McClure N, McGuigan J. Objective psychomotor skills assessment of experienced, junior, and novice laparoscopists with virtual reality. World J Surg. 2001;25(11):1478-1483.PubMedCrossRefGoogle Scholar
  13. Gallagher AG, Smith CD, Bowers SP, et al. Psychomotor skills assessment in practicing surgeons experienced in performing advanced laparoscopic procedures. J Am Coll Surg. 2003;197(3):479-488.PubMedCrossRefGoogle Scholar
  14. Gallagher AG, Lederman AB, McGlade K, Satava RM, Smith CD. Discriminative validity of the minimally invasive surgical trainer in virtual reality (MIST-VR) using criteria levels based on expert performance. Surg Endosc. 2004;18(4):660-665.PubMedCrossRefGoogle Scholar
  15. Gallagher AG, Neary P, Gillen P, et al. Novel method for assessment and selection of trainees for higher surgical training in general surgery. Aust N Z J Surg. 2008;78(4):282-290.CrossRefGoogle Scholar
  16. Gallagher AG, Leonard G, Traynor OJ. Role and feasibility of psychomotor and dexterity testing in selection for surgical training. Aust N Z J Surg. 2009;79(3):108-113.CrossRefGoogle Scholar
  17. Ghazisaedy M, Adamczyk D, Sandin DJ, Kenyon RV, DeFanti TA. Ultrasonic calibration of a magnetic tracker in a virtual realityspace. Proceedings of the IEEE Virtual Reality Annual International Symposium (VRAIS 95), Research Triangle Park, NC, March 1995.Google Scholar
  18. Gordon R. Great Medical Disasters. London: House of Stratus Ltd.; 2001.Google Scholar
  19. Hance J, Aggarwal R, Stanbridge R, et al. Objective assessment of technical skills in cardiac surgery. Eur J Cardiothorac Surg. 2005;28(1):157-162.PubMedCrossRefGoogle Scholar
  20. Hanna GB, Drew T, Clinch P, Hunter B, Cuschieri A. Computer-controlled endoscopic performance assessment system. Surg Endosc. 1998;12(7):997-1000.PubMedCrossRefGoogle Scholar
  21. Jordan JA, Gallagher AG, McGuigan J, McClure N. Randomly alternating image presentation during laparoscopic training leads to faster automation to the “fulcrum effect”. Endoscopy. 2000;32(4):317-321.PubMedCrossRefGoogle Scholar
  22. Jordan JA, Gallagher AG, McGuigan J, McClure N. Virtual reality training leads to faster adaptation to the novel psychomotor restrictions encountered by laparoscopic surgeons. Surg Endosc. 2001;15(10):1080-1084.PubMedCrossRefGoogle Scholar
  23. Kazdin AE. Behavior Modification in Applied Settings. Pacific Grove: Brooks/Cole Publishing Co.; 1994.Google Scholar
  24. Kohn LT, Corrigan JM, Donaldson MS. To Err Is Human: Building a Safer Health System. Washington: National Academy Press; 2000:196-197.Google Scholar
  25. Landsberger HA. Hawthorne Revisited. New York: Cornell University Press; 1968.Google Scholar
  26. Larsen CR, Grantcharov T, Aggarwal R, et al. Objective assessment of gynecologic laparoscopic skills using the LapSimGyn virtual reality simulator. Surg Endosc. 2006;20(9):1460-1466.PubMedCrossRefGoogle Scholar
  27. Luchins AS. Primacy-recency in impression formation. In: Hovland CI, ed. The Order of Presentation in Persuasion. New Haven: Yale University Press; 1957:33-61.Google Scholar
  28. Martin JA, Regehr G, Reznick R, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. 1997;84(2):273-278.PubMedCrossRefGoogle Scholar
  29. Meyer K, Applewhite HL, Biocca FA. A survey of position trackers. Presence: Teleoperators and Virtual Environments. 1992;1(2):173-200.Google Scholar
  30. Nixon MA, McCallum BC, Fright WR, Price NB. The effects of metals and interfering fields on electromagnetic trackers. Presence. 1998;7(2):204-218.CrossRefGoogle Scholar
  31. Pearson AM, Gallagher AG, Rosser JC, Satava RM. Evaluation of structured and quantitative training methods for teaching intracorporeal knot tying. Surg Endosc. 2002;16(1):130-137.PubMedCrossRefGoogle Scholar
  32. Perloff RM. The Dynamics of Persuasion. New York: Lawrence Erlbaum Associates; 2008.Google Scholar
  33. Raab FH, Blood EB, Steiner TO, Jones HR. Magnetic position and orientation tracking system. IEEE Trans Aerosp Electron Sys. 1979;AES-15(5):709-718.CrossRefGoogle Scholar
  34. Reber AS, Reber ES. The Penguin Dictionary of Psychology. London: Penguin Books; 1985.Google Scholar
  35. Regenbogen SE, Greenberg CC, Studdert DM, Lipsitz SR, Zinner MJ, Gawande AA. Patterns of technical error among surgical malpractice claims: an analysis of strategies to prevent injury to surgical patients. Ann Surg. 2007;246(5):705.PubMedCrossRefGoogle Scholar
  36. Riva G. Applications of virtual environments in medicine. Methods Inf Med. 2003;42(5):524-534.PubMedGoogle Scholar
  37. Satava RM. Virtual reality surgical simulator. The first steps. Surg Endosc. 1993;7(3):203-205.PubMedCrossRefGoogle Scholar
  38. Senate of Surgery. Response to the General Medical Council Determination on the Bristol Case: Senate Paper 5. London: The Senate of Surgery of Great Britain and Ireland; 1998.Google Scholar
  39. Seymour NE, Gallagher AG, Roman SA, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg. 2002;236(4):458-463; discussion 463-454.PubMedCrossRefGoogle Scholar
  40. Smith SG, Torkington J, Brown TJ, Taffinder NJ, Darzi A. Motion analysis: a tool for assessing laparoscopic dexterity in the performance of a laboratory-based laparoscopic cholecystectomy. Surg Endosc. 2002;16:640-645.PubMedCrossRefGoogle Scholar
  41. Spencer F. Teaching and measuring surgical techniques: the technical evaluation of competence. Bull Am Coll Surg. 1978;63:9-12.Google Scholar
  42. Taffinder N, Sutton C, Fishwick R, McManus I, Darzi A. Validation of virtual reality to teach and assess psychomotor skills in laparoscopic surgery: results from randomised controlled studies using the MIST VR laparoscopic simulator. Stud Health Technol Inform. 1998;50:124-130.PubMedGoogle Scholar
  43. Van Sickle K, Ritter EM, Baghai M, et al. Prospective, randomized, double-blind trial of curriculum-based training for intracorporeal suturing and knot tying. J Am Coll Surg. 2008;207(4):560-568.PubMedCrossRefGoogle Scholar
  44. Wilson MS, Middlebrook A, Sutton C, Stone R, McCloy RF. MIST VR: a virtual reality trainer for laparoscopic surgery assesses performance. Ann R Coll Surg Engl. 1997;79(6):403-404.PubMedGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  • Anthony G. Gallagher
    • 1
  • Gerald C. O’Sullivan
    • 2
  • Gerald C. O’Sullivan
    • 3
  1. 1.School of MedicineUniversity College CorkCorkIreland
  2. 2.Mercy University HospitalCorkIreland
  3. 3.Cork Cancer Research CentreUniversity College CorkCorkIreland

Personalised recommendations