Robust Laparoscopic Instruments Tracking Using Colored Strips

  • Virginia MamoneEmail author
  • Rosanna Maria Viglialoro
  • Fabrizio Cutolo
  • Filippo Cavallo
  • Simone Guadagni
  • Vincenzo Ferrari
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10325)


To assist surgeons in the acquisition of the required skills for the proper execution of the laparoscopic procedure, surgical simulators are used. During training with simulators it is useful to provide a surgical performance quantitative evaluation. Recent research works showed that such evaluation can be obtained by tracking the laparoscopic instruments, using only the images provided by the laparoscope and without hindering the surgical scene. In this work the state of the art method is improved so that a robust tracking can run even with the noisy background provided by realistic simulators. The method was validated by comparison with the tracking of a “chess-board” pattern and following tests were performed to check the robustness of the developed algorithm. Despite the noisy environment, the implemented method was found to be able to track the tip of the surgical instrument with a good accuracy compared to the other studies in the literature.


Optical tracking Single camera Laparoscopic training Surgical simulation Surgical performance evaluation 



This research work was supported by VALVETECH project, FAS fund – Tuscany Region (Realization of a newly developed polymeric aortic valve, implantable through robotic platform with minimally invasive surgical techniques) and SThARS project, grant “Ricerca finalizzata e Giovani ricercatori 2011–2012” Young Researchers - Italian Ministry of Health (Surgical Training in identification and isolation of deformable tubular structures with hybrid Augmented Reality Simulation).


  1. 1.
    Freschi, C., et al.: Technical review of the da Vinci surgical telemanipulator. Int. J. Med. Robot. Comput. Assist. Surg. 9(4), 396–406 (2013)CrossRefGoogle Scholar
  2. 2.
    Champion, H.R., Gallagher, A.G.: Surgical simulation—a ‘good idea whose time has come’. Br. J. Surg. 90(7), 767–768 (2003)CrossRefGoogle Scholar
  3. 3.
    Kneebone, R.: Simulation in surgical training: educational issues and practical implications. Med. Educ. 37(3), 267–277 (2003)CrossRefGoogle Scholar
  4. 4.
    Cavallo, F., et al.: Biomechanics-machine learning system for surgical gesture analysis and development of technologies for minimal access surgery. Surg. Innov. 21(5), 504–512 (2014)CrossRefGoogle Scholar
  5. 5.
    Lahanas, V., Georgiou, E., Loukas, C.: Surgical simulation training systems: box trainers, virtual reality and augmented reality simulators. Int. J. Adv. Robot. Automn. 1, 1–9 (2016)Google Scholar
  6. 6.
    Moglia, A., et al.: Patient specific surgical simulator for the evaluation of the movability of bimanual robotic arms. Stud. Health Technol. Inform. 163, 379–385 (2011)Google Scholar
  7. 7.
    Turini, G., et al.: Patient-specific surgical simulator for the pre-operative planning of single-incision laparoscopic surgery with bimanual robots. Comput. Aided Surg. 17(3), 103–112 (2012)CrossRefGoogle Scholar
  8. 8.
    Ferrari, V., et al.: Augmented reality visualization of deformable tubular structures for surgical simulation. Int. J. Med. Robot. Comput. Assist. Surg. 12, 231–240 (2015)CrossRefGoogle Scholar
  9. 9.
    Allan, M., Chang, P.-L., Ourselin, S., Hawkes, D.J., Sridhar, A., Kelly, J., Stoyanov, D.: Image based surgical instrument pose estimation with multi-class labelling and optical flow. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 331–338. Springer, Cham (2015). doi: 10.1007/978-3-319-24553-9_41 CrossRefGoogle Scholar
  10. 10.
    Loukas, C., Lahanas, V., Georgiou, E.: An integrated approach to endoscopic instrument tracking for augmented reality applications in surgical simulation training. Int. J. Med. Robot. 9(4), e34–e51 (2013)CrossRefGoogle Scholar
  11. 11.
    Viglialoro, R.M., Condino, S., Gesi, M., Ferrari, M., Ferrari, V.: Augmented reality simulator for laparoscopic cholecystectomy training. In: De Paolis, L.T., Mongelli, A. (eds.) AVR 2014. LNCS, vol. 8853, pp. 428–433. Springer, Cham (2014). doi: 10.1007/978-3-319-13969-2_33 Google Scholar
  12. 12.
    Viglialoro, R.M., et al.: AR visualization of “Synthetic Calot’s Triangle” for training in cholecystectomy. In: 12th IASTED International Conference on Biomedical Engineering. BioMed, Austria (2016)Google Scholar
  13. 13.
    Viglialoro, R.M., et al.: A physical patient specific simulator for cholecystectomy training. In: Computer Assisted Radiology and Surgery (CARS) (2012)Google Scholar
  14. 14.
    Condino, S., Viglialoro, R.M., Fani, S., Bianchi, M., Morelli, L., Ferrari, M., Bicchi, A., Ferrari, V.: Tactile augmented reality for arteries palpation in open surgery training. In: Zheng, G., Liao, H., Jannin, P., Cattin, P., Lee, S.-L. (eds.) MIAR 2016. LNCS, vol. 9805, pp. 186–197. Springer, Cham (2016). doi: 10.1007/978-3-319-43775-0_17 CrossRefGoogle Scholar
  15. 15.
    Condino, S., et al.: How to build patient-specific synthetic abdominal anatomies. An innovative approach from physical toward hybrid surgical simulators. Int. J. Med. Robot. 7(2), 202–213 (2011)CrossRefGoogle Scholar
  16. 16.
    Kyriakoulis, N., Gasteratos, A.: Light-invariant 3D object’s pose estimation using color distance transform. In: IEEE International Conference Imaging Systems and Techniques (IST) (2010)Google Scholar
  17. 17.
    Diotte, B., et al.: Multi-modal intra-operative navigation during distal locking of intramedullary nails. IEEE Trans. Med. Imaging 34(2), 487–495 (2015)CrossRefGoogle Scholar
  18. 18.
    Cutolo, F., et al.: Robust and accurate algorithm for wearable stereoscopic augmented reality with three indistinguishable markers. Electronics 5(3) (2016). Article number 59Google Scholar
  19. 19.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)CrossRefGoogle Scholar
  20. 20.
    Chmarra, M.K., et al.: Retracting and seeking movements during laparoscopic goal-oriented movements. Is the shortest path length optimal? Surg. Endosc. 22(4), 943–949 (2008)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Virginia Mamone
    • 1
    Email author
  • Rosanna Maria Viglialoro
    • 1
    • 2
  • Fabrizio Cutolo
    • 1
  • Filippo Cavallo
    • 3
  • Simone Guadagni
    • 4
  • Vincenzo Ferrari
    • 1
    • 5
  1. 1.EndoCAS, Department of Translational Research and of New Surgical and Medical TechnologiesUniversity of PisaPisaItaly
  2. 2.Vascular Surgery UnitCisanello University Hospital AOUPPisaItaly
  3. 3.Sant’Anna School of Advanced StudiesPisaItaly
  4. 4.General Surgery Unit, Department of Oncology Transplantation and New TechnologiesUniversity of PisaPisaItaly
  5. 5.Department of Information EngineeringUniversity of PisaPisaItaly

Personalised recommendations