Skip to main content
SpringerLink
Log in

Augmented reality in a tumor resection model

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Augmented Reality (AR) guidance is a technology that allows a surgeon to see sub-surface structures, by overlaying pre-operative imaging data on a live laparoscopic video. Our objectives were to evaluate a state-of-the-art AR guidance system in a tumor surgical resection model, comparing the accuracy of the resection with and without the system. Our system has three phases. Phase 1: using the MRI images, the kidney’s and pseudotumor’s surfaces are segmented to construct a 3D model. Phase 2: the intra-operative 3D model of the kidney is computed. Phase 3: the pre-operative and intra-operative models are registered, and the laparoscopic view is augmented with the pre-operative data.

Methods

We performed a prospective experimental study on ex vivo porcine kidneys. Alginate was injected into the parenchyma to create pseudotumors measuring 4–10 mm. The kidneys were then analyzed by MRI. Next, the kidneys were placed into pelvictrainers, and the pseudotumors were laparoscopically resected. The AR guidance system allows the surgeon to see tumors and margins using classical laparoscopic instruments, and a classical screen. The resection margins were measured microscopically to evaluate the accuracy of resection.

Results

Ninety tumors were segmented: 28 were used to optimize the AR software, and 62 were used to randomly compare surgical resection: 29 tumors were resected using AR and 33 without AR. The analysis of our pathological results showed 4 failures (tumor with positive margins) (13.8%) in the AR group, and 10 (30.3%) in the Non-AR group. There was no complete miss in the AR group, while there were 4 complete misses in the non-AR group. In total, 14 (42.4%) tumors were completely missed or had a positive margin in the non-AR group.

Conclusions

Our AR system enhances the accuracy of surgical resection, particularly for small tumors. Crucial information such as resection margins and vascularization could also be displayed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Pessaux P, Diana M, Soler L, Piardi T, Mutter D, Marescaux J (2014) Robotic duodenopancreatectomy assisted with augmented reality and real-time fluorescence guidance. Surg Endosc 28(8):2493–2498

    Article  PubMed  Google Scholar 

  2. Marescaux J, Rubino F, Arenas M, Mutter D, Soler L (2004) Augmented-reality-assisted laparoscopic adrenalectomy. JAMA 292(18):2214–2215

  3. Pessaux P, Diana M, Soler L, Piardi T, Mutter D, Marescaux J (2015) Towards cybernetic surgery: robotic and augmented reality-assisted liver segmentectomy. Langenbecks Arch Surg Dtsch Ges Für Chir 400(3):381–385

  4. Simpfendörfer T, Baumhauer M, Müller M, Gutt CN, Meinzer H-P, Rassweiler JJ et al (2011) Augmented reality visualization during laparoscopic radical prostatectomy. J Endourol Endourol Soc 25(12):1841–1845

    Article  Google Scholar 

  5. Grimson WL, Ettinger GJ, White SJ, Lozano-Perez T, Wells WM, Kikinis R (1996) An automatic registration method for frameless stereotaxy, image guided surgery, and enhanced reality visualization. IEEE Trans Med Imaging 15(2):129–140

    Article  CAS  PubMed  Google Scholar 

  6. Bourdel N, Collins T, Pizarro D, Bartoli A, Da Ines D, Perreira B et al (2016) Augmented reality in gynecologic surgery: evaluation of potential benefits for myomectomy in an experimental uterine model. Surg Endosc 2016

  7. Bourdel N, Collins T, Pizarro D, Debize C, Grémeau A, Bartoli A et al (2017) Use of augmented reality in laparoscopic gynecology to visualize myomas. Fertil Steril 107(3):737–739

  8. Wolf I, Vetter M, Wegner I, Böttger T, Nolden M, Schöbinger M et al (2005) The medical imaging interaction toolkit. Med Image Anal 9(6):594–604

    Article  PubMed  Google Scholar 

  9. Collins T, Pizarro D, Bartoli A, Canis M, Bourdel N (2013) Realtime wide-baseline registration of the uterus in laparoscopic videos using multiple texture maps. In: Liao H, Linte CA, Masamune K, Peters TM, Zheng G (eds) Augmented reality environments for medical imaging and computer-assisted interventions [Internet]. Springer, Berlin; [cité 10 oct 2015], pp 162–171 (Lecture Notes in Computer Science). Disponible sur: http://link.springer.com/chapter/10.1007/978-3-642-40843-4_18

  10. Teber D, Guven S, Simpfendörfer T, Baumhauer M, Güven EO, Yencilek F et al (2009) Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? Preliminary in vitro and in vivo results. Eur Urol 56(2):332–338

    Article  PubMed  Google Scholar 

  11. Hafez KS, Fergany AF, Novick AC (1999) Nephron sparing surgery for localized renal cell carcinoma: impact of tumor size on patient survival, tumor recurrence and TNM staging. J Urol 162(6):1930–1933

    Article  CAS  PubMed  Google Scholar 

  12. Herr HW (1999) Partial nephrectomy for unilateral renal carcinoma and a normal controlateral kidney: 10-year followup. J Urol 161(1):33–35

    Article  CAS  PubMed  Google Scholar 

  13. Lee CT, Katz J, Shi W, Thaler HT, Reuter VE, Russo P (2000) Surgical management of renal tumors 4 cm or less in a contemporary cohort. J Urol 163(3):730–736

    Article  CAS  PubMed  Google Scholar 

  14. McKiernan J, Simmons R, Katz J, Russo P (2002) Natural history of chronic renal insufficiency after partial and radical nephrectomy. Urology 59(6):816–820

    Article  PubMed  Google Scholar 

  15. Clark PE, Schover LR, Uzzo RG, Hafez KS, Rybicki LA, Novick AC (2001) Quality of life and psychological adaptation after surgical treatment for localized renal cell carcinoma: impact of the amount of remaining renal tissue. Urology 57(2):252–256

    Article  CAS  PubMed  Google Scholar 

  16. Hafez KS, Novick AC, Campbell SC (1997) Patterns of tumor recurrence and guidelines for followup after nephron sparing surgery for sporadic renal cell carcinoma. J Urol 157(6):2067–2070

    Article  CAS  PubMed  Google Scholar 

  17. Breda A, Stepanian SV, Liao J, Lam JS, Guazzoni G, Stifelman M et al (2007) Positive margins in laparoscopic partial nephrectomy in 855 cases: a multi-institutional survey from the United States and Europe. J Urol 178(1):47–50

    Article  CAS  PubMed  Google Scholar 

  18. Gill IS, Kavoussi LR, Lane BR, Blute ML, Babineau D, Colombo JR et al (2007) Comparison of 1,800 laparoscopic and open partial nephrectomies for single renal tumors. J Urol 178(1):41–46

    Article  PubMed  Google Scholar 

  19. Yossepowitch O, Thompson RH, Leibovich BC, Eggener SE, Pettus JA, Kwon ED et al (2008) Positive surgical margins at partial nephrectomy: predictors and oncological outcomes. J Urol 179(6):2158–2163

    Article  PubMed  PubMed Central  Google Scholar 

  20. Ani I, Finelli A, Alibhai SMH, Timilshina N, Fleshner N, Abouassaly R (2013) Prevalence and impact on survival of positive surgical margins in partial nephrectomy for renal cell carcinoma: a population-based study: positive surgical margins after partial nephrectomy for RCC. BJU Int 111(8):E300–E305

    Article  PubMed  Google Scholar 

  21. Tabayoyong W, Abouassaly R, Kiechle JE, Cherullo EE, Meropol NJ, Shah ND et al (2015) Variation in surgical margin status by surgical approach among patients undergoing partial nephrectomy for small renal masses. J Urol 194(6):1548–1553

    Article  PubMed  Google Scholar 

  22. Vermooten V (1950) Indications for conservative surgery in certain renal tumors: a study based on the growth pattern of the cell carcinoma. J Urol 64(2):200–208

    Article  CAS  PubMed  Google Scholar 

  23. Russo P (2000) Renal cell carcinoma: presentation, staging, and surgical treatment. Semin Oncol 27(2):160–176

    CAS  PubMed  Google Scholar 

  24. Sutherland SE, Resnick MI, Maclennan GT, Goldman HB (2002) Does the size of the surgical margin in partial nephrectomy for renal cell cancer really matter? J Urol 167(1):61–64

    Article  PubMed  Google Scholar 

  25. Castilla EA, Liou LS, Abrahams NA, Fergany A, Rybicki LA, Myles J et al (2002) Prognostic importance of resection margin width after nephron-sparing surgery for renal cell carcinoma. Urology 60(6):993–997

    Article  PubMed  Google Scholar 

  26. Rizzo S, Calareso G, De Maria F, Zanagnolo V, Lazzari R, Cecconi A et al (2013) Gynecologic tumors: how to communicate imaging results to the surgeon. Cancer Imaging Off Publ Int Cancer Imaging Soc 13(4):611–625

    Google Scholar 

  27. Nakamoto M, Nakada K, Sato Y, Konishi K, Hashizume M, Tamura S (2008) Intraoperative magnetic tracker calibration using a magneto-optic hybrid tracker for 3-D ultrasound-based navigation in laparoscopic surgery. IEEE Trans Med Imaging 27(2):255–270

    Article  PubMed  Google Scholar 

  28. Hughes-Hallett A, Mayer EK, Marcus HJ, Cundy TP, Pratt PJ, Darzi AW et al (2014) Augmented reality partial nephrectomy: examining the current status and future perspectives. Urology 83(2):266–273

    Article  PubMed  Google Scholar 

  29. Simpfendörfer T, Gasch C, Hatiboglu G, Müller M, Maier-Hein L, Hohenfellner M et al (2016) Intraoperative computed tomography imaging for navigated laparoscopic renal surgery: first clinical experience. J Endourol 30(10):1105–1111

    Article  PubMed  Google Scholar 

  30. Allen BC, Tirman P, Jennings Clingan M, Manny J, Del Gaizo AJ, Leyendecker JR (2014) Characterizing solid renal neoplasms with MRI in adults. Abdom Imaging 39(2):358–387

    Article  PubMed  Google Scholar 

  31. Fananapazir G, Lamba R, Lewis B, Corwin MT, Naderi S, Troppmann C (2015) Utility of MRI in the characterization of indeterminate small renal lesions previously seen on screening CT scans of potential renal donor patients. AJR Am J Roentgenol 205(2):325–330

    Article  PubMed  Google Scholar 

  32. Paulus CJ, Haouchine N, Kong S-H, Soares RV, Cazier D, Cotin S (2017) Handling topological changes during elastic registration: application to augmented reality in laparoscopic surgery. Int J Comput Assist Radiol Surg 12(3):461–470

  33. Nakamura K, Naya Y, Zenbutsu S, Araki K, Cho S, Ohta S et al (2010) Surgical navigation using three-dimensional computed tomography images fused intraoperatively with live video. J Endourol 24(4):521–524

    Article  PubMed  Google Scholar 

  34. Ukimura O, Gill IS (2008) Imaging-assisted endoscopic surgery: cleveland clinic experience. J Endourol 22(4):803–810

    Article  PubMed  Google Scholar 

  35. Altamar HO, Ong RE, Glisson CL, Viprakasit DP, Miga MI, Herrell SD et al (2011) Kidney deformation and intraprocedural registration: a study of elements of image-guided kidney surgery. J Endourol 25(3):511–517

    Article  PubMed  Google Scholar 

  36. Herrell SD, Kwartowitz DM, Milhoua PM, Galloway RL (2009) Toward image guided robotic surgery: system validation. J Urol 181(2):783–790

    Article  PubMed  Google Scholar 

  37. Benincasa AB, Clements LW, Herrell SD, Galloway RL (2008) Feasibility study for image‐guided kidney surgery: Assessment of required intraoperative surface for accurate physical to image space registrations. Med Phys 35(9):4251–4261

    Article  PubMed  PubMed Central  Google Scholar 

  38. Nakamoto M, Ukimura O, Gill IS, Mahadevan A, Miki T, Hashizume M et al (2008) Realtime organ tracking for endoscopic augmented reality visualization using miniature wireless magnetic tracker. In: Medical imaging and augmented reality. Springer-Verlag, Heidelberg, German, pp 359–366

  39. Baumhauer M, Simpfendörfer T, Müller-Stich BP, Teber D, Gutt CN, Rassweiler J et al (2008) Soft tissue navigation for laparoscopic partial nephrectomy. Int J Comput Assist Radiol Surg 3(3-4):307–314

    Article  Google Scholar 

  40. Pratt P, Mayer E, Vale J, Cohen D, Edwards E, Darzi A et al (2012) An effective visualisation and registration system for image-guided robotic partial nephrectomy. J Robot Surg 6(1):23–31

    Article  PubMed  Google Scholar 

  41. Su LM, Vagvolgyi BP, Agarwal R, Reiley CE, Taylor RH, Hager GD (2009) Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology 73(4):896–900

    Article  PubMed  Google Scholar 

  42. Nosrati MS, Amir-Khalili A, Peyrat JM, Abinahed J, Al-Alao O, Al-Ansari A et al (2016) Endoscopic scene labelling and augmentation using intraoperative pulsatile motion and colour appearance cues with preoperative anatomical priors. Int J Comput Assist Radiol Surg 11(8):1409–1418

    Article  PubMed  Google Scholar 

  43. Puerto-Souza GA, Cadeddu JA, Mariottini G-L (2014) Toward long-term and accurate augmented-reality for monocular endoscopic videos. IEEE Trans Biomed Eng 61(10):2609–2620

Download references

Acknowledgement

This research has received funding from the EU’s FP7 through the ERC research grant 307483 FLEXABLE.

Author information

Authors and Affiliations

  1. Department of Gynecologic Surgery, Clermont-Ferrand University Hospital Estaing, place Lucie et Raymond Aubrac, Clermont-Ferrand, France

    Pauline Chauvet, Michel Canis & Nicolas Bourdel

  2. Faculté de Médecine, ALCoV UMR6284, CNRS/Université d’Auvergne, ISIT, Clermont-Ferrand, France

    Pauline Chauvet, Toby Collins, Clement Debize, Adrien Bartoli, Michel Canis & Nicolas Bourdel

  3. Faculté de Médecine, CICS Centre Imagerie Cellulaire Santé, 28 Place Henri Dunant, Clermont-Ferrand, France

    Lorraine Novais-Gameiro

  4. Biostatistics Unit (DRCI), University Hospital Clermont-Ferrand, 58, Rue Montalembert, 63003, Clermont-Ferrand Cedex, France

    Bruno Pereira

Authors
  1. Pauline Chauvet
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Toby Collins
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Clement Debize
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lorraine Novais-Gameiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bruno Pereira
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Adrien Bartoli
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Michel Canis
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Nicolas Bourdel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicolas Bourdel.

Ethics declarations

Disclosure

Dr Pauline Chauvet, Toby Collins, Clement Debize, Lorraine Novais-Gameiro, Bruno Pereira, Prs Adrien Bartoli and Michel Canis, and Dr Nicolas Bourdel have no conflicts of interest or financial ties to disclose.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (MP4 56711 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chauvet, P., Collins, T., Debize, C. et al. Augmented reality in a tumor resection model. Surg Endosc 32, 1192–1201 (2018). https://doi.org/10.1007/s00464-017-5791-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-017-5791-7

Keywords

Search

Navigation