Skip to main content
SpringerLink
Log in

Non-linear-Optimization Using SQP for 3D Deformable Prostate Model Pose Estimation in Minimally Invasive Surgery

  • Conference paper
  • First Online:
Advances in Computer Vision (CVC 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 943))

Included in the following conference series:

Abstract

Augmented Reality began to be used in the last decade to guide and assist the surgeon during minimally invasive surgery. In many AR-based surgical navigation systems, a patient-specific 3D model of the surgical procedure target organ is generated from preoperative images and overlaid on the real views of the surgical field. We are currently developing an AR-based navigation system to support robot-assisted radical prostatectomy (AR-RARP) and in this paper we address the registration and localization challenge of the 3D prostate model during the procedure, evaluating the performances of a Successive Quadratic Programming (SQP) non-linear optimization technique used to align the coordinates of a deformable 3D model to those of the surgical environment. We compared SQP results in solving the 3D pose problem with those provided by the Matlab Computer Vision Toolkit perspective-three-point algorithm, highlighting the differences between the two approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Similar results are obtained also for the rotation but are not shown here for space issues.

  2. 2.

    Again, the other components of the solution vector shows similar behavior.

References

  1. Bernhardt, S., Nicolau, S.A., Soler, L., Doignon, C.: The status of augmented reality in laparoscopic surgery as of 2016. Med. Image Anal. 37, 66–90 (2017)

    Article  Google Scholar 

  2. Blender Online Community: Blender (2017). http://www.blender.org

  3. Bradski, G.: The OpenCV library. Dr. Dobb’s J. Softw. Tools 25(11) 120, 122–125 (2000)

    Google Scholar 

  4. Cutolo, F.: Augmented Reality in Image-Guided Surgery, pp. 1–11. Springer, Cham (2017)

    Google Scholar 

  5. Fida, B., Cutolo, F., di Franco, G., Ferrari, M., Ferrari, V.: Augmented reality in open surgery. Updates Surg 70, 389–400 (2018)

    Article  Google Scholar 

  6. Gao, X.S., Hou, X.R., Tang, J., Cheng, H.F.: Complete solution classification for the perspective-three-point problem. IEEE Trans. Pattern Anal. Mach. Intell. 25(8), 930–943 (2003)

    Article  Google Scholar 

  7. Intuitive: da Vinci Surgical Systems. https://www.intuitive.com Accessed 30 Aug 2018

  8. Kersten-Oertel, M., Jannin, P., Collins, D.L.: DVV: a taxonomy for mixed reality visualization in image guided surgery. IEEE Trans. Vis. Comput. Graph. 18(2), 332–352 (2012)

    Article  Google Scholar 

  9. Kong, S.H., Haouchine, N., Soares, R., Klymchenko, A., Andreiuk, B., Marques, B., Shabat, G., Piechaud, T., Diana, M., Cotin, S., Marescaux, J.: Robust augmented reality registration method for localization of solid organs’ tumors using CT-derived virtual biomechanical model and fluorescent fiducials. Surg. Endosc. 31(7), 2863–2871 (2017)

    Article  Google Scholar 

  10. Lepetit, V., Moreno-Noguer, F., Fua, P.: EPnP: an accurate O(n) solution to the PnP problem. Int. J. Comput. Vis. 81(2), 155 (2008)

    Article  Google Scholar 

  11. MATLAB: version 8.6.0 (r2015b) (2015)

    Google Scholar 

  12. Microsoft Inc.: C# Language Specification (2018). https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/language-specification/. Accessed 30 Aug 2018

  13. Nguyen, T.T., Jung, H., Lee, D.Y.: Markerless tracking for augmented reality for image-guided endoscopic retrograde cholangiopancreatography. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 7364–7367, July 2013

    Google Scholar 

  14. O’Gorman, F., Clowes, M.B.: Finding picture edges through collinearity of feature points. IEEE Trans. Comput. 25(4), 449–456 (1976)

    Article  Google Scholar 

  15. Porpiglia, F., Bertolo, R., Amparore, D., Checcucci, E., Artibani, W., Dasgupta, P., Montorsi, F., Tewari, A., Fiori, C.: Augmented reality during robot-assisted radical prostatectomy: expert robotic surgeons’ on-the-spot insights after live surgery. Minerva Urologica e Nefrologica 70(2), 226–229 (2018)

    Google Scholar 

  16. Porpiglia, F., Bertolo, R., Checcucci, E., Amparore, D., Autorino, R., Dasgupta, P., Wiklund, P., Tewari, A., Liatsikos, E., Fiori, C.: The ESUT research group: development and validation of 3D printed virtual models for robot-assisted radical prostatectomy and partial nephrectomy: urologists’ and patients’ perception. World J. Urol. 36(2), 201–207 (2018)

    Article  Google Scholar 

  17. Porpiglia, F., Checcucci, E., Amparore, D., Autorino, R., Piana, A., Bellin, A., Piazzolla, P., Massa, F., Bollito, E., Gned, D., De Pascale, A., Fiori, C.: Augmented-reality robot-assisted radical prostatectomy using hyper-accuracy three-dimensional reconstruction (HA3D\({^{\rm TM}}\)) technology: a radiological and pathological study. BJU Int. (2018)

    Google Scholar 

  18. Porpiglia, F., Fiori, C., Checcucci, E., Amparore, D., Bertolo, R.: Augmented reality robot-assisted radical prostatectomy: preliminary experience. Urology 115, 184 (2018)

    Article  Google Scholar 

  19. Sederberg, T.W., Parry, S.R.: Free-form deformation of solid geometric models. SIGGRAPH Comput. Graph. 20(4), 151–160 (1986)

    Article  Google Scholar 

  20. Simpfendörfer, T., Baumhauer, M., Müller, M., Gutt, C.N., Meinzer, H.P., Rassweiler, J.J., Guven, S., Teber, D.: Augmented reality visualization during laparoscopic radical prostatectomy. J. Endourol. 25, 1841–1845 (2011)

    Article  Google Scholar 

  21. Thompson, S., Schneider, C., Bosi, M., Gurusamy, K., Ourselin, S., Davidson, B., Hawkes, D., Clarkson, M.J.: In vivo estimation of target registration errors during augmented reality laparoscopic surgery. Int. J. Comput. Assist. Radiol. Surg. 13(6), 865–874 (2018)

    Article  Google Scholar 

  22. Unity Technologies ApS: Unity3D (2017). https://unity3d.com. Accessed 30 Aug 2018

  23. Vávra, P., Roman, J., Zonča, P., Ihnát, P., Němec, M., Kumar, J., Habib, N., El-Gendi, A.: Recent development of augmented reality in surgery: a review. J. Healthcare Eng. 2017, Article ID 4574172, 9 (2017)

    Google Scholar 

  24. Yoon, J.W., Chen, R.E., Kim, E.J., Akinduro, O.O., Kerezoudis, P., Han, P.K., Si, P., Freeman, W.D., Diaz, R.J., Komotar, R.J., Pirris, S.M., Brown, B.L., Bydon, M., Wang, M.Y., Wharen, R.E., Quinones-Hinojosa, A.: Augmented reality for the surgeon: systematic review. Int. J. Med. Robot. Comput. Assist. Surg. 14(4), e1914 (2018)

    Article  Google Scholar 

  25. Zhang, J., Zhong, Y., Smith, J., Gu, C.: Energy propagation modeling of nonlinear soft tissue deformation for surgical simulation. Simulation 94(1), 3–10 (2018)

    Article  Google Scholar 

  26. Zheng, Y., Kuang, Y., Sugimoto, S., Åström, K., Okutomi, M.: Revisiting the pnp problem: a fast, general and optimal solution. In: 2013 IEEE International Conference on Computer Vision, pp. 2344–2351, December 2013

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division of Urology, Department of Oncology, School of Medicine, University of Turin-San Luigi Gonzaga Hospital, Regione Gonzole 10, 10043, Turin, Orbassano, Italy

    Daniele Amparore, Enrico Checcucci & Francesco Porpiglia

  2. Dipartimento di Elettronica, Informatica e Bioingegneria, Politecnico di Milano, via Ponzio 51, 20133, Milan, Italy

    Marco Gribaudo

  3. Department of Management and Production Engineering, Politecnico di Torino, Corso Duca degli Abruzzi, 24, 10129, Turin, Italy

    Pietro Piazzolla & Enrico Vezzetti

Authors
  1. Daniele Amparore
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Enrico Checcucci
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marco Gribaudo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pietro Piazzolla
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Francesco Porpiglia
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Enrico Vezzetti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pietro Piazzolla .

Editor information

Editors and Affiliations

  1. Saga University, Saga, Saga, Japan

    Kohei Arai

  2. The Science and Information (SAI) Organization, Bradford, West Yorkshire, UK

    Supriya Kapoor

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Amparore, D., Checcucci, E., Gribaudo, M., Piazzolla, P., Porpiglia, F., Vezzetti, E. (2020). Non-linear-Optimization Using SQP for 3D Deformable Prostate Model Pose Estimation in Minimally Invasive Surgery. In: Arai, K., Kapoor, S. (eds) Advances in Computer Vision. CVC 2019. Advances in Intelligent Systems and Computing, vol 943. Springer, Cham. https://doi.org/10.1007/978-3-030-17795-9_35

Download citation

Publish with us

Policies and ethics

Search

Navigation