Skip to main content

Advertisement

SpringerLink
Log in

EyeSAM: graph-based localization and mapping of retinal vasculature during intraocular microsurgery

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Robot-assisted intraocular microsurgery can improve performance by aiding the surgeon in operating on delicate micron-scale anatomical structures of the eye. In order to account for the eyeball motion that is typical in intraocular surgery, there is a need for fast and accurate algorithms that map the retinal vasculature and localize the retina with respect to the microscope.

Methods

This work extends our previous work by a graph-based SLAM formulation using a sparse incremental smoothing and mapping (iSAM) algorithm.

Results

The resulting technique, “EyeSAM,” performs SLAM for intraoperative vitreoretinal surgical use while avoiding spurious duplication of structures as with the previous simpler technique. The technique also yields reduction in average pixel error in the camera motion estimation.

Conclusions

This work provides techniques to improve intraoperative tracking of retinal vasculature by handling loop closures and achieving increased robustness to quick shaky motions and drift due to uncertainties in the motion estimation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Weiss JN, Bynoe LA (2001) Injection of tissue plasminogen activator into a branch retinal vein in eyes with central retinal vein occlusion. Ophthalmology 108(12):2249–2257

    Article  CAS  PubMed  Google Scholar 

  2. Bynoe LA, Hutchins RK, Lazarus HS, Friedberg MA (2005) Retinal endovascular surgery for central retinal vein occlusion: initial experience of four surgeons. Retina 25:625–632

    Article  PubMed  Google Scholar 

  3. Rogers S, McIntosh RL, Cheung N, Lim L, Wang JJ, Mitchell P, Kowalski JW, Nguyen H, Wong TY (2010) The prevalence of retinal vein occlusion: pooled data from population studies from the United States, Europe, Asia, and Australia. Ophthalmology 117(2):313–319

    Article  PubMed  PubMed Central  Google Scholar 

  4. Nambi M, Bernstein PS, Abbott JJ (2016) A compact telemanipulated retinal-surgery system that uses commercially available instruments with a quick-change adapter. J Med Robot Res 1(2):1630001

    Article  Google Scholar 

  5. Nasseri MA, Eder M, Eberts D, Nair S, Maier M, Zapp D, Lohmann CP, Knoll A (2013) Kinematics and dynamics analysis of a hybrid parallel-serial micromanipulator designed for biomedical applications. In: Proceedings of the IEEE/ASME international conference on advanced intelligent mechatronics, pp 293–299

  6. Wilson JT, Gerber MJ, Prince SW, Chen CW, Schwartz SD, Hubschman JP, Tsao TC (2018) Intraocular robotic interventional surgical system (IRISS): mechanical design, evaluation, and master-slave manipulation. Int J Med Robot Comput Assist Surg 14(1):e1842

    Article  Google Scholar 

  7. Taylor R, Jensen P, Whitcomb L, Barnes A, Kumar R, Stoianovici D, Gupta P, Wang Z, Dejuan E, Kavoussi L (1999) A steady-hand robotic system for microsurgical augmentation. Int J Robot Res 18(12):1201–1210

    Article  Google Scholar 

  8. Üneri A, Balicki MA, Handa J, Gehlbach P, Taylor RH, Iordachita I (2010) New steady-hand eye robot with micro-force sensing for vitreoretinal surgery. In: Proceedings of the 3rd IEEE RAS and EMBS international conference on biomedical robotics and biomechatronics, pp 814–9

  9. Willekens K, Gijbels A, Schoevaerdts L, Esteveny L, Janssens T, Jonckx B, Feyen JH, Meers C, Reynaerts D, Vander Poorten E, Stalmans P (2017) Robot-assisted retinal vein cannulation in an in vivo porcine retinal vein occlusion model. Acta Ophthalmol 95(3):270–275

    Article  PubMed  Google Scholar 

  10. Yang S, MacLachlan RA, Riviere CN (2015) Manipulator design and operation for a six-degree-of-freedom handheld tremor-canceling microsurgical instrument. IEEE ASME Trans Mechatron 20(2):761–772

    Article  PubMed  PubMed Central  Google Scholar 

  11. MacLachlan RA, Riviere CN (2009) High-speed microscale optical tracking using digital frequency-domain multiplexing. IEEE Trans Instrum Meas 58(6):1991–2001

    Article  PubMed  PubMed Central  Google Scholar 

  12. Yang S, Martel JN, Lobes LA Jr, Riviere CN (2018) Techniques for robot-aided intraocular surgery using monocular vision. Int J Robot Res 37(8):931–952

    Article  Google Scholar 

  13. Mukherjee S, Yang S, Maclachlan RA, Lobes LA, Martel JN, Riviere CN (2017) Toward monocular camera-guided retinal vein cannulation with an actively stabilized handheld robot. In: Proceedings of the IEEE international conference on robotics and automation, pp 2951–2956

  14. Becker BC, Riviere CN (2013) Real-time retinal vessel mapping and localization for intraocular surgery. In: Proceedings of the IEEE international conference on robotics and automation, pp 5346–5351

  15. Braun D, Yang S, Martel JN, Riviere CN, Becker BC (2018) EyeSLAM: real-time localization and mapping of retinal vessels during intraocular microsurgery. Int J Med Robot Comput Assist Surg 14:e1848

    Article  Google Scholar 

  16. Olson EB (2009) Real-time correlative scan matching. In: Proceedings of the IEEE international conference on robotics and automation, pp 4387–4393

  17. Dellaert F, Kaess M (2017) Factor graphs for robot perception. Found Trends Robot 6(1–2):1–139

    Google Scholar 

  18. Kaess M, Johannsson H, Roberts R, Ila V, Leonard JJ, Dellaert F (2012) iSAM2: Incremental smoothing and mapping using the Bayes tree. Int J Robot Res 31(2):216–235

    Article  Google Scholar 

  19. Engel J, Sturm J, Cremers D (2013) Semi-dense visual odometry for a monocular camera. In: Proceedings of the IEEE international conference on computer vision, pp 1449–1456

  20. Engel J, Schöps T, Cremers D (2014) LSD-SLAM: large-scale direct monocular SLAM. In: Proceedings of the European conference on computer vision, pp 834–849

  21. Lowe DG (1999) Object recognition from local scale-invariant features. In: Proceedings of the international conference on computer vision, vol 2, pp 1150–1157

  22. Bay H, Ess A, Tuytelaars T, van Gool L (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110(3):346–359

    Article  Google Scholar 

  23. Can A, Shen H, Turner JN, Tanenbaum HL, Roysam B (1999) Rapid automated tracing and feature extraction from live high-resolution retinal fundus images using direct exploratory algorithms. IEEE Trans Inf Technol Biomed 3(2):125–138

    Article  CAS  PubMed  Google Scholar 

  24. Newman P, Ho K (2005) SLAM-loop closing with visually salient features. In: Proceedings of the IEEE international conference on robotics and automation, pp 635–642

Download references

Acknowledgements

This study was partially funded by the U.S. National Institutes of Health (Grant No. R01EB000526). This article does not contain any studies with human participants or animals performed by any of the authors.

Author information

Authors and Affiliations

  1. The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, 15213, USA

    Shohin Mukherjee, Michael Kaess & Cameron N. Riviere

  2. Department of Ophthalmology, University of Pittsburgh, Pittsburgh, PA, 15213, USA

    Joseph N. Martel

Authors
  1. Shohin Mukherjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Kaess
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joseph N. Martel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cameron N. Riviere
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cameron N. Riviere.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mukherjee, S., Kaess, M., Martel, J.N. et al. EyeSAM: graph-based localization and mapping of retinal vasculature during intraocular microsurgery. Int J CARS 14, 819–828 (2019). https://doi.org/10.1007/s11548-019-01925-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-019-01925-1

Keywords

Search

Navigation