Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Ophthalmic anterior segment surgery is among the most technically challenging manual procedures. Penetrating Keratoplasty (PKP) is a well-established transplant procedure for the treatment of multiple diseases of the cornea. In PKP, the full thickness of the diseased cornea is removed and replaced with a donor cornea that is positioned into place and sutured with stitches. Deep Anterior Lamellar Keratoplasty (DALK) is proposed as an alternative method for corneal disorders not affecting the endothelium. The main difference of DALK compared to PKP is the preservation of the patient’s own endothelium. This advantage reduces the risk of immunologic reactions and graft failure while showing similar overall visual outcomes. However, DALK is generally more complicated and time-consuming with a steep learning curve particularly when the host stroma is manually removed layer by layer [4]. In addition, high rate of intraoperative perforation keeps DALK from becoming surgeons’ method of choice [7]. To overcome the long surgical time and high perforation rate of DALK, in [1] Anwar et al. have proposed the big-bubble DALK technique (BB-DALK). The fundamental step of the big-bubble technique is the insertion of a needle into the deep stroma where air is injected with the goal of separating the posterior stroma and the Descemet’s Membrane (DM). The needle is intended to penetrate to a depth of more than \(60\,\%\) of the cornea, where the injection of air in most cases forms a bubble. However, in fear of perforating the DM, surgeons often stop the insertion before the target depth, where air injection results only in diffuse emphysema of the anterior stroma [7]. When bubble formation is not achieved, effort on exposing a deep layer nearest possible to the DM carries the risk of accidental perforation which brings further complications to the surgical procedure.

Optical Coherence Tomography (OCT) has been shown to increase the success rate of the procedure by determining the depth of the cannula before attempting the air injection [2]. Furthermore, recent integration of Spectral Domain OCT (SD-OCT) into surgical microscopes gives the possibility of continuous monitoring of the needle insertion. However, current OCT acquisition configurations and available tools to visualize the acquired scans are insufficient for the purpose. Metallic instruments interfere with the OCT signal leading to obstruction of deep structures. The accurate depth of the needle can only be perceived by removing the needle and imaging the created tunnel since the image captured when the needle is in position only shows the reflection of the top segment of the metallic instrument [2]. Also, limited field of view makes it hard to keep the OCT position over the needle when pressure is applied for insertion.

Here we propose a complete system as a guidance tool for BB-DALK. The system consists of modified 3D+t OCT acquisition using a microscope-mounted scanner, sophisticated visualization, tracking of the epithelium (top) and endothelium (bottom) layers and providing depth information using Augmented Reality (AR). The method addresses all aspects of the indicated complex procedure, hence is a practical solution to improve surgeons’ and patients’ experience.

2 Method

As depicted in Fig. 1, the system is based on an OPMI LUMERA 700 microscope equipped with a modified integrated RESCAN 700 OCT device (Carl Zeiss Meditec, Germany). A desktop computer with a quad-core Intel Core i7 CPU, a single NVIDIA GeForce GTX TITAN X GPU and two display screens are connected to the OCT device. Interaction with the guidance system is done by the surgeon’s assistant via a 3D mouse (3Dconnexion, Germany). The surgeon performs the procedure under the microscope while looking at the screens for both microscopic and OCT feedback. The experiments are performed on ex vivo pig eyes as shown in Fig. 3a using 27 and 30 gauge needles. For evaluations, a micromanipulator and a plastic anterior segment phantom eye are used.

Fig. 1.
figure 1

Experimental setup of the guidance system.

Full size image

2.1 OCT Acquisition

The original configuration of the intraoperative OCT device is set to acquire B-scans consisting of either 512 or 1024 A-scans. It can be set to acquire a single B-scan, 2 orthogonal B-scans or 5 parallel B-scans. For the proposed guidance system, the OCT device is set to provide 30 B-scans each with 90 A-scan samples by reprogramming the movement of its internal mirror galvanometers. B-scans are captured in a reciprocating manner for shorter scanning time. The scan region covered by the system is 2 mm by 6 mm. The depth of each A-scan is 1024 pixels corresponding to 2 mm in tissue. The concept is illustrated in Fig. 2a.

Fig. 2.
figure 2

(a): The modified pattern of OCT acquisition. (b): The lateral visualization of the cornea (orange) and the surgical needle (gray) in an OCT cuboid.

Full size image

The cuboid of \(30 \times 90 \times 1024\) voxels is scanned at the rate of 10 volumes per second. Since the cuboid is a 3D grid of samples from a continuous scene, it is interpolated using tricubic interpolants to the target resolution of \(180 \times 540 \times 180\) voxels (Fig. 2b). For that, frames are first averaged along the depth to obtain 30 frames of \(90 \times 30\) pixels. Then in each cell of the grid, a tricubic interpolant which maps coordinates to intensity values is defined as follows:

$$\begin{aligned} f(x, y, z) = \sum _{i, j, k = 0}^{3}c_{ijk}x^{i}y^{j}z^{k}, x, y, z \in [0, 1], \end{aligned}$$
(1)

in which \(c_{ijk}\) are the 64 interpolant coefficients calculated locally from the grid sample points and their derivatives. The coefficients are calculated by multiplication of a readily available \(64 \times 64\) matrix and the vector of 64 elements consisting of 8 sample points and their derivatives [6]. The interpolation is implemented on the CPU in a parallel fashion.

2.2 Visualization

The achieved 3D OCT volume is visualized on both 2D monitors using GPU ray casting with 100 rays per pixel. Maximum information in OCT images is gained from high-intensity values representing boundaries between tissue layers. Hence, the Maximum Intensity Projection (MIP) technique is employed for rendering to put an emphasis on corneal layers. Many segmentation algorithms in OCT imaging are based on adaptive intensity thresholding [5]. Metallic surgical instruments including typical needles used for the BB-DALK procedure have infrared reflectivity profiles that are distinct from cellular tissues. The 3D OCT volume is segmented into the background, the cornea and the instrument by taking advantage of various reflectivity profiles and employing K-means clustering. The initial cluster mean values are set for the background to zero, the cornea to the volume mean intensity (\(\mu \)) and the instrument to the volume mean intensity plus two standard deviations (\(\mu + 2\sigma \)). The segmentation is used to dynamically alter the color and opacity transfer functions to ensure the instrument is distinctly and continuously visualized in red, the background speckle noise is suppressed and the corneal tissue opacity does not obscure the instrument (Fig. 3b, c).

Fig. 3.
figure 3

(a): Needle insertion performed by the surgeon on the ex vivo pig eye. (b), (c): 3D visualization of the OCT cuboid with frontal and lateral viewpoints. The needle is distinctly visualized in red while endothelium (arrow) is not apparent.

Full size image

The OCT cuboid could be examined from different viewpoints according to the exact need of the surgeon. For this purpose, one of the two displays could be controlled by the surgeon’s assistant using a 3D mouse with zooming, panning and 3D rotating functionalities. The proposed guidance system maintains an automatic viewpoint of the OCT volume next to the microscopic view in the second display using the tracking procedure described below.

2.3 Tracking

The corneal DM and endothelial layer are the main targets of the BB-DALK procedure. The DM must not be perforated while the needle must be guided as close as possible to it. However, the two layers combined do not have a footprint larger than a few pixels in OCT images. As an essential part of the guidance system, DM and endothelium 3D surfaces are tracked for continuous feedback by solid visualization. The advancement of the needle in a BB-DALK procedure is examined and reported by percentage of the stroma that is above the needle tip. Hence, the epithelium surface of the cornea is also tracked to assist the surgeon by the quantitative guidance of the insertion.

Tracking in each volume is initiated by detection of the topmost and bottommost 3D points in the segmented cornea of the OCT volume. Based on the spherical shape of the cornea, two half spheres are considered as models of the endothelium and epithelium surfaces. The models are then fitted to the detected point clouds using iterative closest point (ICP) algorithm. Since the insertion of the needle deforms the cornea, ICP is utilized with 3D affine transformation at its core [3]. If the detected and the model half sphere point clouds are respectively denoted as \(P=\{p_i\}_{i=1}^{N_P} \in \mathbb {R}^{3}\) and \(M=\{m_i\}_{i=1}^{N_M} \in \mathbb {R}^{3}\), each iteration of the tracker algorithm is consecutively minimizing the following functions:

$$\begin{aligned} C(i) = \mathop {{{\mathrm{arg\,min}}}}\limits _{j \in \{1, ..., N_P\}}{\Vert (A_{k-1}m_{i} + t_{k-1}) - p_i\Vert _{2}^{2}}, \text {for all} i \in \{1, .., N_M\}. \end{aligned}$$
(2)
$$\begin{aligned} (A_{k}, t_{k}) = \mathop {{{\mathrm{arg\,min}}}}\limits _{A, t}{\frac{1}{N}\sum _{i = 1}^{N}{\Vert (Am_{i} + t) - p_{C(i)}\Vert _{2}^{2}}}. \end{aligned}$$
(3)

Equation 2 finds the correspondence C(i) between \(N \le min(N_P, N_M)\) detected and model points. Equation 3 minimizes the Euclidean distance between the detected points and the transformed points of the model. \(A_{k}\) and \(t_{k}\) are the desired affine and translation matrices at iteration k. For each incoming volume, ICP is initialized by the transformation that brings the centroid of the model points to the centroid of the detected points. The algorithm stops after 30 iterations.

The lateral view of the OCT volume gives a better understanding of the needle dimensions and advancement. Also, the perception of the distance between the instrument and the endothelium layer is best achieved from viewing the scene parallel to the surface. Therefore, the viewpoint of the second display is constantly kept parallel to a small plane at the center of the tracked endothelium surface (Fig. 4a). The pressure applied for insertion of the needle leads to deformation of the cornea. To keep the OCT field of view centered on the focus of the procedure despite the induced shifts, the OCT depth range is continuously centered to halfway between top and bottom surfaces. This is done automatically to take the burden of manual repositioning away from the surgeon.

Fig. 4.
figure 4

Augmented Reality is used to solidly visualize the endothelium and epithelium surfaces (yellow) using wireframes. A hypothetical surface (green) is rendered to indicate the insertion target depth.

Full size image
Fig. 5.
figure 5

Results of air injection in multiple pig eyes visualized from various viewpoints. The concentration of air in the bottommost region of the cornea indicates the high insertion accuracy. Deep stroma is reached with no sign of perforation.

Full size image

2.4 Augmented Reality

To further assist the surgeon, a hypothetical third surface is composed between the top and bottom surfaces indicating the insertion target depth (Fig. 4). The surgeon can choose a preferred percentage of penetration at which the imaginary surface would be rendered. Each point of the third surface is a linear combination of the corresponding points on the tracked epithelium and endothelium layers according to the chosen percentage. To visualize the detected surfaces, a wireframe mesh is formed on each of the three point sets. The two detected surfaces are rendered in yellow at their tracked position and the third surface is rendered in green at its hypothetical location. Visualization of each surface could be turned off if necessary. After injection, the presence of air leads to high-intensity voxels in the OCT volume. Therefore, the separation region is visualized effectively in red and could be used for validation of separation (Fig. 5).

3 Experiments and Results

The proposed guidance system is tested by an ophthalmic surgeon experienced in corneal transplantation procedure on several ex vivo pig eyes. The visualization gives a new dimension never seen before in conventional systems in his comment. His experience with the system signifies the ability of the real-time guidance solution to help in deep needle insertions with fewer perforation incidents.

For the purpose of real-time OCT acquisition, the surgical scene is sparsely sampled via a grid of A-scans and interpolated. To evaluate the accuracy of interpolation against dense sampling, four fixed regions of a phantom eye (\(2\,\mathrm {mm} \times 6\,\mathrm {mm} \times 2\,\mathrm {mm}\)) are scanned once with real-time sparse sampling (\(30\,px \times 90\,px \times 1024\,px\)) and two times with slow dense sampling (\(85\,px \times 512\,px \times 1024\,px\)). The sparse volumes are then interpolated to the size of the dense volumes. Volume pixels have intensities in the range of [0, 1]. For each of the four regions, Mean Absolute Error (MAE) of pixel intensities is once calculated for the two dense captures and once for one of the dense volumes and the interpolated volume. A maximum pixel intensity error of 0.073 is observed for the dense-sparse comparison while a minimum pixel intensity error of 0.043 is observed for the dense-dense comparison. The reason for the observed error in dense-dense comparison lies in the presence of OCT speckle noise which is a known phenomenon. The error observed for the dense-sparse comparison is comparable with the error induced by speckle noise hence the loss in sparse sampling is insignificant.

Human corneal thickness is reported to be around 500 \(\upmu \)m. To ensure a minimum chance of perforation when insertion is done to the depth of \(90\,\%\), the tracking accuracy required is around 50 \(\upmu \)m. To evaluate tracking accuracy of the proposed solution, a micromanipulator with a resolution of 5 \(\upmu \)m is used. A phantom eye and a pig eye are fixed to a micromanipulator and precisely moved upwards and downwards while the epithelium and endothelium surfaces are tracked. At each position, the change in the depth of the tracked surfaces corresponding points are studied. Results are presented in Table 1 using box-and-whisker plots. The whiskers are showing the minimum and maximum recorded change of all tracked points while the start and the end of the box are the first and third quartiles. Bands and dots represent medians and means of the recorded changes respectively. The actual value against which the tracking accuracy should be compared is highlighted in red on the horizontal axis of the plots. Overall, the maximum tracking error is 8.8 \(\upmu \)m.

Table 1. Evaluation of Tracking
Full size table

4 Conclusion

This work presents a novel real-time guidance system for one of the most challenging procedures in ophthalmic microsurgery. The use of medical AR aims at facilitation of the BB-DALK learning process. Experiments on ex vivo pig eyes suggest the usability and reliability of the system leading to more effective yet shorter surgery sessions. Quantitative evaluations of the system indicate its high accuracy in depicting the surgical scene and tracking its changes leading to precise and deep insertions. Future work will be in the direction of adding needle tracking and navigation, further evaluations and clinical in vivo tests.