Keywords

1 Introduction

Thermotherapy is a clinical procedure that uses thermal energy to induce a desired biological tissue response. Mild and localized hyperthermia can be used in combination with chemotherapy or drug delivery to improve the therapy responseĀ [1, 2]. Thermal ablation can be achieved by applying sufficient thermal energy to reach a complete destruction of various kinds of cancer cells. However, the main challenge is to cover completely the target region while preserving the surrounding healthy tissues. Monitoring the temperature across this region is necessary to control the delivered thermal energy and operating duration to precisely and successfully operate the procedureĀ [3].

A widely accepted approach to measure temperature is the use of invasive thermometersĀ [4]. However, it allows temperature monitoring only at a few spatial locations. Magnetic resonance imaging (MRI) is the current clinical standard to monitor the spatial temperature distributionĀ [5]. In addition to the high cost of MRI, it requires the therapy instruments to be MR-compatible. Furthermore, MRI is not suitable for patients with pacemaker, neurostimulator or metal implants. An alternative is to use portable and affordable ultrasound (US) techniques, and a significant number of related works have been reportedĀ [6]. These approaches exploit the temperature dependent ultrasound properties such as sound velocity and attenuation to estimate the temperature. Sound velocity or attenuation images can be generated using ultrasound tomography techniques, which typically require extensive data acquisition from multiple angles. Ultrasound tomographic images can also be reconstructed using time-of-flight (TOF) information from limited angles using an isothermal modelĀ [7]. To overcome the sparsity of the data, machine learning is a promising alternativeĀ [8].

In this work, we propose a deep learning approach for tomographic reconstruction of sound velocity images. We collected TOFs using a clinical ultrasound transducer and by integrating an active ultrasound element on a bipolar radiofrequency (RF) ablation probe. The number of acquired TOFs is limited by the number of elements in the ultrasound transducer, usually insufficient to solve for the sound velocity in the heated region. Therefore, we implemented a convolutional neural network (CNN) to reconstruct temperature images using this limited information. For the training of the network, thermal images are generated with a computational bioheat model of RF ablation, and then converted to sound velocity images to obtain simulated TOF datasets. We performed simulation and ex-vivo experiments to evaluate the proposed method.

2 Methods

2.1 Thermal Ablation Procedure and Monitoring Setup

The thermal ablation procedure is performed with bipolar RF needles to generate various ablation patternsĀ [9] and an active ultrasound element is used for temperature monitoring as shown in Fig.Ā 1(a). As the element can be integrated with the ablation probe, it does not increase the overall invasiveness of the procedure. Two different ablation patterns were considered: horizontal and diagonal as illustrated in Fig.Ā 1(b). We created the horizontal pattern by activating the two electrodes at the tips of the RF probes, and the diagonal pattern by activating crossing electrodes. During the procedure, the external ultrasound element transmits ultrasound pulses. TOF data are collected with an ultrasound transducer to detect the change in sound velocity. Therefore, the monitored region is the triangular area created between the ultrasound transducer and the element. It belongs to the monitoring image plane between the two RF probes showed in Fig.Ā 1. In this plane, the horizontal pattern showed a round-shaped temperature distribution, while the diagonal pattern showed an ellipsoid one.

Fig. 1.
figure 1

(a) The ultrasound thermal monitoring setup. (b) Left: Horizontal ablation pattern. Right: Diagonal ablation pattern.

Full size image

2.2 Thermal Image Reconstruction Using Neural Network

Training Set Generation: A RFA computational model is used to simulate the temperature evolution in a 3D domain with various tissue parameters to provide temperature images for training. A reaction-diffusion equation (Eq.Ā 1) following the Pennes bioheat modelĀ [10] is used:

$$\begin{aligned} \rho _t c_t \frac{\partial T}{\partial t} = Q + \nabla \cdot (d_t\nabla T) + R(T_{b0}-T) \end{aligned}$$
(1)

where \(\rho _t,~c_t\),Ā \(d_t\) are the density, heat capacity, and conductivity of the tissue.Ā \(T_{b0},~R,~Q\), the blood temperature, reaction term, and source term modeling the heat from the ablation device. The implementation is based on the Lattice Boltzmann Method and inhomogeneous tissue structures can be consideredĀ [11]. To simulate RF ablation with bipolar probes, and thus various ablation lesion shapes, we assume the two RF electrodes as independent heating sources. Their temperatures are imposed as Dirichlet boundary conditionsĀ [11]. For each ablation pattern, we simulated a procedure of 8Ā min of heating followed by 2Ā min of cooling, which corresponds to 600 temperature images having a temporal resolution of 1Ā s. We wanted to mimic the ex-vivo experiment setup, therefore porcine tissue parameters were used, even though a shorter cooling period was achieved due to a data storage limitation in the current experimental setupĀ [11]. For the horizontal pattern, the temperature range was between 22.0Ā \(^{\circ }\)C and 37.8Ā \(^{\circ }\)C, and for the diagonal pattern, between 23.0Ā \(^{\circ }\)C and 35.9Ā \(^{\circ }\)C.

Different ultrasound element locations can also be considered. We defined a 2D image coordinate system as (Axial, Lateral) axis in millimeter scale. The image plane was divided in 60 by 60 pixels. A 6Ā cm linear 128 element ultrasound probe was placed between (0, 0) and (0, 60), and the ultrasound element was located within the image plane. The network training set is made of those images as well as the corresponding simulated TOF information.

In order to simulate the acquisition of TOF dataset, we converted the temperature images into sound velocity images as the sound velocity within the tissue changes with temperature. Since the major component of biological tissue is water, the relationship between sound velocity and temperature for biological tissue has a trend similar to the water oneĀ [12]. In this paper, we used a converting equation acquired from a tissue-mimicking phantom with a sound velocity offset compensationĀ [13] to simulate TOF information affected by a change in temperature and therefore in sound velocity even though a tissue-specific relationship could be used if the tissue type is known.

We simulated 49 different ultrasound element locations around the location used in the ex-vivo experiment, with the heating center kept fixed. For the horizontal pattern, we moved the element location from (36, 40.5) to (42, 46.5) by a 1Ā mm step in both lateral and axial directions. For the diagonal pattern, element locations between (43.5, 51) and (49.5, 57) were considered. For each of the 49 locations, data were split randomly with a 6:1 ratio between training and testing sets. Therefore, for each ablation pattern, the total number of samples was 29,400, split into 4,200 testing and 25,200 training sets. This large dataset may ensure an effective training of the network parameters without over-fitting.

Image Reconstruction Network: FigureĀ 2 shows the temperature image reconstruction neural network, which consists of two fully connected layers wrapping series of CNN. The convolutional network is symmetrically designed, consisting of convolution and trans-convolution layers. After the convolution operation, each CNN layer includes a ReLU followed by a batch normalization operation.

Fig. 2.
figure 2

Temperature image reconstruction network.

Full size image

We concatenated the 128-length initial TOF vector with any TOF vector during the procedure into a 256-length input vector. The initial TOF is always used since it provides the element location and allows to access the TOF differences during the ablation procedure, valuable information for temperature reconstruction. As we reconstructed 3600 pixel temperature images with a 256-length input vector, we expanded the parameters at the beginning of the network.

Training Results: For each ablation pattern, we performed 1000 epochs using the Pytorch libraryĀ [14]. Adam optimizer and mean squared error loss function were used. We compared the results to those obtained from another reconstruction method (CSRM)Ā [13] in TableĀ 1 at the 49 different ultrasound element locations. In this case, the ground truths are the simulated temperature images. The CRSM method used an optimization approach with additional constraints brought by a computational RFA modeling. The CNN reconstruction method had 0.94Ā \(^\circ \)C and 0.72Ā \(^\circ \)C less maximum temperature error in the imaging plane than CSRM for the horizontal and diagonal pattern respectively. We also observed that the standard deviation decreased with the CNN approach. With the CSRM method, the reconstruction accuracy is highly affected by the ultrasound element location. Indeed, for certain locations, the ultrasound propagation paths may not intersect with the heating center. Among the 49 different element locations considered, the maximum error in the sound velocity reconstruction exceeds 5Ā m/s with CSRM at 7 and 2 locations for the horizontal and diagonal pattern respectively. The CNN reconstruction method showed less temperature error at those locations since it could estimate the temperature at the heating center more precisely using information learned from other temperature distributions.

Table 1. Comparison of the CNN approach with a sound velocity reconstruction method using RFA modeling (CSRM) for the 49 different ultrasound element locations. The error is the difference of temperature in the imaging plane between the reconstructed image and the simulated image (ground truth).
Full size table

We also tested with a fully connected network by replacing the middle structure with four dense networks which were the same as the last dense network in Fig.Ā 2. The regression accuracy was similar to the CNN network with more parameters. To minimize over-fitting, we chose the hyper-parameters with the minimal number of layers maintaining the regression accuracy. The initial learning rate was \(10^{-3}\), and we re-trained with a smaller learning rate of \(10^{-5}\). We also tested our network without the last dense layers, the regression accuracy was inferior to the original network.

Fig. 3.
figure 3

Results of the ex-vivo experiments on porcine livers. (a) Temperature reconstruction for the horizontal and diagonal ablation patterns. (b) Temperature evolution over time at three different positions in the imaging plane. (Left): Horizontal ablation pattern. (Right): Diagonal ablation pattern.

Full size image

3 Ex-vivo Liver Ablation with Ultrasound Monitoring

3.1 Experiment Setup

Two ex-vivo porcine liver experiments were performed to test the performance of the trained model. Liver tissues were placed at room temperature for 12Ā h before performing the ablation. We used the setup illustrated in Fig.Ā 1(a). Bipolar ablation probes were inserted 2-cm apart and in parallel by using a holder to perform horizontal and diagonal ablation patterns. The ablation power was provided by a RF generator (Radionics Inc., USA). The ultrasound element was placed within the porcine liver tissue. We adjusted its location to the ultrasound transducer by finding the maximum signal strength within the imaging plane. We used a 10Ā MHz linear transducer L14-5W/60 (Ultrasonix Corp., Canada) with a 5ā€“14 MHz bandwidth and a SonixDAQ (Ultrasonix Corp., Canada) with a sampling frequency of 40Ā MHz. The ultrasound data was collected with a pitch-and-catch mode. The ultrasound element transmitted a pulse while the ultrasound transducer and DAQ received the signal simultaneously. The transmission and collection were synchronized by an external function generator at 1Ā Hz. We performed 8Ā min of ablation, after what the ablation probes remained in the tissue for an extra 1Ā min without RF power.

3.2 Temperature Image Reconstruction

TOF was detected by finding the first peak from the ultrasound channel data. The received signal had a center frequency of 3.7Ā MHz with a bandwidth of 2.5ā€“5.6Ā MHz. During the two ablations, we collected 540 TOF dataset for 9Ā min. The element was localized at (39.0, 43.6) and (46.6, 54.2) in the horizontal and diagonal pattern experiments. We reconstructed temperature images using the model trained with the simulation datasets and we observed a convincing temperature trend over time. The temperature evolutions at three different points: heating center, āˆ’5 and āˆ’10Ā mm away from the center along the axial direction are shown in Fig.Ā 3. The maximum TOF shift was 300Ā ns for the horizontal, and 475Ā ns for the diagonal pattern. In the horizontal pattern experiment, at around 180 and 230Ā s, the TOF increased for few samples compared to previous frames which was unexpected. This induced a temperature decrease at those time points. Nonetheless, we observed an overall temperature increase trend.

4 Discussion and Conclusion

As we use the relative changes in TOF to monitor the temperature during a thermal ablation, the complication of calculating the absolute sound velocity of different tissues is decreased. However, the variety of sound velocity changes against temperature in different tissue types may cause errors in the reconstructed temperature results. To overcome this problem, a calibration method for different tissue types can be usedĀ [13], and dataset from diverse tissue types should also be used to train the network. In this paper, the ablation power was limited due to the ongoing development of the bipolar ablation device, which limited the temperature range. But this method can be applied to ablation where higher temperatures are reached. Moreover, the ex-vivo experiment results could not be validated with other thermometry methods. MR-thermometry for example, was not an option since the ablation system is not MR-compatible. Thermocouples could block the ultrasound propagation paths, and only provide temperature information at few points. Therefore, we validated the method with simulation data, and observed an increasing temperature trend in ex-vivo experiments. Patient motion can affect the reconstruction accuracy, which is the main challenge for many ultrasound thermometry approaches. With our method, patient motion will change the location of the ultrasound element relative to the ultrasound transducer, which can be detected by a sudden change in TOF. The CNN model is trained with various ultrasound element locations, and the system could be further improved in the future to continue reconstructing temperature images using prior temperature information in the occurrence of patient motion.

Ultrasound is a preferable imaging modality due to its accessibility, cost-effectiveness, and non-ionizing nature. We have introduced a temperature monitoring method using an external ultrasound element and CNN. We have trained the model with simulation data, and applied it to ex-vivo experiments. One of the advantages of the proposed method is the fact that we can generate unlimited simulation datasets for the training. This method will be further extended for tomographic applications using sparse datasets.