2D-3D Registration of CT Vertebra Volume to Fluoroscopy Projection: A Calibration Model Assessment
This study extends a previous research concerning intervertebral motion registration by means of 2D dynamic fluoroscopy to obtain a more comprehensive 3D description of vertebral kinematics. The problem of estimating the 3D rigid pose of a CT volume of a vertebra from its 2D X-ray fluoroscopy projection is addressed. 2D-3D registration is obtained maximising a measure of similarity between Digitally Reconstructed Radiographs (obtained from the CT volume) and real fluoroscopic projection. X-ray energy correction was performed. To assess the method a calibration model was realised a sheep dry vertebra was rigidly fixed to a frame of reference including metallic markers. Accurate measurement of 3D orientation was obtained via single-camera calibration of the markers and held as true 3D vertebra position; then, vertebra 3D pose was estimated and results compared. Error analysis revealed accuracy of the order of 0.1 degree for the rotation angles of about 1 mm for displacements parallel to the fluoroscopic plane, and of order of 10 mm for the orthogonal displacement.
KeywordsCalibration Model Euler Angle Fiducial Marker Mass Attenuation Coefficient Calibration Object
Intervertebral kinematics closely relates to the functionality of spinal segments and can provide useful diagnostic information. Direct measurement of the intervertebral kinematics in vivo is very problematic due to its intrinsic inaccessibility. The use of a fluoroscopic device can provide a continuous 2D screening of a specific spinal tract (e.g., cervical, lumbar) during spontaneous motion of the patient, with an acceptable, low X-ray dose. 2D kinematics can be extrapolated from fluoroscopic sequences. Most of the previous works [1, 2, 3, 4, 5, 6, 7, 8] were confined to the estimation of planar motion (most on sagittal plane) and are based on the assumption of absence of out-of-plane coupled motion (e.g., axial rotation). Coupled motion can be neglected in sagittal (flexion-extension) motion (mainly due to anatomic symmetry), but in lateral bending, where a coupled axial rotation is certainly present , this approximation is no longer valid.
The knowledge of 3D positioning (pose) of vertebrae with time can lead to full 3D kinematics analysis, or at least to evaluate the presence of out-of-plane motion (rotation). External skin markers do not provide accurate intervertebral motion description [10, 11], and invasive positioning of markers inserted in the vertebrae is not generally viable. In order to allow clinical application, 3D kinematics analysis should be performed by means of readily available, and minimally invasive, instrumentation, combined with an appropriate image processing technique.
To assess the accuracy and the repeatability of the method invitro, a calibration model consisting of a sheep dry vertebra rigidly fixed to an X-raytransparent frame of reference was designed in order to independently evaluate its 3D pose by means of a single camera calibration procedure. The errors with respect to the estimate provided by the calibration procedure were evaluated.
It is anticipated that such a method will also be helpful in a number of other contexts such as Computer Assisted Surgery (CAS) [18, 19, 20, 21, 22, 23], Radiotherapy Planning (RTP) and functional evaluation of implanted prosthesis [24, 25, 26].
2. Materials and Methods
2.1. Summary of the Methodology
In this study an in vitro assessment is performed of a method for 3D pose estimation of a vertebra with respect to a fluoroscopic imaging system, based upon the comparison between Digital Video Fluoroscopy (DVF) images and DRRs. The 3D pose estimation is a first step in a full 3D motion analysis [12, 13, 24, 25, 26].
Previous studies [4, 6, 14, 27] showed that fluoroscopy is well-suited to in vivo spine kinematics analysis because of the capability of screening patients during free motion with an acceptably low X-ray dosage and because of the wide availability of fluoroscopes in the clinical environment. However, this technique is limited to planar motion analysis and this assumption is valid only for pure flexion-extension movements. More information is needed in order to estimate 3D motion. Segmented CT volumes can provide an accurate model of a 3D vertebral shape and different X-ray attenuation of its features. Such a 3D vertebral model can be used to compute DRRs.
To estimate the 3D pose of a vertebra, the true DVF image is compared to an opportune set of differently oriented DRRs; the 3D parameters belonging to the DRR which maximise a predefined similarity index, are held as the vertebra 3D pose estimation. The similarity index employed in this study is the image zero-mean, cross-correlation coefficient. Zero-mean Normalized Cross-Correlation (ZNCC) is a widely used similarity function in template matching as well as in motion analysis. Let Open image in new window be the image under examination, of size Open image in new window by Open image in new window pixels, Open image in new window the template, of size Open image in new window by Open image in new window pixels, and Open image in new window ( Open image in new window ) be the subimage of Open image in new window at position ( Open image in new window ) having the same size as the template Open image in new window , and let 〈 Open image in new window 〉 and 〈 Open image in new window 〉 be the computed mean of Open image in new window and of Open image in new window ( Open image in new window ) respectively. The Normalized Zero-mean Normalized Cross-Correlation between the template Open image in new window and the image Open image in new window at position ( Open image in new window ) is defined as
The DRR that best matches the real DVF image is searched by means of an iterative, gradient-driven procedure.
In this work the in vivo accuracy and repeatability of the method was assessed using a calibration object. The results of the registrations between the DVF images and the CT volume were compared to an accurate 3D pose estimation obtained by means of fiducial markers embedded in the calibration object.
2.2. Reconstruction of Digital Radiographs
A few simplifications are involved: X-ray is monochromatic radiation, traversing as straight lines throughout the matter (scattering effects are neglected); x-rays source (focus) is small (ideally a point) and is positioned at a finite distance from the radiographic plane; the attenuation through the body is described by the following equation for monochromatic radiation :
where Open image in new window is the intensity of the X-ray radiation at the radiographic plane, Open image in new window is the intensity of X-ray radiation at the source, Open image in new window is the linear attenuation coefficient of the voxel at Open image in new window whose width is Open image in new window (notice that the effect of beam-hardening  is not considered). The relationship between CT-data (represented as Hounsfield Units, HU) and the corresponding attenuation coefficient Open image in new window is represented by the equation :
To reconstruct a radiograph, the radiographic plane (DRR image plane in Figure 2) is subdivided into a number of pixels, each pixel is ideally connected by a straight line with the X-ray source, and absorption coefficients in the CT volume are summated along this line (generic X-ray beam in Figure 2). Therefore the summation provides the brightness value of the radiographic pixel under examination. Three-linear interpolation  of CT data was used to estimate X-ray absorption at a generic point within the CT volume.
2.3. CT and DVF X-Rays Energy Equalisation
Typical CT scanners operate at a peak voltage of about 120 kVp. In fluoroscopy, peak voltages of 40–80 kVp are used. The energy of the X-ray photons, and therefore the Open image in new window values, are influenced by the peak voltage. The energy dependency of the attenuation coefficients is highly nonlinear . The energy corresponding to the peak of the spectrum has the strongest effect on the attenuation coefficients: we use this energy to perform the correction. The wavelength at the peak ( Open image in new window ) is approximately given by : Open image in new window , where Open image in new window is the peak voltage expressed in kVp and Open image in new window is expressed in nm. The corresponding energy can be calculated using the following formula : Open image in new window , where E is expressed in keV. Before reconstruction of DRRs, CT data must be corrected by multiplication of a correction factor Open image in new window , computed using the following formula:
where Open image in new window and Open image in new window indicate the Open image in new window of the material at DVF and at CT energy respectively; the ratio between Open image in new window and the Open image in new window (density of the material) is called the mass attenuation coefficient. Notice that the lower the peak voltage, the larger the difference between the correction factors for bone and for the other tissues. So the peak voltage is a parameter that determines the contrast between the bone and the other tissues on DVF images. Tabled mass attenuation coefficients versus x-ray energy, for several materials, can be found in Johns and Cunningham ; mass attenuation coefficients corresponding to the voltages of the actual devices used in this work are not reported in the table, and were therefore obtained by linear interpolation. To perform energy correction, two thresholds are fixed, in order to separate high density (compact) bone and low density (cancellous) bone within the CT volume: for each voxel whose Hounsfield Unit lies in the predefined range the corresponding correction factor is computed.
2.4. Estimation of the 3D Pose by Maximising the Cross-Correlation Index
CT volume of the vertebra, actual 2D fluoroscopic projection and information about CT and DVF device (e.g., DVF focal distance, pixels and voxel dimensions, X-ray KVp, etc. ) are the input data. A preliminary step to correct the CT Hounsfield numbers was performed (see the CT and DVF X-rays energy equalisation paragraph). To start the search for vertebra orientation parameters an initial range of search with a low accuracy for the 6 degrees of freedom parameters ( Open image in new window ) was chosen. A set of differently oriented DRR are then computed (for the DRR procedure refer to the Reconstruction of digital radiographs paragraph). Each DRR is then cross-correlated with the actual DVF image and a likelihood map (function of the 6 parameters) is generated. The maximum of the likelihood map is searched and the corresponding 6 parameters are considered the current best estimate of vertebra pose and orientation. The process is iterated, by refining the range of search and by increasing the accuracy of the parameters, until a desired accuracy is reached. Since the DRR algorithm is particularly time consuming, the multiscale approach (increasing accuracy) and the gradient-driven maximum search helps to reduce the total computation time.
2.5. The Calibration Model
CT data were acquired by means of a spiral CT scanner (GE 16-slice) set to 120 kVp, 100 mAs with an image resolution 0.5 mm/pixel, 0.75 mm slice thickness. The DVF image was acquired by means of a 9 inch, digital video-fluoroscopy system(GE Advantx) set to 50 kVp, 1 mAs with an image resolution of 0.3 mm/pixel. An accurate estimate of the 3D pose of the calibration model with respect to the fluoroscopic system was obtained with a single camera calibration procedure, using the eight fiducial markers [37, 38].
An intensity-weighted method was used to find the coordinates of the centroid of the fiducial markers both in the CT scan and in the DVF image [39, 40]. The markers were previously segmented using a threshold-based algorithm. For each marker the 2D coordinates of the centroid of its projection on the DVF image were measured. The centroid was calculated as the intensity-weighted average of the coordinates of the pixels belonging to the marker. Measurement error was estimated by repeating measures using various thresholds. In the CT data each marker occupied three-to-four slices. The 3D coordinates of a marker centroid were measured as an intensity-weighted average of the pixel coordinates on each slice in which the marker was visible. In order to evaluate the precision of the single camera calibration, a Monte Carlo simulation was performed adding random noise (standard deviation equal to estimated noise on measured markers of 2 mm) to measured marker coordinates and recalculating the 3D pose of the model.
An example of the 3D pose parameters of the calibration model, estimated via a single camera calibration by means of the metallic markers.
Focus Co-ordinates (mm); Translation
Euler's angles (deg); Rotations
To evaluate the accuracy and the precision of the method for 3D pose estimation, presented in the previous section, a set of trials was carefully designed.
It is assumed that an accurate simulation of the radiographic process should produce the best results in 3D pose estimation. However, the more accurate the simulation process, the larger is the computing time required. With this in mind, 4 cases were considered in order to evaluate the influence of the simulation process on the accuracy and precision of 3D pose estimation.
Preprocessing of CT data (segmentation and X-ray energy correction) was performed in the Matlab environment. To speed up DRR computation, all the software required was developed in C++.
Means and standard deviations (SD) of the errors in computing the 6 parameters of the 3D vertebral pose with respect to the reference value obtained via metallic marker camera calibration. The letters refer to the type of correction performed (see text).
The results show that, for each trial, the mean error is of approximately 0.1 degree for Euler angles and about 2 mm for coordinates parallel to the radiographic plane. The co-ordinate perpendicular to the radiographic plane, instead, is subjected to a more significant error (about 15 mm). In general errors decrease by applying the described corrections.
4. Discussion and Conclusions
This paper has described an automatic method for 3D pose estimation of a vertebra by means of CT and digital video-fluoroscopy and an in vitro assessment using a calibration object was performed.
The method is based upon a comparison (by means of cross-correlation) between DVF images and DRRs obtained using the CT 3D vertebral model. The DRR is obtained by simulating the radiographic process in an accurate manner, taking into account the effect of the difference of X-ray energy between CT and DVF imaging modalities. The algorithm for the reconstruction of digital radiographs is based on a ray-casting approach.
The 3D parameters belonging to the DRR that maximises the cross-correlation were considered as the estimators of the 3D pose of the vertebra. An iterative (gradient driven), step-refinement, multiscale approach was used to reliably estimate the absolute maxima of the cross-correlation function described by variables representing the 6 degrees of freedom of a rigid body (vertebra).
To perform an assessment of the method, a calibration model, with embedded fiducial markers, was designed in order to allow an accurate estimate of the 3D pose as a baseline. An appropriate set of trials was designed to investigate the possibility of not simulating accurately the radiographic process.
From the results it can be inferred that accuracy and precision of the 3D pose estimation increases when the simulation becomes more accurate, in particular, taking into account the effect of X-ray energies and the exponential attenuation of X-ray through matter.
The method presents satisfactory results in the computation of Euler angles and of focus coordinates parallel to the radiographic plane. However, the coordinates orthogonal to the radiographic plane are subjected to more significant errors. Nevertheless, in practical cases, this information is known via positioning of the patient with respect to the DVF device.
To explain this fact it should be observed that a displacement in the direction perpendicular to the radiographic plane causes a "zoom effect". This "zoom" is clearly visible only if the displacement is of the order of 10–15 mm. Therefore the cross-correlation index is not able to distinguish between 2 DVF images obtained with orthogonal coordinates differing less than 10–15 mm.
It is worth mentioning that by using the calibration model, the effect of soft tissue and adjacent vertebrae has been neglected. Therefore the presented results reasonably represent an upper limit for the accuracy and precision achievable in real applications. However, recent further study involving the analysis of the cervical spine motion in human patients seems to confirm the applicability of such a methodology . Current work aims to evaluate the use of first derivatives of DVF images to improve and speed up the template matching process .
- 4.von Mameren H, Drukker J, Sanches H, Beursgens J: Cervical spine motion in the sagittal plane (I) range of motion of actually performed movements, an X-ray cinematographic study. European Journal of Morphology 1990, 28(1):47-68.Google Scholar
- 7.Bifulco P, Cesarelli M, Allen R, Muggleton J, Bracale M: Automatic vertebrae recognition throughout a videofluoroscopic sequence for intervertebral kinematics study. In Time Varying Image Processing and Moving Object Recognition. Volume 4. Edited by: Cappellini. Elsevier, Amsterdam, The Netherlands; 1996:213-218.Google Scholar
- 12.Bifulco P, Cesarelli M, Roccasalva Firenze M, Verso E, Sansone M, Bracale M: Estimation of the 3D positioning of anatomic structures from radiographic projection and volume knowledge. Proceedings of the 8th Mediterranean Conference on Medical and Biological Engineering and Computing (MEDICON '98), 1998 CD-ROM: 2.5Google Scholar
- 13.Bifulco P, Cesarelli M, Verso E, Roccasalva Firenze M, Sansone M, Bracale M: Simulation of the radiography formation process from CT patient volume. Proceedings of the 8th Mediterranean Conference on Medical and Biological Engineering and Computing (MEDICON '98), 1998 CD-ROM: 2.5Google Scholar
- 19.Mac-Thiong J-M, Aubin CE, Dansereau J, de Guise JA, Brodeur P, Labelle H: Registration and geometric modelling of the spine during scoliosis surgery: a comparison study of different pre-operative reconstruction techniques and intra-operative tracking systems. Medical and Biological Engineering and Computing 1999, 37(4):445-450. 10.1007/BF02513328CrossRefGoogle Scholar
- 20.Weese J, Gocke R, Penney GP, Desmedt P, Buzug TM, Schumann H: Fast voxel-based 2D/3D registration algorithm using a volume rendering method based on the shear-warp factorization. Medical Imaging 1999: Image Processing, 1999, Proceedings of SPIE 3661: 802-810.Google Scholar
- 22.Lavalle S, Szeliski R, Bruniee L: Anatomy based registration of three-dimensional medical images, range images, X-ray projections, and three-dimensional models using octree-splines. In Computer Integrated Surgery: Technology and Clinical Applications. MIT Press, Cambridge, Mass, USA; 1995.Google Scholar
- 26.Hirokawa S, Abrar Hossain M, Kihara Y, Ariyoshi S: A 3D kinematic estimation of knee prosthesis using X-ray projection images: clinical assessment of the improved algorithm for fluoroscopy images. Medical and Biological Engineering and Computing 2008, 46(12):1253-1262. 10.1007/s11517-008-0398-8CrossRefGoogle Scholar
- 27.Bifulco P, Cesarelli M, Sansone M, Allen R, Bracale M: Fluoroscopic analysis of inter-vertebral lumbar motion: a rigid model fitting technique. Proceedings of the World Congress on Medical Physics and Biomedical Engineering, 1997 714.Google Scholar
- 31.Cho Z, Jones JP, Singh M: Foundations of Medical Imaging. Wiley Interscience, New York, NY, USA; 1993.Google Scholar
- 32.Johns HE, Cunningham JR: The Physics of Radiology. Charles C. Thomas, Springfield, Ill, USA; 1983.Google Scholar
- 34.van der Plaats GJ, Vijlbrief P: Medische Rontgentechniek in de Diagostiek. Uitgeversmaatschappij de Tijdstroom; 1978.Google Scholar
- 36.Bifulco P, Sansone M, Cesarelli M, Bracale M: In-vivo evaluation of cervical spine intervertebral kinematics by means of digital fluoroscopy: experimental set-up. Proceedings of the 2nd European Medical & Biological Engineering Conference (EMBEC '02), December 2002, Vienna, AustriaGoogle Scholar
- 41.Sansone M, Bifulco P, Cesarelli M, Bracale M: Evaluation of spine kinematics after surgical intervention: a feasibility study. Proceedings of the Medicon and Health Telematics, July-August 2004, Ischia, ItalyGoogle Scholar
- 42.Bifulco P, Cesarelli M, Romano M, Allen R, Cerciello T: Vertebrae tracking through fluoroscopic sequence: a novel approach. Proceedings of the World Congress on Medical Physics and Biomedical Engineering: The Triennial Scientific Meeting of the IUPESM, September 2009, Munich, GermanyGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.