Robust surface tracking combining features, intensity and illumination compensation
- 617 Downloads
Recovering tissue deformation during robotic-assisted minimally invasive surgery procedures is important for providing intra-operative guidance, enabling in vivo imaging modalities and enhanced robotic control. The tissue motion can also be used to apply motion stabilization and to prescribe dynamic constraints for avoiding critical anatomical structures.
Image-based methods based independently on salient features or on image intensity have limitations when dealing with homogeneous soft tissues or complex reflectance. In this paper, we use a triangular geometric mesh model in order to combine the advantages of both feature and intensity information and track the tissue surface reliably and robustly.
Synthetic and in vivo experiments are performed to provide quantitative analysis of the tracking accuracy of our method, and we also show exemplar results for registering multispectral images where there is only a weak image signal.
Compared to traditional methods, our hybrid tracking method is more robust and has improved convergence in the presence of larger displacements, tissue dynamics and illumination changes.
KeywordsNon-rigid surface tracking Multispectral imaging Minimally invasive surgery Illumination compensation
Xiaofei Du is supported by the China Scholarship Council scholarship. Neil Clancy is supported by an Imperial College Junior Research Fellowship. Shobhit Arya is supported by an NIHR-HTD 240 Grant. The authors would like to thank the Northwick Park Institute for Medical Research (NPIMR) for their assistance with surgical arrangements.
Compliance with ethical standards
Conflicts of interest
The authors declare no conflict of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all patients who were included in the study.
Supplementary material 1 (mpg 5780 KB)
Supplementary material 2 (mpg 5789 KB)
- 2.Bay H, Tuytelaars T, Van Gool L (2006) Surf: speeded up robust features. In: Computer Vision-ECCV 2006, Springer, Berlin, Heidelberg, pp 404–417Google Scholar
- 3.Bradski G (2000) The opencv library. Doctor Dobbs Journal 25(11):120–126Google Scholar
- 4.Braux-Zin J, Dupont R, Bartoli A (2013) Combining features and intensity for wide-baseline non-rigid surface registration. In: British machine vision conference (BMVC), BMVAGoogle Scholar
- 6.Delabarre B, Marchand E (2012) Visual servoing using the sum of conditional variance. In: Intelligent robots and systems (IROS), 2012 IEEE/RSJ international conference on, IEEE, pp 1689–1694Google Scholar
- 7.Fua P, Brechbühler C (1996) Imposing hard constraints on soft snakes. In: Computer vision-ECCV 1996, Springer, pp 495– 506Google Scholar
- 8.Giannarou S, Visentini-Scarzanella M, Yang GZ (2009) Affine-invariant anisotropic detector for soft tissue tracking in minimally invasive surgery. In: Biomedical imaging: from Nano to Macro, 2009. ISBI’09. IEEE international symposium on, IEEE, pp 1059–1062Google Scholar
- 10.Gröger M, Sepp W, Ortmaier T, Hirzinger G (2001) Reconstruction of image structure in presence of specular reflections. In: Pattern recognition, Springer, pp 53–60Google Scholar
- 11.Kalal Z, Mikolajczyk K, Matas J (2010) Forward–backward error: automatic detection of tracking failures. In: Pattern recognition (ICPR), 2010 20th international conference on, IEEE, pp 2756–2759Google Scholar
- 12.Keerthi SS, DeCoste D (2005) A modified finite newton method for fast solution of large scale linear svms. J Mach Learn Res 6:341–361Google Scholar
- 16.Mountney P, Yang GZ (2008) Soft tissue tracking for minimally invasive surgery: learning local deformation online. In: Medical image computing and computer-assisted intervention-MICCAI 2008, Springer, pp 364–372Google Scholar
- 17.Ortmaier TJ (2003) Motion compensation in minimally invasive robotic surgery. PhD thesis, Universität MünchenGoogle Scholar
- 18.Pickering MR, Muhit AA, Scarvell JM, Smith PN (2009) A new multi-modal similarity measure for fast gradient-based 2d–3d image registration. In: Engineering in medicine and biology society, 2009. EMBC 2009, annual international conference of the IEEE, IEEE, pp 5821–5824Google Scholar
- 20.Puerto-Souza GA, Mariottini GL (2012) Hierarchical multi-affine (hma) algorithm for fast and accurate feature matching in minimally-invasive surgical images. In: Intelligent robots and systems (IROS), 2012 IEEE/RSJ international conference on, IEEE, pp 2007–2012Google Scholar
- 21.Richa R, Sznitman R, Taylor R, Hager G (2011) Visual tracking using the sum of conditional variance. In: Intelligent robots and systems (IROS), 2011 IEEE/RSJ international conference on, IEEE, pp 2953–2958Google Scholar
- 22.Selka F, Nicolau SA, Agnus V, Bessaid A, Marescaux J, Soler L (2013) Evaluation of endoscopic image enhancement for feature tracking: a new validation framework. In: Augmented reality environments for medical imaging and computer-assisted interventions, Springer, pp 75–85Google Scholar
- 23.Stoyanov D (2012) Stereoscopic scene flow for robotic assisted minimally invasive surgery. In: Medical Image Computing and Computer-Assisted Intervention-MICCAI 2012, Springer, pp 479–486Google Scholar
- 25.Stoyanov D, Yang GZ (2005) Removing specular reflection components for robotic assisted laparoscopic surgery. In: Image processing, 2005. ICIP 2005. IEEE international conference on, IEEE, vol 3, pp III-632Google Scholar
- 26.Stoyanov D, Darzi A, Yang GZ (2004) Dense 3d depth recovery for soft tissue deformation during robotically assisted laparoscopic surgery. In: Medical image computing and computer-assisted intervention-MICCAI 2004, Springer, pp 41–48Google Scholar
- 27.Stoyanov D, Darzi A, Yang GZ (2005) A practical approach towards accurate dense 3d depth recovery for robotic laparoscopic surgery. Comput Aided Surg 10(4):199–208Google Scholar
- 28.Stoyanov D, Rayshubskiy A, Hillman E (2012) Robust registration of multispectral images of the cortical surface in neurosurgery. In: Biomedical imaging (ISBI), 2012 9th IEEE international symposium on, IEEE, pp 1643–1646Google Scholar
- 29.Yip MC, Lowe DG, Salcudean SE, Rohling RN, Nguan CY (2012) Real-time methods for long-term tissue feature tracking in endoscopic scenes. In: Information processing in computer-assisted interventions, Springer, pp 33–43Google Scholar