Wide-Baseline Dense Feature Matching for Endoscopic Images

  • Gustavo A. Puerto-Souza
  • Gian-Luca Mariottini
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8333)

Abstract

Providing a feature-matching strategy to accurately recover tracked features after a fast and large endoscopic-camera motion or a strong organ deformation, is key in many endoscopic-imaging applications, such as augmented reality or soft-tissue shape recovery. Despite recent advances, existing feature-matching algorithms are characterized by limiting assumptions, and have not yet met the necessary levels of accuracy, especially when used to recover features in distorted or poorly-textured tissue areas. In this paper, we present a novel feature-matching algorithm that accurately recovers the position of image features over the entire organ’s surface. Our method is fully automatic, it does not require any explicit assumption about the organ’s 3-D surface, and leverages Gaussian Process Regression to incorporate noisy matches in a probabilistically sound way. We have conducted extensive tests with a large database of more than 100 endoscopic-image pairs, which show the improved accuracy and robustness of our approach when compared to current state-of-the-art methods.

Keywords

Query Image Correct Match Laparoscopic Partial Nephrectomy Reprojection Error Coherent Point Drift 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Download to read the full conference paper text

References

  1. 1.
    Shi, J., Tomasi, C.: Good features to track. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 593–600 (June 1994)Google Scholar
  2. 2.
    Cohen, D., Mayer, E., Chen, D., Anstee, A., Vale, J., Yang, G.Z., Darzi, A., Edwards, P.: Augmented reality image guidance in minimally invasive prostatectomy. In: Prostate Cancer Imaging. Computer-Aided Diagnosis, Prognosis, and Intervention, pp. 101–110 (2010)Google Scholar
  3. 3.
    Su, L.M., Vagvolgyi, B.P., Agarwal, R., Reiley, C.E., Taylor, R.H., Hager, G.D.: Augmented reality during robot-assisted laparoscopic partial nephrectomy: Toward real-time 3D-CT to stereoscopic video registration. Urology 73(4), 896–900 (2009)Google Scholar
  4. 4.
    Higgins, E.W., Helferty, P.J., Lu, K., Merritt, A.S., Lav, R., Kun-Chang, Y.: 3d ct-video fusion for image-guided bronchoscopy. Computerized Medical Imaging and Graphics 32(3), 159–173 (2008)Google Scholar
  5. 5.
    Mountney, P., Yang, G.-Z.: Motion compensated SLAM for image guided surgery. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010, Part II. LNCS, vol. 6362, pp. 496–504. Springer, Heidelberg (2010)Google Scholar
  6. 6.
    Mountney, P., Stoyanov, D., Yang, G.Z.: Three-dimensional tissue deformation recovery and tracking. IEEE Signal Processing Magazine 27(4), 14–24 (2010)Google Scholar
  7. 7.
    Hu, M., Penney, G.P., Rueckert, D., Edwards, P.J., Bello, F., Casula, R., Figl, M., Hawkes, D.J.: Non-rigid reconstruction of the beating heart surface for minimally invasive cardiac surgery. In: Proc. of the 12th Int. Conf. on Med. Image Comp. and Comp.-Ass. Int., pp. 34–42 (2009)Google Scholar
  8. 8.
    Lo, B.P.L., Visentini-Scarzanella, M., Stoyanov, D., Yang, G.Z.: Belief propagation for depth cue fusion in minimally invasive surgery. In: Proc. of the 11th Int. Conf. on Med. Image Comp. and Comp.-Ass. Int, pp. 104–112 (2008)Google Scholar
  9. 9.
    Visentini-Scarzanella, M., Mylonas, G.P., Stoyanov, D., Yang, G.Z.: i-brush: A gaze-contingent virtual paintbrush for dense 3d reconstruction in robotic assisted surgery. In: Proc. of the 12th Int. Conf. on Med. Image Comp. and Comp.-Ass. Int., pp. 353–360 (2009)Google Scholar
  10. 10.
    Totz, J., Mountney, P., Stoyanov, D., Yang, G.Z.: Dense surface reconstruction for enhanced navigation in MIS. In: Proc. of the 14th Int. Conf. on Med. Image Comp. and Comp.-Ass. Int., pp. 89–96 (2011)Google Scholar
  11. 11.
    Lepetit, V., Fua, P.: Monocular model-based 3-d tracking of rigid objects: A survey. Foundations and Trends in Computer Graphics and Vision 1, 1–89 (2005)Google Scholar
  12. 12.
    Puerto-Souza, G.A., Mariottini, G.L.: A comparative study of correspondence-search algorithms in mis images. In: Proc. of the 15th Int. Conf. on Med. Image Comp. and Comp.-Ass. Int., pp. 625–633 (2012)Google Scholar
  13. 13.
    Stoyanov, D., Mylonas, G., Deligianni, F., Darzi, A., Yang, G.Z.: Soft-tissue motion tracking and structure estimation for robotic assisted mis procedures. In: Proc. of the 8th Int. Conf. on Med. Image Comp. and Comp.-Ass. Int., pp. 139–146 (2005)Google Scholar
  14. 14.
    Richa, R., Bo, A.P., Poignet, P.: Towards robust 3d visual tracking for motion compensation in beating heart surgery. Medical Image Analysis 15(3), 3012–3315 (2010)Google Scholar
  15. 15.
    Giannarou, S., Visentini-Scarzanella, M., Yang, G.Z.: Probabilistic tracking of affine-invariant anisotropic regions. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (2012)Google Scholar
  16. 16.
    Yip, M., Lowe, D., Salcudean, S., Rohling, R., Nguan, C.: Real-time methods for long-term tissue feature tracking in endoscopic scenes. In: Information Processing in Computer-Assisted Interventions, pp. 33–43 (2012)Google Scholar
  17. 17.
    Tola, E., Lepetit, V., Fua, P.: Daisy: An efficient dense descriptor applied to wide-baseline stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(5), 815–830 (2010)Google Scholar
  18. 18.
    Puerto-Souza, G.A., Mariottini, G.L.: Hierarchical multi-affine (HMA) algorithm for fast and accurate feature matching in minimally-invasive surgical images. In: Proc. IEEE/RSJ Int. Conf. Intel. Robots Syst., pp. 2007–2012 (October 2012)Google Scholar
  19. 19.
    Del Bimbo, A., Franco, F., Pernici, F.: Local shape estimation from a single keypoint. In: Proc. Comp. Vis. Patt. Rec. Workshops, pp. 23–28 (2010)Google Scholar
  20. 20.
    Cho, M., Lee, J., Lee, K.M.: Feature correspondence and deformable object matching via agglomerative correspondence clustering. In: Proc. 9th Int. Conf. Comp. Vis., pp. 1280–1287 (2009)Google Scholar
  21. 21.
    Puerto-Souza, G.A., Mariottini, G.L.: A Fast and Accurate Feature-Matching Algorithm for Minimally-Invasive Endoscopic Images. IEEE Transactions on Medical Imaging (in Press, 2013)Google Scholar
  22. 22.
    HMA feature-matching toolbox (Web), http://ranger.uta.edu/~gianluca/feature_matching/
  23. 23.
    Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge Univ. Press (2000)Google Scholar
  24. 24.
    Pilet, J., Lepetit, V., Fua, P.: Fast Non-Rigid Surface Detection, Registration and Realistic Augmentation. International Journal of Computer Vision 76(2) (2008)Google Scholar
  25. 25.
    Zhu, J., Hoi, S., Lyu, L.: Nonrigid shape recovery by gaussian process regression. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1319–1326 (2009)Google Scholar
  26. 26.
    Belongie, S., Malik, J., Puzicha, J.: Shape Matching and Object Recognition Using Shape Contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 509–522 (2001)Google Scholar
  27. 27.
    Myronenko, A., Song, X.: Point Set Registration: Coherent Point Drift. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(12), 2262–2275 (2010)Google Scholar
  28. 28.
    Pizarro, D., Bartoli, A.: Feature-based deformable surface detection with self-occlusion reasoning. International Journal of Computer Vision 97(1), 54–70 (2012)Google Scholar
  29. 29.
    Kim, J.-H., Bartoli, A., Collins, T., Hartley, R.: Tracking by detection for interactive image augmentation in laparoscopy. In: Dawant, B.M., Christensen, G.E., Fitzpatrick, J.M., Rueckert, D. (eds.) WBIR 2012. LNCS, vol. 7359, pp. 246–255. Springer, Heidelberg (2012)Google Scholar
  30. 30.
    Rasmussen, C., Williams, C.: Gaussian processes for machine learning, vol. 1. MIT press, Cambridge (2006)Google Scholar
  31. 31.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(10), 1615–1630 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Gustavo A. Puerto-Souza
    • 1
  • Gian-Luca Mariottini
    • 1
  1. 1.Dept. of Computer Science and EngineeringUniv. of Texas at ArlingtonTexasUSA

Personalised recommendations