Online Scene Association for Endoscopic Navigation

  • Menglong Ye
  • Edward Johns
  • Stamatia Giannarou
  • Guang-Zhong Yang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8674)


Endoscopic surveillance is a widely used method for monitoring abnormal changes in the gastrointestinal tract such as Barrett’s esophagus. Direct visual assessment, however, is both time consuming and error prone, as it involves manual labelling of abnormalities on a large set of images. To assist surveillance, this paper proposes an online scene association scheme to summarise an endoscopic video into scenes, on-the-fly. This provides scene clustering based on visual contents, and also facilitates topological localisation during navigation. The proposed method is based on tracking and detection of visual landmarks on the tissue surface. A generative model is proposed for online learning of pairwise geometrical relationships between landmarks. This enables robust detection of landmarks and scene association under tissue deformation. Detailed experimental comparison and validation have been conducted on in vivo endoscopic videos to demonstrate the practical value of our approach.


Online Learning Average Precision Near Neighbour Query Image Feature Tracking 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Ye, M., Giannarou, S., Patel, N., Teare, J., Yang, G.Z.: Pathological site retargeting under tissue deformation using geometrical association and tracking. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part II. LNCS, vol. 8150, pp. 67–74. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  2. 2.
    Atasoy, S., Mateus, D., Meining, A., Yang, G., Navab, N.: Endoscopic video manifolds for targeted optical biopsy. IEEE Trans. Med. Imag. 31(3), 637–653 (2012)CrossRefzbMATHGoogle Scholar
  3. 3.
    Kwitt, R., Vasconcelos, N., Rasiwasia, N., Uhl, A., Davis, B., Häfner, M., Wrba, F.: Endoscopic image analysis in semantic space. Med. Image Anal. 16(7), 1415–1422 (2012)CrossRefzbMATHGoogle Scholar
  4. 4.
    André, B., Vercauteren, T., Buchner, A.M., Wallace, M.B., Ayache, N.: A smart atlas for endomicroscopy using automated video retrieval. Med. Image Anal. 15(4), 460–476 (2011)CrossRefGoogle Scholar
  5. 5.
    Johns, E., Yang, G.Z.: From images to scenes: Compressing an image cluster into a single scene model for place recognition. In: ICCV, pp. 874–881 (2011)Google Scholar
  6. 6.
    Johns, E., Yang, G.Z.: Generative methods for long-term place recognition in dynamic scenes. Int. J. Comput. Vision 106(3), 297–314 (2014)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  8. 8.
    Kalal, Z., Mikolajczyk, K., Matas, J.: Forward-backward error: Automatic detection of tracking failures. In: ICPR, pp. 2756–2759 (2010)Google Scholar
  9. 9.
    Kalal, Z., Mikolajczyk, K., Matas, J.: Tracking-learning-detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1409–1422 (2012)CrossRefGoogle Scholar
  10. 10.
    Ozuysal, M., Calonder, M., Lepetit, V., Fua, P.: Fast keypoint recognition using random ferns. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 448–461 (2010)CrossRefGoogle Scholar
  11. 11.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. 2. Cambridge University Press (2004)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Menglong Ye
    • 1
  • Edward Johns
    • 2
  • Stamatia Giannarou
    • 1
  • Guang-Zhong Yang
    • 1
  1. 1.The Hamlyn Centre for Robotic SurgeryImperial College LondonUK
  2. 2.Department of Computer ScienceUniversity College LondonUK

Personalised recommendations