Semi-automated Query Construction for Content-Based Endomicroscopy Video Retrieval

  • Marzieh Kohandani Tafresh
  • Nicolas Linard
  • Barbara André
  • Nicholas Ayache
  • Tom Vercauteren
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8673)


Content-based video retrieval has shown promising results to help physicians in their interpretation of medical videos in general and endomicroscopic ones in particular. Defining a relevant query for CBVR can however be a complex and time-consuming task for non-expert and even expert users. Indeed, uncut endomicroscopy videos may very well contain images corresponding to a variety of different tissue types. Using such uncut videos as queries may lead to drastic performance degradations for the system. In this study, we propose a semi-automated methodology that allows the physician to create meaningful and relevant queries in a simple and efficient manner. We believe that this will lead to more reproducible and more consistent results. The validation of our method is divided into two approaches. The first one is an indirect validation based on per video classification results with histopathological ground-truth. The second one is more direct and relies on perceived inter-video visual similarity ground-truth. We demonstrate that our proposed method significantly outperforms the approach with uncut videos and approaches the performance of a tedious manual query construction by an expert. Finally, we show that the similarity perceived between videos by experts is significantly correlated with the inter-video similarity distance computed by our retrieval system.


Scale Invariant Feature Transform Video Retrieval Confocal Laser Endomicroscopy Temporal Segmentation Scale Invariant Feature Transform Descriptor 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    André, B., Vercauteren, T., Buchner, A.M., Wallace, M.B., Ayache, N.: Retrieval evaluation and distance learning from perceived similarity between endomicroscopy videos, 289–296 (2011)Google Scholar
  2. 2.
    André, B., Vercauteren, T., Buchner, A.M., Wallace, M.B., Ayache, N.: A smart atlas for endomicroscopy using automated video retrieval 15(4), 460–476 (2011)Google Scholar
  3. 3.
    André, B., Vercauteren, T., Buchner, A.M., Wallace, M.B., Ayache, N.: Learning semantic and visual similarity for endomicroscopy video retrieval 31(6), 1276–1288 (2012)Google Scholar
  4. 4.
    Buchner, A.M., Shahid, M.W., Heckman, M.G., Krishna, M., Ghabril, M., Hasan, M., Crook, J.E., Gomez, V., Raimondo, M., Woodward, T., Wolfsen, H.C., Wallace, M.B.: Comparison of probe-based confocal laser endomicroscopy with virtual chromoendoscopy for classification of colon polyps 138(3), 834–842 (2010)Google Scholar
  5. 5.
    Cooper, M., Liu, T., Rieffel, E.: Video segmentation via temporal pattern classification 9(3), 610–618 (2007)Google Scholar
  6. 6.
    Gargi, U., Kasturi, R., Strayer, S.H.: Performance characterization of video-shot-change detection methods 10(1), 1–13 (2000)Google Scholar
  7. 7.
    Hu, W., Xie, N., Li, L., Zeng, X., Maybank, S.: A survey on visual content-based video indexing and retrieval 41(6), 797–819 (2011)Google Scholar
  8. 8.
    Li, B., Meng, M.H.: Capsule endoscopy video boundary detection, 373–78 (June 2011)Google Scholar
  9. 9.
    Liu, C., Yuen, J., Torralba, A.: SIFT flow: Dense correspondence across scenes and its applications 33(5), 978–994 (2011)Google Scholar
  10. 10.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints 60, 91–110 (2004)Google Scholar
  11. 11.
    Oh, J., Hwang, S., Lee, J., Tavanapong, W., Wong, J., de Groen, P.C.: Informative frame classification for endoscopy video 11(2), 110–127 (2007)Google Scholar
  12. 12.
    Sun, Z., Li, B., Zhou, R., Zheng, H., Meng, M.H.: Removal of non-informative frames for wireless capsule endoscopy video segmentation, 294–299 (August 2012)Google Scholar
  13. 13.
    Vercauteren, T., Meining, A., Lacombe, F., Perchant, A.: Real time autonomous video image registration for endomicroscopy: Fighting the compromises. In: Conchello, J.A., Cogswell, C.J., Wilson, T. (eds.) Proc. SPIE BIOS - Three-Dimensional and Multidimensional Microscopy: Image Acquisition and Processing XV. SPIE, San Jose (2008)Google Scholar
  14. 14.
    Wang, H., Kläser, A., Schmid, C., Liu, C.L.: Dense trajectories and motion boundary descriptors for action recognition 103(1), 60–79 (2013)Google Scholar
  15. 15.
    Zhang, J., Lazebnik, S., Schmid, C.: Local features and kernels for classification of texture and object categories: a comprehensive study 73, 213–238 (2007)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Marzieh Kohandani Tafresh
    • 1
    • 2
  • Nicolas Linard
    • 2
  • Barbara André
    • 2
  • Nicholas Ayache
    • 1
  • Tom Vercauteren
    • 2
  1. 1.Inria Asclepios Project-TeamSophia AntipolisFrance
  2. 2.Mauna Kea TechnologiesParisFrance

Personalised recommendations