A robust method to track colonoscopy videos with non-informative images
- 454 Downloads
Continuously, optical and virtual image alignment can significantly supplement the clinical value of colonoscopy. However, the co-alignment process is frequently interrupted by non-informative images. A video tracking framework to continuously track optical colonoscopy images was developed and tested.
A video tracking framework with immunity to non-informative images was developed with three essential components: temporal volume flow, region flow, and incremental egomotion estimation. Temporal volume flow selects two similar images interrupted by non-informative images; region flow measures large visual motion between selected images; and incremental egomotion processing estimates significant camera motion by decomposing each large visual motion vector into a sequence of small optical flow vectors. The framework was extensively evaluated via phantom and colonoscopy image sequences. We constructed two colon-like phantoms, a straight phantom and a curved phantom, to measure actual colonoscopy motion.
In the straight phantom, after 48 frames were excluded, the tracking error was \(<\)3 mm of 16 mm traveled. In the curved phantom, the error was \(<\)4 mm of 23.88 mm traveled after 72 frames were excluded. Through evaluations with clinical sequences, the robustness of the tracking framework was demonstrated on 30 colonoscopy image sequences from 22 different patients. Four specific sequences among these were chosen to illustrate the algorithm’s decreased sensitivity to (1) fluid immersion, (2) wall contact, (3) surgery-induced colon deformation, and (4) multiple non-informative image sequences.
A robust tracking framework for real-time colonoscopy was developed that facilitates continuous alignment of optical and virtual images, immune to non-informative images that enter the video stream. The system was validated in phantom testing and achieved success with clinical image sequences.
KeywordsColonoscopy Tracking Region flow Temporal volume flow Egomotion
The authors thank the Walter Reed Army Medical Center at National Cancer Institute for providing optical colonoscopy videos and corresponding CT scans. These data were used for virtual colonoscopy study, and all subjects gave their informed consent prior to their inclusion in the study. Conflict of interest The authors declare that they have no conflict of interest.
ESM 1 (MOV 16387 kb)
- 1.NCI: colon and rectal cancer (2010) National Cancer Institute. http://www.cancer.gov/cancertopics/types/colon-and-rectal
- 5.Liu J (2011) From pixel to region to temporal volume: a robust motion processing framework for visually-guided navigation. Ph.D. thesis, University of North Carolina at CharlotteGoogle Scholar
- 6.Hwang S, Oh J, Lee J, Tavanapong W, de Groen PC, Wong J (2007) Informative frame classification for endoscopy video. Med Image Anal 11–2:110–127Google Scholar
- 8.Mori K, Deguchi D, Akiyama K, Kitasaka T, Maurer CR Jr, Suenaga Y, Takabatake H, Mori M, Natori H (2005) Hybrid bronchoscope tracking using a magnetic tracking sensor and image registration. In: Proceedings of 8th MICCAI, pp 543–555Google Scholar
- 9.Deligianni F, Chung A, Yang GZ (2006) Non-rigid 2d–3d registration with catheter tip em tracking for patient specific bronchoscope simulation. In: Proceedings of 9th MICCAI, pp 281–288Google Scholar
- 12.Bricault I, Ferretti G, Cinquin P (1998) Multi-level strategy for computer-assisted transbronchial biopsy. In: Proceedings of 1th MICCAI, pp 161–268Google Scholar
- 13.Helferty JP, Sherbondy AJ, Kiraly AP, Higgins WE (2005) System for live virtual-endoscopic guidance of bronchoscopy. In: Proceedings of IEEE CVPR, p 68Google Scholar
- 14.Helferty JP, Higgins WE (2002) Combined endoscopic video tracking and virtual 3d ct registration for surgical guidance. In: Proceedings of IEEE ICIP, pp 961–964Google Scholar
- 15.Rai L, Merritt SA, Higgins WE (2006) Real-time image-based guidance method for lung-cancer assessment. In: Proceedings of IEEE CVPR, pp 2437–2444Google Scholar
- 16.Deguchi D, Mori K, Suenaga Y, Hasegawa J, Toriwaki J, Batake HT, Natori H (2003) New image similarity measure for bronchoscope tracking based on image registration. In: Proceedings of 6th MICCAI, pp 399–406Google Scholar
- 17.Nagao J, Mori K, Enjouji T, Deguchi D (2004) Fast and accurate bronchoscope tracking using image registration and motion prediction. In: Proceedings of 7th MICCAI, pp 551–558Google Scholar
- 24.Matas J, Chum O, Urban M, Pajdla T (2002) Robust wide baseline stereo from maximally stable extremal regions. In: Proceedings of the British machine vision conference, pp 384–393Google Scholar
- 26.Liu R, Li Z, Jia J (2008) Image partial blur detection and classification. In: Proceedings of the IEEE CVPR (2008) June 27–28. Anchorage, AlaskaGoogle Scholar
- 27.Brox T, Bregler C, Malik J (2009) Large displacement optical flow. In: Proceedings of the IEEE CVPR, pp 41–48Google Scholar
- 28.Brox T, Bruhn A, Papenberg N, Weickert J (2004) High accuracy optical flow estimation based on a theory for warping. In: Proceedings of 8th ECCV, vol 4, pp 25–36Google Scholar
- 29.Brox T, Malik J (2010) Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans Pattern Anal Mach Intell 33(3):500–513Google Scholar
- 30.Lindeberg T (1993) Scale-space theory in computer vision, 1st edn. Springer, BerlinGoogle Scholar
- 31.Young DM (1971) Iterative solution of large linear systems (Computer science and applied mathematics), 1st edn. Academic Press, LondonGoogle Scholar
- 32.Tuytelaars T, Mikolajczyk K (2008) Local invariant feature detectors: a survey, 1st edn. Now Publishers Inc, HanoverGoogle Scholar
- 33.Kadir T, Zisserman A, Brady M (2004) An affine invariant salient region detector. In: Proceedings of the European conference on computer vision, pp 404–416Google Scholar
- 35.Lucas BD, Kanade T (1981) An iterative image registration technique with an application to stereo vision. In: Proceedings of international joint conference on artificial intelligence, pp 281–288Google Scholar
- 36.Papenberg N, Bruhn A, Brox T, Didas S, Weickert J (2006) Highly accurate optic flow computation with theoretically justified warping. Int J Comput Vis 67(2):141–158Google Scholar
- 37.Ryan TW (1981) The prediction of cross-correlation accuracy in digital stereo-pair images. Ph.D. thesis, University of ArizonaGoogle Scholar
- 42.Reiger J, Lawton D (1985) Processing differential image motion. J Opt Soc Am A 2(2):354–359 Google Scholar
- 43.Heeger D, Jepson A (1992) Subspace methods for recovering rigid motion 1: algorithm and implementation. Int J Comput Vis 7(2):95–117Google Scholar
- 45.LEGO-Group T (2010) Company profile: an introduction to the lego group. http://www.lego.com
- 49.Nister D, Naroditsky O, Bergen J (2004) Visual odometry. In: Proceedings of IEEE CVPR, pp 652–659Google Scholar