Intra-operative Tumor Tracking Using Optical Flow and Fluorescent Imaging

  • Daniel Y. Kim
  • John H. Phan
  • May D. WangEmail author
Conference paper
Part of the IFMBE Proceedings book series (IFMBE, volume 64)


Image-guided surgery (IGS) can assist surgeons by modeling and visualizing objects of interest (tumors, nerves, etc.) that may be obstructed or difficult to recognize during surgery. Models based on pre-operative images are often not applicable during surgery because of motion and deformation. Therefore, real-time updates to IGS models are required. We propose an automated intra-operative tumor tracking system in which the initial tumor location is predicted using near infrared (NIR) fluorescence with indocyanine green (ICG), and the tumor is tracked using the Lucas-Kanade (LK) algorithm, a multi-resolution coarse-to-fine optical flow method. We simulate various conditions of intra-operative tumor movement, including movement speed and variations in image brightness. The LK method can accurately track tumors when speed of tumor movement and image brightness changes are low. However, when the speed of tumor movement increases or when image brightness changes by more than 30%, the LK method fails to track the tumor location. We compare the LK method to several other optical flow algorithms and find that LK has relatively high accuracy and tolerance in both speed and brightness changes, although each algorithm has strengths and weaknesses. In addition to the proposed intra-operative system, the simulations and metrics that we use in this study may serve as benchmarks to assess the performance of intra-operative tumor tracking algorithms.



The authors thank Chanchala Kaddi for reviewing and critiquing the manuscript. This work was supported in part by grants from National Institutes of Health (Center for Cancer Nanotechnology Excellence U54CA119338, and R01 CA163256), Georgia Cancer Coalition (Distinguished Cancer Scholar Award to Professor Wang).


  1. 1.
    T. Sielhorst, M. Feuerstein, “Advanced Medical Displays: A Literature Review of Augmented Reality,” Journal of Display Technology, vol. 4, no. 4, pp. 451–467, 2008.CrossRefGoogle Scholar
  2. 2.
    L. A. Aliperti, J. D. Predina, “Local and Systemic Recurrence is the Achilles Heel of Cancer Surgery,” Ann Surg Oncol, vol. 18, no. 3, pp. 603–607, 2011.CrossRefGoogle Scholar
  3. 3.
    J. M. Blackall, “Alignment of Sparse freehand 3-D Ultrasound with preoperative images of the liver using models of respiratory motion and demoration,” IEEE Trans. Image Process, pp. 83–90, 2005.Google Scholar
  4. 4.
    X. Qian, X. H. Peng, “In vivo tumor targeting and spectroscopic detection with surface-enhanced Raman nanoparticle tags,” Nature biotechnology, pp. 83–90, 2007.CrossRefGoogle Scholar
  5. 5.
    B. E. Schaafsma, J. S. Mieog, “The clinical use of indocyanine green as a near-infrared fluorescent contrast agent for image-guided oncologic surgery,” Journal of Surgical Oncology, vol. 104, no. 3, pp. 323–332, 2011.CrossRefGoogle Scholar
  6. 6.
    S. L. Troyan, V. Kianzad, “The FLARETM Intraoperative Near-Infrared Fluorescence Imaging System: A First-in-Human Clinical Trial in Breast Cancer Sentinel Lymph Node Mapping,” Society of Surgical Oncology, vol. 10, pp. 2943–52, 2009.CrossRefGoogle Scholar
  7. 7.
    CFR - Code of Federal Regulations Title 21 (2014, April 1). In U.S. Food and Drug Administration. Retrieved February 16, 2015, from Code of Federal Regulations (21CFR882.1480).Google Scholar
  8. 8.
    J. L. Barron, D. J. Fleet, “Systems and Experiment Performance of Optical Flow Techniques,” International Journal of Computer Vision, vol. 12, no. 1, pp. 43–77, 1994.CrossRefGoogle Scholar
  9. 9.
    B. K. P. Horn, B. G. Schunck, “Determining Optical Flow,” Artificial Intelligence, pp. 185–203, 1980.CrossRefGoogle Scholar
  10. 10.
    T. Gautama, M. Marc, “A Phase-Based Approach to the Estimation of the,” IEEE, pp. 1127–1136, 2002.Google Scholar
  11. 11.
    T. Brox, A. Bruhn, N. Papenberg, J. Weickert, “High Accuracy Optical Flow Estimation Based on a Theory for Warping,” In Proc. 8th European Conference on Computer Vision, pp. 25–36, 2004.Google Scholar
  12. 12.
    Q. Xu, R. Hamilton, “Lung tumor tracking in fluoroscopic video based on optical flow,” Medical physics, p. 5351, 2008.CrossRefGoogle Scholar
  13. 13.
    E. L. T. Amiaz, “Coarse to Over-Fine Optical Flow Estimation,” Pattern Recognition, vol. 40, no. 9, pp. 2496–2503, 2007.CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Biomedical Engineering DepartmentGeorgia Institute of Technology and Emory UniversityAtlantaUSA
  2. 2.Electrical and Computer Engineering DepartmentGeorgia Institute of TechnologyAtlantaUSA

Personalised recommendations