Accurate Correspondence of Cone Photoreceptor Neurons in the Human Eye Using Graph Matching Applied to Longitudinal Adaptive Optics Images

  • Jianfei Liu
  • HaeWon Jung
  • Johnny TamEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10434)


Loss of cone photoreceptor neurons is a leading cause of many blinding retinal diseases. Direct visualization of these cells in the living human eye is now feasible using adaptive optics scanning light ophthalmoscopy (AOSLO). However, it remains challenging to monitor the state of specific cells across multiple visits, due to inherent eye-motion-based distortions that arise during data acquisition, artifacts when overlapping images are montaged, as well as substantial variability in the data itself. This paper presents an accurate graph matching framework that integrates (1) robust local intensity order patterns (LIOP) to describe neuron regions with illumination variation from different visits; (2) a sparse-coding based voting process to measure visual similarities of neuron pairs using LIOP descriptors; and (3) a graph matching model that combines both visual similarity and geometrical cone packing information to determine the correspondence of repeated imaging of cone photoreceptor neurons across longitudinal AOSLO datasets. The matching framework was evaluated on imaging data from ten subjects using a validation dataset created by removing 15% of the neurons from 713 neuron correspondences across image pairs. An overall matching accuracy of 98% was achieved. The framework was robust to differences in the amount of overlap between image pairs. Evaluation on a test dataset showed that the matching accuracy remained at 98% on approximately 3400 neuron correspondences, despite image quality degradation, illumination variation, large image deformation, and edge artifacts. These experimental results show that our graph matching approach can accurately identify cone photoreceptor neuron correspondences on longitudinal AOSLO images.


Adaptive optics Split detection Graph matching Sparse coding Cone photoreceptor neurons 



This research was supported by the intramural research program of the National Institutes of Health, National Eye Institute.


  1. 1.
    Dai, W., Milenkovic, O.: Subspace pursuit for compressive sensing signal reconstruction. IEEE Trans. Inf. Theory 55(5), 2230–2249 (2009)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Dubra, A., Sulai, Y.: Reflective afocal broadband adaptive optics scanning ophthalmoscope. Biomed. Opt. Express 2(6), 1757–1768 (2011)CrossRefGoogle Scholar
  3. 3.
    Dzyubachyk, O., van Cappellen, W., Essers, J., et al.: Advanced level-set-based cell tracking in time-lapse fluorescence microscopy. IEEE Trans. Med. Imaging 29(3), 852–867 (2010)CrossRefGoogle Scholar
  4. 4.
    Langlo, C., Erker, L., Parker, M., et al.: Repeatability and longitudinal assessment of foveal cone structure in CNGB3-associated achromatopsia. Retina (EPub Ahead of Print)Google Scholar
  5. 5.
    Liu, J., Dubra, A., Tam, J.: A fully automatic framework for cell segmentation on non-confocal adaptive optics images. In: SPIE Medical Imaging, p. 97852J (2016)Google Scholar
  6. 6.
    Padfield, D., Rittscher, J., Roysam, B.: Coupled minimum-cost flow cell tracking for high-throughput quantitative analysis. Med. Image Anal. 15(4), 650–668 (2011)CrossRefGoogle Scholar
  7. 7.
    Scoles, D., Sulai, Y., Langlo, C., et al.: In vivo imaging of human cone photoreceptor inner segments. Invest. Ophthalmol. Vis. Sci. 55(7), 4244–4251 (2014)CrossRefGoogle Scholar
  8. 8.
    Talcott, K., Ratnam, K., Sundquist, S., et al.: Longitudinal study of cone photoreceptors during retinal degeneration and in response to ciliary neurotrophic factor treatment. Invest. Ophthalmol. Vis. Sci. 54(7), 498–509 (2011)Google Scholar
  9. 9.
    Torresani, L., Kolmogorov, V., Rother, C.: A dual decomposition approach to feature correspondence. IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 259–271 (2013)CrossRefGoogle Scholar
  10. 10.
    Wang, Z., Fan, B., Wang, G., Wu, F.: Exploring local and overall ordinal information for robust feature description. IEEE Trans. Pattern Anal. Mach. Intell. 38(11), 2198–2211 (2016)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG (outside the US) 2017

Authors and Affiliations

  1. 1.National Eye InstituteNational Institutes of HealthBethesdaUSA

Personalised recommendations