Hard Exudates Detection Method Based on Background-Estimation

Abstract

Hard exudates (HEs) are one kind of the most important symptoms of Diabetic Retinopathy (DR). A new method based on background-estimation for hard exudates detection is presented. Firstly, through background-estimation, foreground map containing all bright objects is acquired. We use the edge information based on Kirsch operator to obtain HE candidates, and then we remove the optic disc. Finally, the shape features, histogram statistic features and phase features of the HE candidates are extracted. We use the SVM classifier to acquire the accurate extraction of HEs. The proposed method has been demonstrated on the public databases of DIARETDB1 and HEI-MED. The experiment results show that the method’s sensitivity is 97.3 % and the specificity is 90 % at the image level, and the mean sensitivity is 84.6 % and the mean predictive value is 94.4 % at the lesion level.

1 Introduction

Diabetic Retinopathy (DR) is a severe complication of diabetes mellitus and one of the most important causes of blindness. Early signs of DR include red lesions such as microaneurysms (MA), intraretinal microvascular abnormalities (IRMA) and hemorrhages (HA), bright lesions like hard exudates (HEs) and cotton-wool spots (CW). Among the early symptoms, HEs are main performance of macular edema, which appear as bright structures with well-defined edges and variable shapes (as shown in Fig. 1). Thus, the detection of HEs is very important for clinical diagnosis.

Fig. 1.

HE in fundus image and the enlarged area

HE detection methods presented in literatures can be divided into three different categories.

Mathematical morphology based methods: Welfer et al. [1] gave a method based on mathematical morphology. Firstly, contrast of image was enhanced in LUV color space. Then a set of morphological operations such as regional minima detection, morphological reconstruction and H-maxima transform were performed to detect exudates. At last, SVM classifier was used to determine the severity of lesion. Gandhi et al. [2] identified HEs after blood vessels were removed by mathematical morphology methods.

Clustering-based methods: Jayakumari et al. [3] utilized contextual clustering algorithm to divide a fundus image into background and bright areas, then identified true exudates using Echo State Neural Network.

Machine learning methods: Sánchez et al. [4] detected HEs by Fisher linear classifier. High sensitivity and specificity are achieved in test of 58 images. García et al. [5] proposed a neural network based method for exudates extraction and tested it on 67 images (27 normal images and 40 DR images). Sae-Tang et al. [6] proposed a non-uniform illumination background subtraction method. Weighted surface fitting is used in fundus image background-estimation after performing image compensation. Estimated background is subtracted from the image and then exudates are detected from foreground of image using level-set evolution without re-initialization.

Mathematical morphology based methods detected exudates according to brightness characteristic without taking other characteristics into account, so that resulting in poor robustness and high noise sensitivity. Clustering-based methods are not only too sensitive to noise, but also much heavy in computation. On the other hand, it needs initial assumption due to unknown class center location and characteristics. Machine learning methods may get high miss-detection and false-detection rate if we can’t choose suitable features and get all-sided training samples. According to the characteristics of HEs and the shortages of existing methods, this paper presents a method based on background-estimation and SVM classifier. Firstly, foreground map containing all bright objects is obtained by estimated background subtraction. After that, exudates candidates are gotten using edge information based on kirsch operator, and then optic disc is removed. Finally, the features of HE candidates are extracted and HEs are detected by SVM classifier. The block diagram of the proposed method is shown in Fig. 2.

Fig. 2.

Block diagram of the proposed method

2 HE Candidates Detection Based on Background-Estimation

HE candidates extraction includes two steps: (a) Due to high gray value of HEs, all bright targets of fundus image are extracted by estimating background; (b) According to the characteristics of HEs’ sharp edge, the edge strength based on Kirsch operator of all bright objects are calculated. Then, final exudate candidates can be screened.

2.1 Background-Estimation

Firstly, in order to improve efficiency, we resize the fundus images’ height to 750 pixels while keeping width/height ratio of each images. For extracting foreground structure of fundus images, background need to be estimated. The procedure of background-estimation is shown in Fig. 3. A median filter, whose size is 25 × 25, is applied on gray-scale image $I_{gray}$ to estimate background $I_{bg}$. In order to make background more adapted to the original gray image, morphological reconstruction [7] is used on $I_{bg}$.

Fig. 3.

Block diagram of background-estimation

Morphological reconstruction is usually used to highlight the parts of mark image which is consistent with mask image, and ignore other parts. Here, the image after median filtering is a mark image. Mask image is the image whose pixel value is the maximum of corresponding original gray-scale image pixel and median filtered image pixel. A fundus image is shown in Fig. 4(a), and the background-estimation result is shown in Fig. 4(b).

Fig. 4.

HE candidates detection. (a) Original gray-scale image; (b) result of background-estimation; (c) foreground map after background subtraction; (d) result of removing bright targets from green channel image; (e) Kirsch edge difference figure; (f) binary image of HE candidates

A difference matrix image I is obtained after subtracting the estimated background from original gray image. In the resulting image, the peak of histogram is always centered on zero. The histogram shows a clear distinction between dark and bright structures. The former represents vasculature, macula, dark lesions and other structures. Bright structures are found in the positive side of histogram and include bright lesions and other structures. The pixels whose value is below zero are set to zero so that the exudate candidate image $I_{cand}$ is obtained. However, there are still lots of false positive lesions, such as cotton wool spots and nerve fiber layer reflection. The image $I_{cand}$ is shown in Fig. 4(c).

2.2 HE Candidates Extraction Based on Kirsch Operator

Compared to the false positive lesions, HEs have sharp edges. Hence edge intensity information is used to eliminate false positive lesions whose edges are fuzzy. Because fundus images show more clear structure characteristics in green channel of RGB color space, green channel image is used for calculating edge intensity. Here edge detector is based on Kirsch operator [8] at eight different directions on $I_{g}$. Kernel outputs are combined together by selecting the maximum value on each pixel output. Kirsch operator is fitted for HE extraction due to good performance on directivity, precision and anti-noise property.

In the first step, we delineated the boundaries $I_{kirsch}$ of $I_{g}$ using Kirsch operator. To avoid influence of vessel edges, these contours are removed from $I_{kirsch}$. The elements of $I_{cand}$ are set to zero in green channel and $I_{bge}$ is obtained (as shown in Fig. 4(d)), as following,

$$ I_{bge} \left( {i,j} \right) = \left\{ {\begin{array}{*{20}l} {I_{g} \left( {i,j} \right),} \hfill & {I_{cand} \left( {i,j} \right) = 0} \hfill \\ {0,} \hfill & {I_{cand} \left( {i,j} \right) \ne 0} \hfill \\ \end{array} } \right. $$

(1)

Then, image $I_{bge}$ was performed morphological reconstruction using mask $I_{g}$. In this way, all the bright regions detected are removed from $I_{g}$. After applying Kirsch operator to the result, only the contours of blood vessels and dark elements are detected, that is $I_{0kirsch}$. The difference $I_{cankirsch}$ between $I_{kirsch}$ and $I_{0kirsch}$ is calculated, hence the contours of bright regions are obtained. The result is shown in Fig. 4(e). The edge strength [9] $\delta$ of each candidate object $\theta_{i}$ in $I_{cand}$ is defined as the mean intensity of the object in edge-enhanced image $I_{cankirsch}$, as following,

$$ \delta \left( {\theta_{i} } \right) = \frac{{\sum\limits_{{j \in\Pi _{i} }} {I_{cankirsch} \left( j \right)} }}{{\sum\limits_{{j \in\Pi _{i} }} 1 }} $$

(2)

Here, $\Pi _{i}$ is the set of pixels in the candidate object $\theta_{i}$ using 8-connected neighborhood.

A candidate is considered HE if $\delta > Th$. Th represents a parameter of the algorithm. It determines the value which is used to consider if an edge has sharp boundary. If it is chosen low, more HEs are detected, meanwhile the number of false positives increases. After experiments, when Th = 2, the result can ensure less false positives and no miss-detection, as shown in Fig. 4(f).

2.3 Optic Disk Remove

In color retinal fundus images, Optic Disk (OD) usually shows as approximately circular spot, whose color is light yellow or white. Its color and brightness characteristic are very similar to HE, which means it usually appears in HE candidates. Therefore, accurate positioning and removing of OD is necessary.

OD is positioned and removed based on main vessels extraction. Firstly, main blood vessels are extracted using morphological method, and a cross map is obtained by vascular convergence direction. Then the optimal location of OD is determined by weighted convergent optimization. OD segmentation and removal is performed utilizing Hough transform. HE candidates are obtained after OD area is removed, as shown in Fig. 5.

Fig. 5.

HE candidates after removing optic disk (area in green color) (Color figure online)

3 Accurate Extraction of HE

Most of false targets can be removed, because of HE candidates’ brightness and edge features. However, there are still some areas which are similar to true HEs on brightness level and edge shape. In this paper, shape features, histogram statistical features and phase features of HE candidate areas are utilized to distinguish true HEs from clear cotton wool spots, laser spot and serious macula reflection, and a SVM classifier is designed to complete HE accurate extraction.

3.1 Feature Extraction

(1) Shape Features.

For the easily mistaken laser spots, whose main characteristic is with regular shape of approximating circles, so the eccentricity and compactness are used to distinguish between HEs and laser spots.

Eccentricity: The ratio of long axis and short axis.

Compactness: The ratio of circumference’s square and area.

(2) Histogram Statistical Features.

HE’s grayscale distribution is concentrated, but cotton-wool spots which are easy to be mistakenly identified present the characteristic that brightness changes lower from the center to around. Histogram is a powerful tool to describe images’ grayscale distribution. Therefore, histogram statistical features are used to distinguish between cotton wool spots and HEs.

The calculation formula of image’s histogram is shown below:

$$ h(k) = \frac{{n_{k} }}{N},\quad (k = 0, 1, 2, \ldots ,L - 1) $$

(3)

where k is a variable represents gray scale levels, N is the pixel number of a gray image, n _k is the pixel number in gray scale level k, and L is the number of gray scale levels.

The utilized histogram statistical features are as follows:

• (1) Variance: the secondary moment $u_{2}$ of mean value m. Where nth moment of m is defined as:

  $$ u_{n} = \sum\limits_{k = 0}^{L - 1} {(k - m)^{n} h(k)} $$

  (4)

The mean value m is defined as:

$$ m = \sum\limits_{k = 0}^{L - 1} {kh(k)} $$

(5)

• (2) Smoothness: representation of texture depth degree of images. R is proportional to the range of regional gray value. It is defined as

  $$ R = 1 - [1 + \frac{{u_{2} }}{{(L - 1)^{2} }}]^{ - 1} $$

  (6)

• (3) Third moment: histogram skewness measure. When $u_{3} > 0$ histogram skews to the right, and when $u_{3} < 0$ histogram skews to the left.

• (4) Consistency: representation of gray value similarity. When gray values are equal in certain area, U reaches the maximum. When the difference degree increases, U decreases. It is defined as

  $$ U = \sum\limits_{k = 0}^{L - 1} {h^{2} (k)} $$

  (7)

• (5) Entropy: Random measure. It reflects the roughness of texture. The greater the randomness, the bigger of e. It is defined as

  $$ e = - \sum\limits_{k = 0}^{L - 1} {h(k)\log_{2} (h(k))} $$

  (8)

(3) Phase Features.

The features explained above are all based on amplitude. Because of its sensitiveness to illumination change, the classification results are not ideal if only amplitude features are utilized. Phase information is stabilized to illumination and contrast, and it is with good noise immunity and in accordance with human visual perception.

Phase Congruency (PC) means that the most phase consistent points in Fourier component are set as the feature points. Morrone and Owens [10] defined the PC function as:

$$ PC\left( {x,y} \right) = \mathop {\hbox{max} }\nolimits_{{\bar{\phi } \in [0,2\pi ]}} \frac{{\sum\nolimits_{n} {A_{n} \cos (\phi_{n} (x) - \bar{\phi }(x))} }}{{\sum\nolimits_{n} {A_{n} } }} $$

(9)

where A _n is amplitude of the nth harmonic cosine component. $\phi_{n} (x)$ is the Fourier series local phase in point of x. $\bar{\phi }(x)$ is the weighted average value of all Fourier items’ local phase in this point. Finding out the maximum PC point equals to finding out the least changing point of local phase.

In fact, PC is a value which is difficult to calculate. It is found in reference [11] that local energy is the product of PC and the sum of Fourier amplitude, which means local energy is proportional to PC function. The peak value of local energy corresponds to the maximum PC. Therefore, seeking maximum PC is converted to seeking local energy. In this research, the local energy is calculated by log-Gabor wavelet function.

To apply PC to HE detection in fundus image, the PC expression needs to be converted to be two-dimensional. The method is: firstly, calculate the energy E(x) of every point and compensate noise in each direction; then, sum up the values of all directions; finally, normalize the energy sum. The two-dimensional PC formula is as following

$$ PC\left( {x,y} \right) = \frac{{\sum\nolimits_{o} {\left\lfloor {E_{o} \left( {x,y} \right) - T_{o} } \right\rfloor } }}{{\sum\nolimits_{o} {\sum\nolimits_{n} {A_{no} \left( {x,y} \right) + \varepsilon } } }} $$

(10)

where E is the local energy, o means directions, n means the scale of wavelet function, $\varepsilon$ is a small positive number preventing the denominator of formula to be 0, $PC\left( {x,y} \right)$ means the PC value in point $\left( {x,y} \right)$, and T is the response of filter to noises in all scales. In order to balance the complexity of the method and the accuracy of results, in our research, o is 8, n is 3, $\varepsilon$ is 0.0001.

Features like step, roof and Mach band are always appeared in high Phase Congruency [12] points. As shown in Fig. 5, in HE candidate areas, there are another common false target called reflective of the retina, in addition to the true HE, laser spots and cotton wool spots which are removed by shape and histogram features. HE whose brightness is high and boundary is clear is obviously different with fundus image background. It is easy to identify and its local energy is bigger, so that it has high phase congruency. Reflective of the retina is similar with HEs in gray-scale, but it is fused with healthy fundus background smoothly and with no obvious step, so the phase congruency is low. After phase congruency calculation, the result of HE candidate area is shown in Fig. 6(a), and the original gray level information is shown in Fig. 6(b). It is evident that the phase information can be used to distinguish between HEs and no HEs more effectively. In order to avoid the influence of other red lesions and blood vessels, each set of pixels in HE candidates are extracted, then calculate the following statistics: mean value, median value, variance, maximum value and minimum value.

Fig. 6.

Comparison between phase information and grayscale information of HE candidates. (a) phase information; (b) grayscale information

A 12 dimension feature vector is formed by combining shape features, histogram statistical features and phase features.

3.2 SVM Classifier

According to the feature vector after feature extraction, HE candidate areas need to be divided into two classes: HE and no HE. So a two-class SVM classification model is adopted here. SVM classifier used in this method selects the Radial Basis Function (RBF) kernel, and adopts the method of cross validation to select the optimal parameters. In experiments, 510 samples of HE candidates were used for training and validation of the model, in which 345 samples were used for training and 165 samples were used for validation. Other fundus images containing HE candidates were tested with this model and accurate extraction of HEs was achieved. The final result is shown in Fig. 7.

Fig. 7.

Results of HE accurate detection

4 Experiments and Analysis

4.1 Experiment Material and Evaluation Parameters

This method is tested on public databases named DIARETDB1 [13] and HEI-MED [14]. DIARETDB1 database contains 89 color fundus images whose resolution is 1500 × 1152. Among them, 47 images are with HEs, 33 images are with cotton wool spots, hemorrhage or microaneurysm. HEI-MED database is a collection of 169 fundus images to train and test image processing algorithms for detection of exudates and diabetic macular edema.

The performance of the proposed method is measured using sensitivity (SE), specificity (SP), positive predictive value (PPV) and accuracy (AC) [15]. Their calculation formulae are as follows:

$$ SE = {{TP} \mathord{\left/ {\vphantom {{TP} {\left( {TP + FN} \right)}}} \right. \kern-0pt} {\left( {TP + FN} \right)}} $$

(11)

$$ SP = {{TN} \mathord{\left/ {\vphantom {{TN} {\left( {FP + TN} \right)}}} \right. \kern-0pt} {\left( {FP + TN} \right)}} $$

(12)

$$ PPV = {{TP} \mathord{\left/ {\vphantom {{TP} {(TP + FP)}}} \right. \kern-0pt} {(TP + FP)}} $$

(13)

$$ AC = {{\left( {TP + TN} \right)} \mathord{\left/ {\vphantom {{\left( {TP + TN} \right)} {(TP + FP + TN + FN)}}} \right. \kern-0pt} {(TP + FP + TN + FN)}} $$

(14)

where,

• TP - True Positives: Lesion regions which are correctly classified.

• FP - False Positives: Non-lesion regions which are wrongly classified as lesion regions.

• TN - True Negatives: Non-lesion regions which are correctly classified.

• FN - False Negatives: Lesion regions which are wrongly classified as non-lesion regions.

4.2 Results and Analysis

Figure 8 shows HEs detection results and comparison results with groundtruth of different fundus images. It is better to declare that the groundtruth provided in database are areas labeled by experts manually (shown as purple block areas in the 3^rd column of Fig. 8). In addition, accurate results are confirmed by experts from Tianjin Medical University Eye Hospital and Tianjin Medical University Metabolic Disease Hospital. It can be seen from Fig. 8 that whether low brightness images with more HEs (as show in 1^st and 2^nd row of Fig. 8), or high brightness images with less HEs (as shown in 3^rd row of Fig. 8), this method can detect HEs accurately.

Fig. 8.

Results of HEs detection. (a) Original images; (b) results by the proposed method; (c) contrast of results of experts labeled image

We evaluated the experimental results at the lesion level and the image level respectively. The parameter SE and PPV, which had higher discrimination, were used to evaluate the performance at lesion level, for the proportion of FP in recognition results is usually very low relative to the TN, and the parameter SP is close to 100 %, so that with no obvious discrimination. We used SE, SP and AC to evaluate the performance at image level. The comparison between our method and other typical HEs detection methods is shown in Table 1.

Table 1. Comparison of different methods

It shows that our method can achieve higher SP and AC. The method of Sánchez et al. can achieve 100 % in SE, SP and AC relies on his testing images. Although our method has lower sensitivity at lesion level, the missing detection of single HE is within tolerance for the distribution of HEs is always group by group. Our results improved the positive predictive value on the basis of getting better performance than the standard of diabetic lesion screening algorithm proposed by British Diabetic Association [16].

5 Conclusion

A HEs detection method based on background-estimation and SVM classifier is presented in this paper. Firstly, foreground map containing all bright objects is obtained by background-estimation. HEs candidates are gotten by using edge information based on Kirsch operator. Then optic disc is removed. Finally, shape features, histogram statistic features and phase features of HE candidates are extracted before using SVM classifier, so that accurate extraction of HEs is realized. Compared with other methods, this method is based on brightness information and edge information, thus HEs candidates contain all HEs and the minimum false targets. In addition, fundus images have uncertain brightness and contrast because of different photographic environment. Phase information has great noise resistance and it is stabilized in brightness and contrast. So it has obvious advantages combining with Phase Congruency in feature extraction stage. The feasibility and superiority are demonstrated by the experiments on databases of DIARETDB1 and HEI-MED.