Robust Crack Defect Detection in Inhomogeneously Textured Surface of Near Infrared Images

Abstract

Robust crack defect detection in solar cells has been challenging because of the inhomogeneously textured surface, low contrast between crack defect and background, the diversity of crack types, and so on. To overcome these challenges, this paper presents a new robust crack defect detection scheme for multicrystalline solar cells. Firstly, a steerable evidence filter is designed to process EL image to obtain the response map, which enhances the contrast between crack and background and provides evidence for the presence of crack defect. Secondly, complete crack extraction from the response map is employed. Finally, the complete crack can be located in the inspection image by the crack skeleton extraction. Experimental results on defective and defect-free EL images show that the proposed scheme is robust, and various cracks can be effectively detected, which outperforms the previous methods.

1 Introduction

Solar cells are the critical component of solar power system, which can convert solar energy into electricity. However, the crystal structure of multicrystalline solar cells is fragile, so crack defects are inevitably generated in many fabrication and installation links such as wire sawing, pick and place, transmission, collision, etc. [1]. The presence of crack defect will greatly reduce the power generation efficiency of solar cells and usable lifetime of photovoltaic modules [2]. Thus, the quality control of solar cells is a crucial aspect for solar power system. In the past, crack defects inspection relied on experienced technicians to identify the presence of crack from collected solar cells images. However, human inspection fails to meet the requirement of rapidity, reliability, and robustness for mass production of solar cells in industrial scenes. Hence, the requirement of automatic crack detection of solar cells is demanding.

To capture efficacious crack information inside the wafer surface, we obtain the near infrared images of solar cells with a wavelength of 950 nm–1250 nm using the electroluminescence (EL) imaging technique [3, 4]. Compared with the defect-free regions, crack defects appear as dark characteristic with curvilinear and complicated geometry structures in the EL images. The difficulties of crack detection in EL images mainly are (1) the inhomogeneously textured surface of near infrared images; (2) low contrast between crack defect and surrounding background including random crystal grains; and (3) the diversity of crack types.

Figure 1(a) is a defect-free EL image and there are crystal grains with random shapes, sizes, positions and orientations forming the inhomogeneously textured surface. Specially, as shown in the red label frames of Fig. 1(b)–(c), the cracks are submerged by the randomly distributed crystal grains. Moreover, the crystal grains appearing as curvilinear shapes in the green dotted label frames are easy to be mistaken as crack. Therefore, robust crack defect detection has always been a challenging task in the field of defect detection.

Fig. 1.

Challenges of crack defect detection in solar cells. (a) Defect-free EL image. (b)–(c) Crack defective EL images with randomly distributed crystal grains.

Many computer vision-based methods have focused on the crack defect detection in road, bridge, and solar cells surfaces. Most of spatial methods were used by comparing the pixel intensities difference between crack defect and background. [5, 6] used intensity information for pavement crack detection, but the illumination and texture would affect the crack segmentation performance. For crack detection in solar cells, Tsai et al. [7] applied an anisotropic diffusion scheme and took the gray level and gradient features to adjust the diffusion coefficient. Chiou et al. [8] proposed a local thresholding-based crack extraction method. However, for these methods, the intensity information is the major consideration, but the intensity of crystal grains is very similar to crack. So, the crack defects cannot be correctly detected.

Spectral methods mainly use a set of filters to process the images and obtain the features that can distinguish the defect from background based the filter response. [9,10,11] used the Gabor filter to detect pavement cracks. Tsai [12] described crack defects as line-shaped and removed them by setting the corresponding frequency components to zero. However, these methods are applied to detect longitudinal or transverse cracks and the cracks with complicated geometry do not get involved.

Although the current research methods have achieved certain results, robust crack defect detection in solar cells is still challenging for the following reasons. (1) current methods fail to extract crack defect from the inhomogeneously textured surface, leading to the misdetection of some crystal grains as crack defect; and (2) most of current methods are proposed to detect line-shape cracks, so dendritic cracks with bends and bifurcations are not well solved. In this paper, we try to propose a robust crack detection scheme that can solve the above challenging problems.

The remaining part of this paper is organized as follows. In Sect. 2, a novel robust crack defect detection scheme in solar cells is described concretely. Section 3 presents the experimental results on defective and defect-free EL images. Finally, Sect. 4 gives the conclusion.

2 Crack Defect Detection Scheme

The overall scheme of the proposed crack defect detection method is shown in Fig. 2. The steerable evidence filter is designed, which contains a basic steerable filter and two additional oriented filters including a certain offset in angle and space distance. Firstly, a steerable evidence filter is used to process EL image to obtain the response map, which enhances the contrast between crack and background and provides evidence for the presence of crack defect. Secondly, complete crack extraction from the response map is employed. In this procedure, a local threshold based on sliding sub-image is applied to segment crack defect. Then, the morphology operation is used to remove some isolated non-crack pixels and minimum spanning tree is used to connect crack fragments. Finally, the complete crack can be located in the inspection image by using the skeleton extraction.

Fig. 2.

Overall scheme of the proposed method.

2.1 Steerable Evidence Filter Design and Its Response Map

The steerable filters derived from a linear combination of basic filter with arbitrary orientations [13]. In this study, the basic filter generates from the Hessian matrix represents as $H\left( p \right)$, which is a square matrix and describes the local curvature of the function. For a 2-D image $f\left( p \right)$, at a position $p_{0} = \left( {x,y} \right)$, the hessian function can be obtained by Eq. (1).

$$ H\left( {p_{0} } \right) = \left( {\begin{array}{*{20}c} {g_{xx} \left( {p_{0} } \right)} & {g_{xy} \left( {p_{0} } \right)} \\ {g_{xy} \left( {p_{0} } \right)} & {g_{yy} \left( {p_{0} } \right)} \\ \end{array} } \right) *f\left( {p_{0} } \right) $$

(1)

Equation (2) gives the Gaussian kernel with variance $\sigma$ and its corresponding second derivatives form part of the hessian function.

$$ g\left( {p_{0} ;\sigma } \right) = \frac{1}{{\sqrt {2\pi } \sigma }}e^{{ - \frac{{x^{2} + y^{2} }}{{2\sigma^{2} }}}} $$

(2)

Considering the orientation $u_{\theta } = \left( {\cos \theta ,\sin \theta } \right)^{T}$, the basic steerable filter with $\theta \in \left[ { - \pi /2,\pi /2} \right]$ can be obtained by Eq. (3). To put it in another way, its detailed expression is shown in Eq. (4).

$$ e\left( {p_{0} ,\theta ;\sigma } \right) = u_{\theta }^{T} H\left( {p_{0} } \right)u_{\theta } $$

(3)

$$ e\left( {p_{0} ,\theta ;\sigma } \right) = g_{xx} cos^{2} \theta + g_{yy} sin^{2} \theta + g_{xy} \,sin\,2\theta $$

(4)

After the basic steerable filter $e$ convolutes an image $f\left( p \right)$, the filter response $E$ at a position $p_{0}$ can be calculated by Eq. (5). In addition, the parameter $\sigma$ in the steerable basic filter can be fixed to adapt different widths. As shown in Fig. 3(a), the first row shows three basic steerable filters with $\theta = 0, - \pi /4, \pi /3$, respectively.

Fig. 3.

(a) Examples of the basic steerable filters in the first row and steerable evidence filters in the second row. (b) The local search region of steerable evidence filter.

$$ E\left( {p_{0} ,\theta ;\sigma } \right) = e\left( {p_{0} ,\theta ;\sigma } \right) *f\left( {p_{0} } \right) $$

(5)

However, the single basic steerable filter is incapable of detecting sharp bends, intensity variations, and completed crack morphology. Thus, inspired by the local directional evidence filtering [14], we design two additional oriented filters that include a certain offset in the angle and space distance of the detection point. Equations (6)–(7) are the corresponding two offset point $p_{1}$ and $p_{2}$.

Figure 3(b) is the demonstration of the local search region around the detection point $p_{0} \left( {x,y} \right) = \left( {1,1} \right)$, and the parameter settings are $d = 1$, $\theta \in \left[ { - \pi /2,\pi /2} \right]$ and $\varphi \in \left[ { - \pi /6,\pi /6} \right]$ with step of $\pi /18$. The yellow point represents the detection point $p_{0}$, and the red points represent the $p_{1}$, and $p_{2}$ are displayed with green points, all of which appear as a circular search region with the center of detection point $p_{0}$ and the radius of $d$. Specially, the partial overlap of the red points and green points, which ensures the detection of crack defect at the entire circular region of the detection point, is caused by the offset angle $\varphi$ and the offset distance $d$.

$$ p_{1} = \left[ {x - d\,cos\left( {\theta + \varphi } \right), y + sin\left( {\theta + \varphi } \right)} \right] $$

(6)

$$ p_{2} = \left[ {x + d\,cos\left( {\theta + \varphi } \right), y - sin\left( {\theta + \varphi } \right)} \right] $$

(7)

$$ e^{*} \left( {p;\sigma ,\theta ,\varphi } \right) = e\left( {p_{0} ;\sigma ,\theta } \right) + e\left( {p_{1} ;\sigma ,\theta + \varphi } \right) + e\left( {p_{1} ;\sigma ,\theta + \varphi } \right) $$

(8)

Thus, a linear superposition of the basic steerable filter and two additional oriented filters form the steerable evidence filter $e^{*}$ in Eq. (8), which provide evidence for the presence of crack defect. As shown in Fig. 3(a), the second row shows three sets steerable evidence filter with different orientations. It shows the steerable evidence filter is consistent with the geometric characteristics of the crack defect fragments including line and curve, so the problem of crack defect bending, bifurcation and complicated geometry can be addressed.

In our application, we aim at finding the crack that lies in some uncertain position and orientation in an EL image. When convoluting an image with the steerable evidence filter in Eq. (9), a higher magnitude will be obtained if there is a crack in the EL image. To better highlight the crack information, the maximum response magnitudes $R^{*}$ are calculated by Eq. (10). Figure 4(a) presents two defective images and Fig. 4(b) shows the response maps obtained by the steerable evidence filter. The response maps show the response magnitudes can be approximately treated as zero in the defect-free region. In contrast, the crack region has higher response magnitudes, which contributes to the further crack defect acquirement.

Fig. 4.

Example of the response maps. (a) Defective images. (b) The response maps obtained by the steerable evidence filter.

$$ R\left( {p;\sigma ,\theta ,\varphi } \right) = e^{*} \left( {p;\sigma ,\theta ,\varphi } \right) *f\left( p \right) $$

(9)

$$ R^{ *} \left( {p;\sigma } \right) = \hbox{max} {}_{\theta ,\varphi }R\left( {p;\sigma ,\theta ,\varphi } \right) $$

(10)

2.2 Complete Crack Extraction from the Response Map

In this section, we mainly extract the complete crack defect from the response map, which is implemented by local threshold segmentation, connecting crack fragments by minimum spanning tree and locating complete crack defect by crack skeleton extraction.

2.2.1 Local Threshold Segmentation

Owing to the random crystal grains, the defect-free region also represents certain response magnitudes in the response map. In order to obtain the crack defect structure, we first apply a local threshold by sliding sub-image to segment crack from the response map. Then, the threshold for the response map is given by

$$ T\left( {x,y} \right) = u_{R} \left( {x,y} \right) + k\sigma_{R} \left( {x,y} \right) $$

(11)

where $u_{R} \left( {x,y} \right)$ and $\sigma_{R} \left( {x,y} \right)$ are the mean and standard deviation of each sliding sub-image of size $N \times N$ in the response map, and the size of mask is set $9 \times 9$ in our application. $k$ is a predetermined constant.

We still select this EL images with crystal grains around the crack, and the corresponding segmentation results are given by Eq. (12). Although some background pixels are removed after the threshold, there are still some crystal grains structures in the threshold segmentation results, so morphological operation is adopted to remove some small non-crack pixels and the final crack defect fragments are shown in Fig. 5(a). For these EL images with crystal grains interference, the segmented crack is disconnected near the crystal grains, which makes crack defect incomplete and is unfavorable to detection performance.

Fig. 5.

Complete crack extraction by minimum spanning tree. (a) Crack fragments. (b) Complete crack. (c) Located crack. (Color figure online)

$$ B\left( {x,y} \right) = \left\{ {\begin{array}{*{20}l} {1,} \hfill & {if\, R\left( {x,y} \right) > T\left( {x,y} \right)} \hfill \\ {0,} \hfill & {otherwise} \hfill \\ \end{array} } \right. $$

(12)

2.2.2 Connecting Crack Fragments and Locating Complete Crack Defect

To get the complete crack defect, minimum spanning tree (MST) based on Kruskal’s algorithm is applied. Minimum spanning tree is an important model in graph theory and it is used to solve the problem of minimum path cost. In this procedure, minimum spanning tree can minimize the sum of the weights of corresponding edges and connect these nodes. According to this function, we use minimum spanning tree to connect crack fragments. Figure 5(a) shows the nodes of a1 and b1, c1 and d1 belonging to the vertexes of crack fragments to be connected, and Fig. 5(b) shows the complete crack. In order to mark the crack defects, we compute the crack skeleton and locate them in the inspection images. As shown in Fig. 5(c), the complete crack defects are correctly located as red points.

3 Experimental Results and Analysis

3.1 Data Set and Parameter Setting

In order to verify the performance of the proposed method, we collect the defective and defect-free EL images from the actual production line in. The defect-free EL images have no crack defect but crystal grains randomly distributed in the background. The defective EL images can be divided into five types according to the texture background and geometric property of crack defect. Some representative defective and defect-free EL images are shown in Fig. 6. Here, 300 defective EL images including different types of cracks and 200 defect-free EL images of size $125 \times 125$ are used to verify the proposed method.

Fig. 6.

Representative defective images and defect-free images. (a1)–(a2) Pure-type cracks. (b1)–(b2) Submerged-type cracks. (c1)–(c2) Dendritic-type cracks. (d1)–(d2) Break. (e1)–(e2) Dark-type cracks. (f1)–(f2) Defect-free.

Although there are several parameters in the proposed method, not all of them play important roles in crack detection performance. Three parameter are essential: scale size $\sigma$, offset $d$ of the steerable evidence filter and threshold coefficient $k$. So, we select a certain number of defective and defect-free EL images to determine the optimal parameter values. The detailed characterizations of the data set used in the experiment are presented in Table 1.

Table 1. Details of data set used in the experiment

In the designed steerable evidence filtering procedure, we have to determine two parameter for better performance: the scale size $\sigma$ and the offset $d$. The parameter $\sigma$ fits different widths of crack defects. If $\sigma$ is set too small, the crack cannot be well highlight and it is easy to be missed. Conversely, if $\sigma$ is set too large, the crystal grains near the crack will be treated as crack, and affect the detection performance. As shown in Fig. 7(a), for different types of defective and defect-free images, a better detection results can be obtained when the scale size $\sigma$ is 1. Similarly, the offset $d$ controls the locality of the steerable evidence filter. While a small value of $d$ does not contribute enough, a large value will introduce false pixels that do not belong to the same crack structure. According to Fig. 7(b), the optimal performance is achieved when $d$ is 1.

Fig. 7.

Optimal parameter selection. (a) Effect of scale size. (b) Effect of offset. (c) Effect of threshold coefficient.

Besides, the threshold coefficient k will affect the detection results in the segmentation procedure. Too small a control value of k gives a tight threshold and may identify the background pixels as crack defects.

However, too large a control value gives a loose threshold and may miss some true crack defect pixels. As shown in Fig. 7(c), although the same detection results can be obtained for dendritic crack images and defect-free images when k is 1 or 1.5, the threshold coefficient k is set to 1 for all the whole types of EL images. The detection results show the optimal parameters are adapted for defective and defect-free images.

3.2 Robustness Analysis in Brightness Levels

In this study, we also test some dim defective images that present much low contrast between crack and background and dim defect-free images, and the proposed method still achieve satisfactory performance. As shown in Fig. 8(a), there is a horizontal line labeled as red, which passes through both crack defect and background. The corresponding gray level intensity is shown in Fig. 8(b). It shows the low contrast between crack defect and background, which greatly increases the difficulty of crack defect detection in the dim EL images. Figure 8(c1)–(d1) are two typical low-efficiency solar cells that show weak differences between crack and background, and Fig. 8(e1) is a dim defect-free EL image. The detection result of Fig. 8(e2) shows that no crack defect is detected in the defect-free EL image. At the same time, the crack defects can be completely located in the dim defective EL images. Thus, the conclusion is that the proposed method is robust to the inhomogeneously textured background and different brightness levels.

Fig. 8.

Robustness to brightness levels of the proposed method. (a) A dim defective image. (b) Gray level intensity of red horizontal line in (a). (c1)–(e1) Two dim defective images and one defect-free image, respectively. (c2)–(e2) The corresponding response maps. (c3)–(e3) The detection results. (Color figure online)

3.3 Performance Evaluation

In this paper, our method is compared with some representative crack defect detection methods to prove the effectiveness of the proposed method. Specifically, the Anisotropic diffusion (AD) [7], the Local thresholding (LT) [8] and the basic steerable filter (BSF) [13] methods are used on the given image data set. As an illustration, Fig. 9 shows representative five types of defective EL images and a defect-free EL image and the corresponding detection results. Figure 9(a6)–(f6) show the crack defect ground truth by human labeled.

Fig. 9.

Comparison of the proposed method with previous method. The first row is five different types of defective images and one defect-free image. The second to fifth row are the detection results of AD method [7], LT method [8], BSF method [13] and our method, respectively. The defective regions are labeled as red. The sixth shows crack defect ground truth.

The AD method assumes that crack defect presents low level and high gradient, and applies anisotropic diffusion model to smooth crack defect and preserve background, simultaneously. The detection results in Fig. 9(a2)–(f2) show that some crystal grains are mistaken as crack defect because random crystal grains also exhibit the features of low gray level and high gradient. Moreover, the crack detection results are intermittent and this method is less adept at detecting crack defect in dim EL images, such as in Fig. 9(a2) and (e2). As shown in Fig. 9(f2), for defect-free image, this method easily generates false detections.

The LT method uses a local thresholding-based method to segment crack defect by calculating each sub-image’s gray level mean and standard deviation. The detection results in Fig. 9(a3)–(f3) show the crystal grains are detected as crack defect due to taking gray level information into account only, and crack defect in dim EL image cannot be detected, such as in Fig. 9(e3). For Fig. 9(f3), this method is similar to the AD method and it is not robust to defect-free image.

The BSF method uses the single basic steerable filter to process the EL images. Figure 9(a4) shows that the BSF is less adept at solving intensity variations, making crack defect incomplete. Moreover, as shown in Fig. 9(c4), this method cannot well address the bifurcations due to the lack of consideration of local neighborhood information. In particular, the vertical dark region is not crack defect but finger interruption defect. Fortunately, the BSF method is robust to the dim and defect-free images, as shown in Fig. 9(e4)–(f4).

The detection results based on our method are shown in Fig. 9(a5)–(f5). Although the inhomogeneously textured background caused by random crystal grains, even crystal grains distributed around the crack, the proposed method can detect various type crack defects and completely locate them on the inspection images. Moreover, all the false detections of crystal grains by the AD and LT methods are well addressed. Furthermore, the proposed method can well detect crack defect in dim EL image and it is robust to defect-free image. Overall, it preforms better than other three methods.

Furthermore, a quantitative evaluation is also given to compare our method with the above three representative methods. In this study, three performance indices including Precision (Pr), Recall (Re), and F-measure (Fm) are defined as follows:

$$ Pr = \frac{TP}{TP + FP} $$

(13)

$$ Re = \frac{TP}{TP + FN} $$

(14)

$$ Fm = 2 \times \frac{Pr \times Re}{Pr + Re} $$

(15)

where TP, FP, and FN represent true positives, false positives, and false negatives, respectively. The Fm can evaluate the overall performance of crack defect detection methods. The quantitative evaluation results are shown in Table 2. It shows the proposed method outperforms other methods.

Table 2. Comparison of quantitative evaluation using different methods

4 Conclusion

In this paper, we proposed a robust crack defect detection scheme in inhomogeneously textured surface of near infrared images for multicrystalline solar cells. A steerable evidence filter is designed to provide evidence for the presence of crack defect. Moreover, the performance was evaluated on the challenging data set collected from solar cells production line. The experimental results show the proposed scheme is robust to the inhomogeneously textured background, crack defect types and brightness levels change, which can achieve satisfactory detection results.