Non-invasive Edge Detection of Leaves Based on Order Morphology

Abstract

Non-invasive edge detection of leaves is the key step of leaf feature extraction. Traditional algorithms for edge detection of leaves are usually invasive, since the detection is carried out after the leaves are picked. We apply order morphology in this study to non-invasive edge detection of leaves. First we analyze the algorithm for order morphology edge detection of leaves. In particular, the impact of structural elements and percentile on the detection is discussed. Based on the theory of order morphology transform of leaf images, the operator for detecting the leaf edge is constructed. Finally simulation experiment is carried out on leaf images under the conditions of natural illumination and artificial noise, respectively. Results show that the proposed algorithm is accurate and fast in leaf edge extraction and not sensitive to noise.

1 Introduction

In agricultural field, non-contact detection and measurement of plants as a way of monitoring plant growth status is an emerging topic. Leaves are the organs where photosynthesis takes place, and the morphological features of leaves represent an important aspect of plant growth status [1]. Shape is one major morphological feature of leaves [2,3,4,5,6,7], and edge extraction of leaves is one step towards leaf shape extraction.

Traditional edge detection algorithms are invasive and involve the use of picked leaves. In non-invasive detection and measurement, leaf images may present the leaves as overlapped with complex shape features. It is difficult to determine the curvature at different positions along the edge. Moreover, illumination also has a great impact on the clarity of the extracted edge. It is now urgent to establish an algorithm for non-invasive edge detection of leaves in open field.

Edge detection algorithms generally depend on the discontinuity of pixel features in different regions. Edges exist between adjacent regions that have different pixel features, and the grayscale values of the pixels along the edge are discontinuous. Considering this fact, edge detection can be carried out using differential operators. Good segmentation can be achieved with Roberts operator, Sobel operator, Prewitt operator and Canny operator where the edges between different regions are clear and the grayscale values of the pixels vary more obviously. But these operators are sensitive to noise [4, 8]. Leaf images collected under natural light usually contain considerable noise and the edge information extracted is obscure. Conditional differential operators are not applicable in that case.

More studies are concerned with edge detection of picked leaves than with non-invasive edge detection of leaves. Husin et al. [9] proposed threshold-based segmentation for edge detection of leaves, but the algorithm was applied to the entire leaf rather than to the edge. Therefore, this method is not conducive to shape extraction, and only specific types of crops can be detected. More importantly, the leaves were detected only after being picked. Li et al. [10] proposed the algorithm for edge detection of leaves in cotton images based on Mean-shift and lifted wavelet transform. This algorithm reduced the non-edge noise and more effectively extracted the edge of overlapping leaves. But this algorithm was more fit for images collected under laboratory conditions than under natural light. Lin et al. [11] described the method of edge detection of leaves based on fuzzy logics, which reduced the pseudo-edge of leaves. But this method was confined to leaf images collected in greenhouse environment. Xu et al. [12] put forward the method of edge detection of leaf veins based on fuzzy order morphology. Hu et al. [13] established the method of edge detection of seedlings of broad-leaved trees based on intuitionistic fuzzy set. The above algorithms are based on fuzzy theory and they are capable of extracting clear edges, though the computing load is very large and the complexity is high.

Order morphology theory is the combination of mathematical morphology and order statistics. Order morphology transform of digital images is the ordering of finite data of grayscale values of pixels within the range of structural elements [14]. Order morphology provides good filtering effect and noise inhibiting effect, especially in the open field. We first analyze the order morphology theory and the order morphology transform of leaf images. Structural elements and percentile are identified as the influence factors of the detection result. Finally, the leaf edge is extracted by order morphology transform. Simulation experiments are carried out under natural light condition, and the extracted leaf edges are clear and accurate and insensitive to noise.

2 Theory of Order Morphology Transform

2.1 Basic Concepts

The concept and property of order morphology transform are first defined [15, 16].

Definition 1:

Suppose the set of discrete points is A, B is structural element, 0 ≤ µ(B) = k<+ ∞ (metricµ(.) is the count of B points). Thus the order morphology transform of A with respect to B is: $A {\textcircled{P}}B (\text{p}= 0,1/\left( {{\text{k}} - 1} \right), 2/(\text{k}-1),\ldots,1)$, defined as

$$ X=A \textcircled{P} B = \left\{ {x\text{:}\mu (A \cap B\text{)} = (k\,\,\text{1)}p + \text{1}} \right\},p\,{ = 0,}\frac{ 1}{k - 1},\frac{ 2}{k - 1} \cdots ,1 $$

(1)

where p is the percentile for order morphology transform. Discrete order morphology transform can be based either on morphology sum and morphology difference.

$$ A \textcircled{0} B= \{ x:\mu (A \cap B) = 1\} = A \odot B $$

(2)

$$ A \textcircled{1} B= \{ x:\mu (A \cap B) = k\} = A \oplus B $$

(3)

The geometrical meaning of discrete is: $A \textcircled{P} B$ is the x set of (k − 1) p + 1 points which B include A.

The properties of order morphological transformation $A \textcircled{P} B$ are: monotonicity, expansibility and inverse expansibility.

Property 1.

$A \textcircled{0} B \subset A \textcircled{P} B \subset A \textcircled{1} B$

Property 2.

$A \subset C \Rightarrow A \textcircled{P} B \subset C \textcircled{P} B$

Property 3.

$(A \textcircled{P} B)^{c}=A^{\text{c}} \textcircled{q}B, p+q=1$

Property 3 show that the morphology transformation $A \textcircled{P} B$ and $A \textcircled{q} B$, p + q = 1, are the dual transformation.

Definition 2:

Suppose 0 ≤ p, q ≤ 1, A(p, q)B = $A \textcircled{P} B \textcircled{q}$ is called as multiplex order morphology transformation, obviously:

$$ A(0,1)B = A \circ B $$

(4)

$$ A(1,0)B = A \bullet B $$

(5)

$$ A(0,1)B \subset A \subset A(1,0)B $$

(6)

$$ A^{c} (p,q)B = [A(1 - p,1 - q)B]^{c} $$

(7)

Definition 3:

n multiplex order morphology is defined as:

$$ A \textcircled{P}^{\text{n}} B = (\ldots (A \textcircled{P} B) \textcircled{P} B) \textcircled{P} B) \ldots \textcircled{P} B $$

(8)

2.2 Order Morphology Transformation of Plan Leaf

The order morphology is applied to the plant leaf, which is the order transformation of plan leave.

• (1) Z(n) is the n-D digital space

• (2) f or g is the gray image of plant leaf

• (3) B is the structure element

• (4) $nB = B \oplus B \oplus B \oplus B \cdots \cdots \oplus B$

• (5) $f \le g\,{\text{is}}\,f (x )\le g (x ) ,\,\,f{ - }g\,{\text{is}}\,f (x )- g (x ) ,\,x \in Z (n )$

Definition 1. Supposed plant leaf image $f:0 \le f(x) \le m$ (m is the gray value), $B = \left\{ {x_{1} ,x_{2} , \ldots x_{k} } \right\}$ is the structure element, $0 \le \mu (B) = k < + \infty \,\,\mu (.)$ is the count of B. The order of k values in structure element B is: $f(x_{1}^{*} ) \le f(x_{2}^{*} ) \le \ldots \le f(x_{k}^{*} )$, define the d rank order variable in structure element B:

$$ order(d:f/B) = f(x_{d}^{*} )\quad (d = 1,2, \ldots ,k) $$

(9)

Definition 2. The d rank order variable of f(x) in B (d = (k − 1)p + 1 is rank) is defined as the order morphological transformation of f about B, marked $f \textcircled{P} B$, that is

$$ g_{ij}^{(d)} = f \textcircled{P} B = order(d:f/B) = f(x_{d} *)(p = 0,1/(k - 1),2/(k - 1), \cdots ,1) $$

(10)

Definition 3. f (p, q) $\text{B}=(f \textcircled{P} B) \textcircled{q} B$, (p, q = 0,1/(k − 1),…,1) is named duplicate symmetrical mix order morphological transformation of f about structure element B.

3 Analysis of Edge Features in Order Morphology Transform

As shown above, order morphology transform can act as a filter using different percentile p on the grayscale images. Thus the grayscale images can be treated by various mathematical operations [17,18,19,20,21,22,23,24,25,26,27].

3.1 Filtering and Edge Detection by Order Morphology Transform

The leaf images contain much noise, which can be filtered by order morphology transform using different percentile. When 0 < p < 1/2, the positive pulse noise in the image can be inhibited; when 1/2 < p < 1, the negative pulse noise is inhibited [28]. Two transforms can be performed separately, that is, using p value of 0-1/2 and p of 1/2-1. Thus both positive and negative pulse noise is eliminated. Suppose 0 < p < 1/2 and 1/2 < q < 1. Let f (p, q) B be open operation and f (q, p) B closed operation, both of which can act as filter. The open operation can filter out the high-luminance noise and the closed operation the smaller noise. Therefore, all types of noise can be removed by setting different p and q values.

The noise in the image can be removed and the edge be detected using the above operators. Figure 1 is the schematic diagram of edge detection based on order morphology transform. When structural element B is located in the flat region, the grayscale values of pixels within B do not vary much. So the difference is not great before and after the transform. When B is located in the region of abrupt change of grayscale values, the grayscale values of the pixels within B vary considerably. The output image will be very different from the original image. When 0 < p < 1/2, the output image $f \textcircled{P} \text{B}$ is subtracted from the original image f. Given the anti-extensibility of order morphology transform, the difference f-$f \textcircled{P} \text{B}$ (0 < p < 1/2) is the edge information of the original image. When 1/2 < p < 1, the original image f is subtracted from the output image $f \textcircled{P} \text{B}$, and the difference $f \textcircled{P} \text{B}$ – f (1/2 < p < 1) is the edge information of the original image.

Fig. 1.

Schematic diagram of edge detection

The working principle of edge detection of leaves based on order morphology transform can be described briefly as follows: When the structural element B is located within the range of the leaf, the grayscale values of the pixels within B vary slightly. Therefore, the input and output grayscale values differ little. When B is located over the edge of the leaves, the grayscale values within B vary greatly, so the input and output grayscale values also show large difference. By amplifying the changes of grayscale values of the edges through the transform, the leaf edges can be accurately recognized. Thus the edge detection operator based on order morphology has the following form:

Order morphology has unparallel advantages in edge detection by choosing proper structural elements and percentile. We build the operator based on order morphology for edge detection of leaf images.

$$ \begin{aligned} & G(f) = f - f \textcircled{P}{\text{B}} \quad \quad \quad \quad (p < 1/2) \\ & G(f) = f \textcircled{P}{\text{B}}-f \quad \quad \quad\quad (p > 1/2) \\ \end{aligned} $$

(11)

3.2 Building the Edge Detection Operator

If only one structural element is used, then the output image will contain only one type of geometrical information, which means a loss of leaf edge details. To detect the edge more accurately, several structural elements may used. These structural elements may differ in size and shape. We present the edge detection operator based on several structural elements.

$$ G(f) = f\textcircled{q}{\text{n}}_{1} B_{1} \textcircled{P}{\text{n}}_{2} B_{2} \textcircled{q}{\text{n}}_{3} B_{3} - f\textcircled{q}{\text{n}}_{4} B_{4}\textcircled{P}{\text{n}}_{5} B_{5}) $$

(12)

$$ G(f) = f\textcircled{q}{\text{n}}_{1} B_{1} \textcircled{P}{\text{n}}_{2} B_{2} -f \textcircled{q}{\text{n}}_{3} B_{3} \textcircled{P}{\text{n}}_{4} B_{4}\textcircled{P}{\text{n}}_{5} B_{5}) $$

(13)

$$ G(f) = f\textcircled{q}{\text{n}}_{1} B_{1} \textcircled{P}{\text{n}}_{2} B_{2} \textcircled{q}{\text{n}}_{3} B_{3} -f \textcircled{q}{\text{n}}_{4} B_{4}\textcircled{P}{\text{n}}_{5} B_{5} \textcircled{P}{\text{n}}_{6} B_{6}) $$

(14)

In the operator, n_i represents different structural element; B_i indicates the shape of the structural element. The outer edge of the image is extracted in formula (12), and the edge spanning over the Euclidean border is extracted in formula (14). Different shape of the structural elements corresponds to order morphology operation in different directions of the leaf. The final leaf edge is obtained by combining together the edges extracted in all directions. For a given structural element, different size corresponds to different scale of extraction. Small-size structural elements have lower noise-removing capacity, but they are more sensitive to edge details. On the contrary, large-size structural elements have stronger noise-removing capacity, but the more details may be lost. So the best way is to use several structural elements with varying size and shape. An illustration is carried out using formula (14). This formula extracts the Euclidean edge of the leaf. Suppose the structural elements n₁B₁ and n₂B₂, n₄B₄ and n₅B₅ have identical shape and size, respectively. If n₃B₃ and n₆B₆ have the same shape but different size, then the larger the n₃ and n₆, the wider and the brighter the extracted edge; otherwise, the edge will be narrower and darker. If n₃B₃ and n₆B₆ have the same size but different shape, it is necessary to choose appropriate shape of the structural element depending on specific leaf so as to extract the edge more accurately.

3.3 Impact of Structural Element and Percentile on Leaf Edge Detection

In mathematical morphology, the choice of structural elements has a large impact on the extracted edge [29]. This is also true in order morphology transform, but no relevant studies have been reported yet. We discuss the impact of structural elements and percentile on leaf edge detection.

The thickness and direction of the extracted edge can be controlled by controlling the shape and size of the structural elements. The structural element is expressed as the morphology sum of several structural element Bj, which is n_iB_j (n is the size and B is the shape). Figure 2 shows the edges of different thickness when the size of the structural element is 3 × 3, 4 × 4 and 5 × 5, respectively, with the constant shape of a rectangle.

Fig. 2.

Leaf edge of different thickness

It can be seen from Fig. 2 that the larger the size of the structural element, the thicker the extracted edge is. But thicker edges are usually obscure with the generation of some pseudo-edges, which increases the difficulty of further processing. To overcome this defect, the size of the structural element can be varied depending on the needs.

Similarly, by changing the shape of the structural elements, the edges extracted will show different features. Given the same size of the structural elements, different shape of the structural elements will result in different features of edges. Here we choose the constant size of 3 × 3 and vary the shape of the structural elements. The results are shown in Fig. 3.

Fig. 3.

Edge of different shape

As shown in Fig. 3, edges with different features are extracted by setting different shape of the structural elements. For example, a cross-like shape results in satisfactory edges in transverse and vertical direction, but poor edges in other directions. In Fig. 3(b), both the cross-like shape and the diagonal shape result in clear edges. Because the leaf edges are approximately elliptical or circular or oblique lines but rarely rectangular, the use of rectangular structural element results in less clear edges (See Fig. 3(a)). It is important to choose the appropriate shape of structural elements for specific leaves, as discussed below.

In order morphology transform, the values of percentile (p and q) are important to edge detection. We discuss the impact of percentile on edge detection by choosing different percentile values. Let 0 < p < 1/2 and 1/2 < q < 1 and the structural element is rectangular with size of 6 × 6. The leaf vein edges are extracted at different p and q values, as shown in Fig. 4. When p = 1/3 and q = 35/36, the extracted leaf edges are shown in Fig. 4(a); when p = 1/36 and q = 35/36, the extracted leaf edges are shown in Fig. 4(b); when p = 1/3 and q = 35/36, the leaf edges are shown in Fig. 4(c).

Fig. 4.

Edge of different percent

It is easy to see that if p and q differ greatly, the extracted edges are more clear and thicker, though some details are lost (comparing (a) and (b), the leaf veins are more clear). But pseudo-edges appear in (a). After several experiments, it is found that a large difference existing between p and q is more conducive to edge detection.

4 Simulation Experiment on Order Morphology Edge Detection of Leaves

We carry out simulation experiment on order morphology detection of leaves under natural light condition. Figure 5(a) is the original leaf image, and the leaf edges are extracted using traditional Sobel operator and Canny operator. The results are shown in Fig. 5(b) and (c). We apply order morphology edge detection operator in formula (11) to extract leaf edges, as shown in Fig. 5(d). Using the proposed operator based on several structural elements of different size and shape, the extracted edges are shown in Fig. 5(e) and (f).

Fig. 5.

Edge detection of plant leaf

The leaf edges extracted by Sobel operator are less clear and discontinuous with considerable loss of details. Canny operator fails to extract leaf edges and the resulting image is obscure and does not show the general contour of the leaves. Though the traditional order morphology operators can detect the leaf edges, the edges have low contrast and do not present a clear contour with loss of some of the edge information. Formula (14) is used as the edge detection operator based on several structural elements of varying size and shape. The edges in Fig. 5(e) and (f) are extracted using different structural elements. In Formula (14), n₁ is 5 × 5 and B₁ is rectangular; n₂ is 3 × 3 and B₂ is rectangular; n₃ is 7 × 7 and B₃ is cross-like shape. n₄B₄ and n₅B₅, n₁B₁ and n₂B₂ have identical shape and size, respectively. When n₆ is 5 × 5 and B₆ is rectangular, the detection result is shown in Fig. 5(e). Then n₁ is 5 × 5 and B₁ is rectangular; n₂ is 3 × 3 and B₂ is rectangular; n₃ is 5 × 5 and B₃ is diagonal; n₄B₄ and n₅B₅, n₁B₁ and n₂B₂ have identical shape and size, respectively. When n₆ is 5 × 5 and B₆ is cross-like shape, the detection result is shown in Fig. 5(f). All edges are clear and intact using different structural elements. The edges in Fig. 3 have the highest clarity and contrast, because the structural elements used are larger and have more variable sizes than those in Fig. 5(f).

One major advantage of order morphology edge detection is that the noise is filtered simultaneously. So we perform another experiment where 10% salt and pepper noise is added into the leaf images. The edge detection of leaves is carried out using Sobel operator, Canny operator and the proposed operator in Formula (11), respectively. The results are shown in Fig. 6.

Fig. 6.

Edge detection of noised plant leaf

It can be seen from the figure that the traditional operators are sensitive to noises and they cannot filter the noise; the edges extracted are obscure and contain a large amount of noise. The traditional order morphology transform operator only extracts the general contour of the leaves contaminated by the noise. Our operator obtains very clear leaf edges. The structural elements in Fig. 6(e) and (f) are the same as those in Fig. 5. It can be seen that the operator 1 is little affected by noise and the extracted leaf edges are clear and intact, with noise basically filtered. Operator 2 is also effective in removing a major part of noise. As shown in Fig. 6(f), the extracted leaf edges are basically not affected by noise, and compared with Fig. 5, the edges are clear and intact. This proves the ability of the proposed operator in filtering noise and its high applicability to natural light condition. Since the proposed operator allows for the adjustment of size and shape of structural elements and the percentile value, it can adapt to a variety of leaf features perfectly.

5 Conclusion

We propose a method of non-invasive order morphology edge detection of leaves. Starting with thorough analysis of the order morphology theory and order morphology transform of leaf images, we proceed to the noise filtering and edge detection ability of order morphology. Structural elements and percentile have large impact on the detection results. We then build the order morphology edge detection operator based on several structural elements with varying size and shape. The algorithm is verified by simulation experiment.

The experiment consists of two parts. One is the edge extraction from leaf images collected under natural light condition. The other is edge extraction in the presence of artificial noise. Results show that the leaf edges extracted by the proposed operator are more clear and accurate and insensitive to noise. This operator can adapt to natural light condition and can be used for shape recognition of plant leaves and plant growth monitoring in the future.