Structured Pointcloud Segmentation for Individual Mangrove Tree Modeling

Abstract

Tree structure parameters of mangrove forests are hard to measure in the field and therefore inventories of this type of forests are impossible to keep up to date. In this article, we tested a structured pointcloud segmentation method for extracting individual mangrove trees. Structure parameters of individual trees were estimated from the segmented pointcloud and its 3d geometry was generated using revolution surfaces. Estimated parameters were then assessed at both plot and tree levels using field data. It was observed that the number of segments in each test plot agreed well with the number of trees observed in the field. Nonetheless, the estimated parameters exhibited mixed accuracy with top height being the most accurate.

This research was funded by “Fondo Institucional de Fomento Regional (FORDECyT)” grant number 273646 and JLS was funded by “Fondo Sectorial CONACyT-INEGI” grant number 268773.

1 Introduction

The study and monitoring of fragile ecosystems such as mangrove forests is of vital importance worldwide as they are among the most productive and most Carbon-rich forest in the tropic; they provide habitats to over 1300 species of animals and act as protection of shorelines, hurricanes and tidal surges. It is estimated that between 35–50% of mangrove coverage have been lost in the past 60 years due mainly to human activities and, despite it accounts for only the 0.7% of tropical forest areas, it amounts as much as around 10% of emissions from deforestation globally [3]. Their assessment have been largely carried out through forest inventories that are time consuming, costly and, consequently, not frequently updated. More recently, efforts have been made to streamline inventorying processes through remote sensing and computational technologies, including pointcloud analysis, which is still limited due to the lack of efficient automation processes.

Among the many alternatives found in the literature, the voxel method is one of the most widely used method for three-dimensional models of trees due to its relative easy structure [14]. Other approaches use simple geometric models such as paraboloids or spheres to approximate the tree crown [7, 9, 12]. More realistic models have been also investigated, but tend to be computationally costly. For instance, radial basis functions and isosurfaces have been shown to achieve more natural forms of tree crowns [4], while others have focused on the reconstruction of the skeleton [1] or adjust free shapes to pointclouds [2]. From an inventorying viewpoint, the most useful models are those that can encode the most relevant structure parameters with the lowest complexity. One fundamental processing step is the extraction of individual trees pointclouds through a segmentation method. Major pointcloud segmentation strategies have been revised by [13], but all of them can be grouped in one of three types depending on the format of the input: 1) based on unstructured pointclouds, such as unsupervised clustering or primitives fitting [5, 15], 2) raster-based methods, which impose a regular spatial structure, either in 2-d (pixels) or in 3-d (voxels) on the pointcloud [6, 11], and 3) based on structured pointclouds, where the neighbourhood relationship is imposed using a directed graph. Although the latter approach can be the most accurate, it is also the most difficult to implement and thus the least researched approach. One of the few studies is that of [10] who used graph partition and connected components labelling (CCL) to delineate tree crown over a pine-dominated site. Nonetheless, they used a raster-based segmentation for building a hierarchical structuring prior to graph partitioning.

In order to fill this gap, we propose a novel structured pointcloud-based segmentation method that integrates a priori knowledge about the shape of target objects for the neighbourhood definition. The general strategy consists in creating an initial neighborhood connectivity matrix used with the CCL algorithm that is then progressively pruned and re-labelled. In the following sections we present 1) a brief description of the study site and data used, 2) the segmentation method, 3) the 3-d modelling approach, 4) the accuracy assessment and 5) the major conclusions.

2 Study Site and Data Used

The study site is around El Cometa, a small lagoon (67 ha) within the Pantanos de Centla reserve located at the north of the Mexican state of Tabasco (Fig. 1a). This site is mostly covered by the red mangrove (Rhizophora mangle), a threatened species (SEMARNAT, 2010) that mixes with medium sub evergreen tropical forest of pucté (Bucida buceras L.). There is also a low population of white mangrove (Laguncularia racemosa) around the lagoon and channels.

Fig. 1.

(a) Location of the study area and (b) LiDAR-based DSM and plot locations

Tree structure data was collected from 10 plots of 50 m by 50 m in 2014 (plots 1–10), 20 plots of 25 m by 50 m in 2016 (plots 1–20) and 7 plots of 25 m by 50 m in 2017 (plots 21–27). Plot center and orientation were measured with sub-meter precision GPS system. Tree individuals data included species name, xy-location (measured with respect to plot center), total height, diameter at breast height (DBH), diameter above and bellow highest root (mangrove trees only), crown diameters along E-W and S-N orientations, root diameters along E-W and S-N orientations, roots total height, and time and date of measurement. Individuals with lower DBH than 10 cm were not sampled. Only data from 25 plots were used as some were either not fully covered by LiDAR data or repeated in two different years.

The LiDAR sensor was flown in March 26, 2014 over an area of 2.5 km by 3.6 km (Fig. 1b) with an average point density of 20 pts/m$^2$ and vertical accuracy of $\pm 0.15$ m. Data was delivered with ground points identified, from which a terrain surface was generated. Subsets for each sampling plot with a buffer of 20% where extracted and normalized by subtracting the terrain surface. These data were the main input to the individual tree point extraction method described next.

3 Segmentation Method

Major stages of the proposed pointcloud segmentation method are described in the following subsections, where parameters referred throughout are defined in Table 1.

Table 1. Segmentation parameters used. $z_{max}$ is the maximum z-coordinate of segment.

3.1 Point Connectivity and Pruning

Pointcloud neighbourhood is represented through an adjacency or connectivity matrix $C=[C_{i,j}]_{i,j=0,\ldots ,n}$, so that a point j is said to be in the neighbourhood of point i if $C_{i,j}$ is one, otherwise it is zero. The connectivity matrix of any order, including the zeroth, shall be denoted by $C^*$. Recall that the k-th order neighbours are given by the power matrix $C^k$.

There are several alternatives for defining pointcloud neighbourhoods, such as maximum distance, k-nearest neighbours, and Delaunay tessellation, among others. Here we used the latter option as its computation is efficient yielding a relatively high sparsity, i.e., high fraction of zeroes in C. Hence, the initial connectivity matrix is given by:

$$\begin{aligned} C_{i,j} = {\left\{ \begin{array}{ll} 1 &{} \text {if (}{} \textit{i, j}\text {) is a Delaunay edge} \\ 0 &{} \text {otherwise} \end{array}\right. } \end{aligned}$$

(1)

The sparsity of the connectivity matrix is further increased through applying the following pruning:

• 1. All the neighbors must be lower than the point, i.e., $C_{i,j} = 0, \text { if } z_i \le z_j$

• 2. All points must be non-terrain points, i.e., $C_{i,j} = 0, \text { if } z_i \le z_{grmax}$

• 3. Every point is at most in the neighborhood of the nearest option, i.e., $C_{i,j} = 0, \text { if } d_{i,j} > \min _k\{d_{k,j}\}$

• 4. Horizontal distance between a pair of neighbors is within a maximum distance, i.e., $C_{i,j} = 0, \text { if } r_{i,j} > r_{max}$

The first criterion ensure that the graph defined by the connectivity matrix is top-down directed, the second criterion discard terrain points, and the third one ensures no loops are present in the directed graph, thus defining a tree-like structure. The threshold distance $r_{max}$ in the fourth criterion controls the size and number of segments. A large threshold value favors the inclusion of points from distinct nearby trees into a single segment (under segmentation), whereas a small value will tend to separate points from a single tree into several segments (over segmentation). Because of the high density of mangrove forests, no optimal value for $r_{max}$ can be satisfactory and one always needs to undertake further steps to either merge or split segments. The favored option here is to select a relative large threshold value and to apply a progressive splitting strategy. This decision is mainly driven by the difficulty of searching points to reconnect over the option of searching edges to eliminate.

3.2 Segment Splitting

Starting with the segmentation induced by labelling the connected components in C, the segment splitting procedure is applied to wide segments as follows. For each selected segment one of the connection in the connectivity matrix is eliminated and the process repeated until either no more wide segments are available or the maximum number of iterations has been reached.

Segments are considered to be wide if its diameter is greater than a maximum allowed diameter ($d_{max}$), whereas segment diameter is computed by averaging the ranges of projections $u_i = x_i\cos \theta +y_i\sin \theta$ along four directions: $\theta = 0$, 45, 90 and 135$^\circ$. Such diameter computation must be based on high points, i.e., the points with z-coordinates greater than threshold ($z_{himin}$). In addition, segments with lower number of points than a threshold ($2w_{min}$) are not considered candidates for splitting.

The segment splitting consists in eliminating, at most, one edge per connected component, where selected edge fulfill three conditions:

• 1. Large horizontal distance

• 2. Similar number of points in generated sub-graphs

• 3. Low points available in generated sub-graphs

The first condition tends to maintain connected components horizontally compact, the second one reduces the number of required iterations by fast reduction of segments size, and the third one warrants connected components will include points that most likely belong to the aerial roots of mangrove individuals. These conditions are met by maximizing the following edge measure:

$$\begin{aligned} E_{i,j} = r_{i,j}\min \{w_j,w_j^c\}-\min \{b_jw_j,b_j^cw_j^c\} \end{aligned}$$

(2)

where $r_{i,j}$ is the horizontal distance of the edge (i, j), $w_j$ is the size of sub-graph at j, $w_j^c$ the size of complement sub-graph at j, so that $w_j+w_j^c$ is the size of original graph, i.e., the total number of points in the segment, and $b_j$ and $b_j^c$ are z-coordinates of lowest point of each sub-graph. Equation (2) can be interpreted through a physical system analogy where points represent particles of unit weight connected trough rigid links, which are defined by the connectivity matrix C. Each link is subject to a pair of forces or weights that excerpt torques with respect to middle point of the link, which is supposed fix. Hence, maximizing of the energy acting on the rigid link while minimizing the tendency to motion yields an expression like Eq. (2). Such energy is required to be over a threshold ($E_{min}$) to cause the link to break.

At each iteration, Eq. (2) is evaluated for each wide segment and the edge with maximum positive value is eliminated. This can be expressed as:

$$\begin{aligned} C_{i,j} = 0, \text { if } E_{i,j} = \max _{(k,l)}\{E_{k,l}\} \text { and } E_{i,j} \ge E_{min} \end{aligned}$$

(3)

4 3-D Modelling

4.1 Tree Structure Estimation

Small segments that did not comply with minimum tree height ($h_{min}$), minimum crown diameter ($d_{min}$) or samples size ($10w_{min})$ were discarded. Points from remaining segments were then classified into low, mid and high based on thresholds $z_{himin}$ and $z_{lomax}$ defined in Table 1, which had been empirically calibrated. Then, tree location and structure parameters were estimated from segmented pointcloud as described in Table 2.

Table 2. Tree structure parameters as computed from segment points.

Assuming a multivariate Gaussian distribution of points belonging to tree crown and to roots, where radial symmetry can be assumed in both cases, and symmetry along z can be further assumed for the roots, one can estimate crown diameter, crown bottom height, root diameter and root height using the ellipse in the rz-plane¹:

$$\begin{aligned} \left( \frac{r}{\sigma _r}\right) ^2+\left( \frac{z}{\sigma _z}\right) ^2=\chi ^2 \end{aligned}$$

(4)

where the semi-axis of the ellipses are given by the standard deviations times a constant ($\chi =2.1459$) that is theoretically determined to enclose 90% of points and $\sigma _r^2=(\sigma _x^2+\sigma _y^2)/2$.

While high points and low points determined crown and root parameters, trunk diameter had to be computed through an empirical relation between the DBH and crown diameter of red mangrove, which was strongly linear and statistical significant ($R^2=0.63$). This was necessary, because trunk points are hard to determine and seldom accessible from airborne LiDAR. Moreover, trunk orientation was defined through the elevation and azimuth angles of a vector joining root top and crown base locations, and where no low points were available, crown location was used, the root height was defined as zero and the root diameter, same as the trunk diameter.

Estimated parameters were represented using a 3-D surface model with three parts (roots, trunk and crown), each of which was represented through a revolution surface of the form:

$$\begin{aligned} x(u,\theta )&=\rho (u)\cos \theta \end{aligned}$$

(5)

$$\begin{aligned} y(u,\theta )&=\rho (u)\sin \theta \end{aligned}$$

(6)

$$\begin{aligned} z(u,\theta )&=a+(a-b)(1-(1-u^{\alpha _6}))^{\alpha _7} \end{aligned}$$

(7)

$$\begin{aligned} \rho (z)&= r\left| \alpha _1\pm \left[ \alpha _2-\alpha _3\left( \frac{a-z}{a-b}\right) \right] ^{\alpha _4}\right| ^{1/{\alpha _5}} \end{aligned}$$

(8)

for $\theta \in [0,2\pi )$ and $u\in [0,1]$, where $\alpha _1,\ldots ,\alpha _7$ are shape parameters, a is the top limit, b the bottom limit and r is the radius of the surface. The ± symbol indicates that there are two types of models: models with the plus sign are of hyperbolic type, whereas the negative are of parabolic type.

Models used for mangrove trees were ellipsoids for crown, with $\alpha$-values of (1,1,2,2,2,1,1), respectively, cylinders for trunk (2,0,1,0,2,1,1) and flattened parabolid for roots (0,0,1,1,4,1,2). Furthermore, trunk inclination was modeled by shifting the xy-coordinates as function of z-coordinate. In this case, xy-coordinates are given by:

$$\begin{aligned} x(u,\theta )&=\rho (u)\cos \theta +\frac{z-h_r}{\tan {\theta _t}}\cos \phi _t \end{aligned}$$

(9)

$$\begin{aligned} y(u,\theta )&=\rho (u)\sin \theta +\frac{z-h_r}{\tan {\theta _t}}\sin \phi _t \end{aligned}$$

(10)

Fig. 2.

Top and lateral view of segmented pointcloud (left) and surface model (right) of extracted trees for two plots.

5 Accuracy Assesment

The segmentation method was applied on the 25 plots from which tree models were generated. Figure 2 shows the top and lateral views of the segmented pointcloud and the corresponding surface model for a couple of plots. It can be seen that the object level representation captures the coarse shape of the pointcloud, yet whether such a representation match the actual forest structure was subject to a quantitative assessment, both at the plot and individual levels.

At the plot level, we selected the segments laying within the plot limits based on their location, ($x_{loc}, y_{loc}$), and compared the averages parameter values. Figure 3-left shows the scatterplot between the number of LiDAR-derived trees within the plots and the groundtruth trees. With exception of plot 8, LiDAR-derived trees seemed to follow a global trend that agrees with respect to those observed in the field, with a median absolute error of 7 trees per plot.

Fig. 3.

LiDAR-derived vs. groundtruth trees based on number of trees within plot limits (left) and number of matched trees (right). See text for further details.

Figure 4a shows the scatterplots of per-plot averages and Table 3a provides the comparative analysis in terms of the coefficient of determination ($R^2$) and of the root mean squared error (RMSE). The scatterplots reveal that the crown diameter, root diameter and root height were generally overestimated, whereas such a bias was not present in the case of top height, which exhibited the largest $R^2$ value (0.18). Nevertheless, the root height showed the lowest RMSE and the root diameter had the largest value. This can be explained in terms of the dynamical range of these structure parameters and in terms of an overestimation of aerial roots for non-mangrove species.

Fig. 4.

Estimated vs. groundtruth scatterplots at the plot- and individual levels for top height (top-left), crown diameter (top-right), root height (bottom-left) and root diameter (bottom right).

At the individual level, LiDAR-derived trees were paired to the ground-truth trees. This process required to correct for systematic errors in the location of ground-truth trees, which was carried out through the spatial correlation of the canopy height models in raster format using a rotation-invariant correlation measure developed for raster data [8]. The resulting translation and rotation parameters were then applied to the ground-truth tree locations to use as ground-truth canopy location. Such canopy locations, together with ground-truth tree height, served to match LiDAR-derived trees to ground-truth trees. A one-to-one pairing was ensured by sequentially pairing the nearest unassigned trees until no more pairs could be formed. Maximum distance between paired trees was limited to 5 m in order to limit the impact from pairing errors over the parameter estimation errors. The number of pairs was generally slightly lower than the actual number of trees in the plot as can be seen in the scatterplot of Fig. 3-right. Figure 4b shows the scatterplot of matched trees and Table 3b, the corresponding $R^2$ and RMSE measures. In this case, it was possible to segregate mangrove trees using the species information from field data. The overestimation bias that was observed at the plot level was again observed at the individual level for crown diameter and root diameter, but not for root height. Top height was again unbiased and showed the highest $R^2$ value (0.61), although in this case the pairing process must have had an influence on these. Scatterplots also reveal that non-null root parameters were retrieved for non-mangrove species thus indicating a limitation of the segmentation method, which may have mistaken understorey vegetation as roots.

Table 3. Coefficient of determination ($R^2$) and root mean square error (RMSE) at the plot and individual levels. (Values in parenthesis were based on mangrove species only).

6 Conclusions

Most existing tree crown segmentation methods have focused on temperate forests, where intertwined branches and aerial roots do not represent an issue. In contrast, in this study we developed and tested a segmentation method that incorporates information about the structure of objects embedded in the pointcloud, namely, the mangrove trees. The method used a sparse matrix that defines the neighbourhood relation among points, from which the connected-component labelling method is repeatedly applied together with a segment splitting strategy. The accuracy assessment showed a good promise for automated mangrove forest inventories. Nonetheless, methods for parameters estimation must be further improved, specially root structural parameters, specially because they are critical for quantifying blue Carbon storage by mangrove forests.