Shape Matching of 3D Topologically Segmented Objects

Abstract

Shape matching of 3D digital objects is an important domain of study from topological as well as geometric point of view. Shape matching of two or more digital objects by an efficient segmentation-based method is reported in this paper. The method receives input objects after segmentation of their articulated components and exploits the topological relation between the articulated components and the central section of the objects for shape matching. The method involves simple calculations and is primarily based on the extent of articulations in the objects. The accuracy of shape matching is dependent on the object size and segmentation of the object and is invariant to rotation. Experimental results are provided to demonstrate the structural similarity in various digital objects.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Shape matching of 3D digital objects is a well-explored area of research that often leads to feature extraction and 3D shape retrieval mechanisms. With the increase in the tools available for efficient extraction and storage of three-dimensional data, 3D shape matching has been useful for a wide variety of disciplines including computer vision, mechanical engineering, artifact searching, molecular biology, chemistry, CAD, virtual reality, medicine, entertainment, etc. Shape matching has been attempted from various perspectives including topological, graph-based, feature-based, etc. In [3], a shape descriptor called shape context is attached to each point on two comparable shapes and the correspondences between such points are used to estimate an aligning transform that aligns the two shapes. Content based 3D shape retrieval by different 3D shape descriptors have been proposed [9, 13]. Similarity between polyhedral models has been accurately identified by a topology matching approach using multi-resolution Reeb graphs [5]. This method has been extended [4] for accurate and fast determination of search key. The shape retrieval method in [2] uses extended Reeb graphs to encode different shape characteristics and compare the graphs by a method of multiple kernel learning. Another approach [7] is based on converting 3D models into skeletal graphs and matching the graphs thereby preserving topological and geometric information. The topology-varying shape matching method in [1] establishes correspondences between shapes with large topological discrepancies by topological operations like part split, duplication, and merging. Other 3D shape matching techniques involve continuous geodesic eccentricity transform [6], representing the signature of an object as a shape distribution [10], using multivariate Gaussian distribution of real valued shape descriptors [12], etc. Shape matching by object segmentation has been proposed in [11] where curve skeleton of an object has been used for segmentation.

The shape matching method proposed in this paper uses 3D segmentation as a tool for shape matching. The areas of the segment of the objects are used to identify geometric and topological properties to be used for shape matching. The rest of the paper is organized as follows. In Sect. 2, the segmented object is represented as topological space and the surface areas of the segments are processed to find similar objects. The paper is concluded with a few shape matching results in Sect. 3.

2 Proposed Work

Given a digital object A and a set of other digital objects $\{A_1,A_2,...,$ $A_n\}$, a shape matching method comparing A with each of $A_i$, $1 \leqslant i \leqslant n$, is presented in this paper. The digital objects, represented as topological spaces, are segmented and the separated articulated components are used to find the extent of similarity between two or more objects.

A digital object A represented as a triangulated surface is closed and orientable (2-manifold). The triangulation is such that exactly two triangles are incident on each edge and the interiors of no two triangles intersect. Let A be imposed on a 3D digital grid ${\mathbb G}$ represented as a set of unit grid cubes (UGCs) each of grid size g. Each grid line of ${\mathbb G}$ represents a grid level $g_i$, where $0 \leqslant i < l$, l representing the length of the object along a given coordinate plane.

2.1 Representation of Digital Object as Topological Space

Let $\mathcal {W}$ be a topological space defined by the set of triangles representing A and endowed with the topology $\Gamma _{\mathcal {W}}$. Let $\beta _{\mathcal {W}}$ be a basis for $\mathcal {W}$ defined as a collection of basis elements such that

• (i.) a basis element $\mathcal {P}_i$ consists of triangles (elements) corresponding to the object-occupied UGCs intercepted between grid values $g_i$ and $g_{i+1}$, where $0 \leqslant i < l$.

• (ii.) if $\exists \mathcal {P}_i, \mathcal {P}_j \in \beta _{\mathcal {W}}$ such that $\mathcal {P}_i \cap \mathcal {P}_j \ne \emptyset$, then $\exists \mathcal {P}_k \in \beta _{\mathcal {W}}$ such that $\mathcal {P}_k \subseteq \mathcal {P}_i \cap \mathcal {P}_j$.

For instance, in Fig. 1, the UGCs intercepted between $g_0$ and $g_1$ are intersected by the triangles $t_1$, $t_2$, $t_3$, $t_5$, $t_6$, $t_7$, $t_8$, and $t_9$. Hence, the basis element $\mathcal {P}_0 = \{t_1,t_2,t_3,t_5,t_6,t_7,t_8,t_9\}$. Similarly, $\mathcal {P}_1 = \{t_1,t_2,t_3,t_4,t_5,t_6,t_7,t_8,t_9,t_{10}\}$, $\mathcal {P}_2 = \{t_4,t_5,t_6,t_{10}\}$, and $\mathcal {P}_3 = \mathcal {P}_0 \cap \mathcal {P}_2 = \{t_5,t_6\}$. Thus, $\beta _{\mathcal {W}} = \{\mathcal {P}_0,\mathcal {P}_1,\mathcal {P}_2,\mathcal {P}_3\}$.

Fig. 1.

Digital object A is represented as the topological space $\mathcal {W}$ defined by the set of triangles $\{t_1,t_2,t_3,t_4,t_5,t_6,t_7,t_8,t_9,t_{10}\}$. The basis elements of the basis $\beta _{\mathcal {W}}$ are defined according to the grid ranges in ${\mathbb G}$.

A digital object represented as the topological space $\mathcal {W}$ is segmented by an efficient algorithm [8]. As $\mathcal {W}$ is a closed and orientable 2-manifold, it is homeomorphic to the topological space representing the 3D isothetic cover of the object for sufficiently small grid size. Quotient topology is imposed on $\mathcal {W}$ such that the basis elements representing the topologically invariant sections of the object are mapped to the elements of the corresponding quotient space. The quotient spaces are represented by weighted Reeb graphs along the yz-, zx-, and xy-planes which are segmented by an efficient algorithm. Natural segmentation is ensured by using dynamic threshold of segmentation decided by exponential averaging of the node weights belonging to the same segment. The segmented quotient spaces corresponding to the three coordinate planes are related to each other and transformed topologically to yield the segmented object in a topological space $\mathcal {W}'$ which is used here for further analysis.

Let triangles $t_1$ and $t_2$ be two elements of $\mathcal {W}$ such that they are incident on the same edge. Hence, there exists elements in $\mathcal {W}$ that do not, in general, possess disjoint neighborhoods. That is, $\mathcal {W}$ is not a Hausdorff space. The segmented topological space $\mathcal {W}'$ having topology $\Gamma _{\mathcal {W}'}$ is defined as a collection of disjoint subsets of $\mathcal {W}$. Hence, each element $t_i \in \mathcal {W}'$ belongs to a single open set in $\mathcal {W}'$ so that the elements (triangles) defining the segmentation contour virtually have disjoint neighborhoods.

2.2 Shape Matching Through Segmentation of Topological Spaces

Let $\mathcal {W}'_1$ and $\mathcal {W}'_2$ be the topological spaces representing the two objects $A_1$ and $A_2$ after segmentation. Let r be the number of disjoint open sets in $\mathcal {W}'_1$ that represent the segments of $A_1$. Let each open set $S_i$ in $\mathcal {W}'_1$, $1 \leqslant i \leqslant r$, be represented by k number of triangles, where $k > 0$. The surface area covered by each open set (segment) is calculated as the total surface area of k number of triangles. The segmentation procedure separates the articulated sections from the rest of the object. The ‘rest of the object’ is henceforth referred to as the ‘central section’. The topological relation between the articulated components and the central section of the object is explored by studying certain properties explained next. Shape matching of $A_1$ and $A_2$ is carried out using these properties.

Let $\mathcal {R} = \{S_1,S_2,..., S_c, ..., S_r\}$ denotes the set of segments of an object. Let $S_c$ represents the central component such that $\mathcal {R}' = \mathcal {R} \setminus \{S_c\}$. Let $\mu$ be the mean surface area of the segments in $\mathcal {R}'$. Let $\sigma _{\mathcal {R}}$ represents the standard deviation of the surface areas of the r segments in $\mathcal {R}$ and let $\sigma _{\mathcal {R}'}$ represents the same for the $r-1$ segments in $\mathcal {R}'$. The topological relation between $S_c$ and the other articulated components is determined by the terms,

• (i.) $\frac{|\alpha (S_c) - \mu |}{\alpha (S_c)}$

• (ii.) $\frac{|\sigma _{\mathcal {R}} - \sigma _{\mathcal {R}'}|}{\sigma _{\mathcal {R}}}$

where $\alpha (S_c)$ denotes the surface area of $S_c$.

Fig. 2.

Graph-theoretic representation of a segmented object.

Figure 2 gives a graph-theoretic representation of the segmented object where each node denotes a segment. As the segmentation procedure separates the articulated components from the rest of the object, each of the nodes in $\mathcal {R}'$ representing the articulated components are adjacent to the node $S_c$ representing the body of the object. Thus, the general structure of the graph, excluding degenerate cases, contains several pendant vertices adjacent to a node of higher degree. The surface area covered by each segment is mentioned in the corresponding node. The variation between the areas of the articulated segments and the central segment provides an idea about the variation of object topology. The term $\frac{|\alpha (S_c) - \mu |}{\alpha (S_c)}$ gives a measure of the variation considering the mean $\mu$ of the articulated segment areas. For instance, if the central segment of an object is of much larger area than its articulated segments, then the value of $\frac{|\alpha (S_c) - \mu |}{\alpha (S_c)}$ is large. Since different articulated components may have different structures and surface areas, use of the mean surface area in the above expression is justified. Again, the variation of the structures of the articulated segments from their mean $\mu$ is captured in the standard deviation $\sigma _{\mathcal {R}'}$. Similarly, variation of the structures of the articulated and the central segment from their mean is measured by $\sigma _{\mathcal {R}}$. The term $\frac{|\sigma _{\mathcal {R}} - \sigma _{\mathcal {R}'}|}{\sigma _{\mathcal {R}}}$ provides information about the contribution of the central segment to the total area of the object surface. For instance, if the articulated components of an object are much smaller in area than the central section, then the value of $\frac{|\sigma _{\mathcal {R}} - \sigma _{\mathcal {R}'}|}{\sigma _{\mathcal {R}}}$ is large.

Let us consider two digital objects $A_1$ and $A_2$ with mean surface areas $\mu _1$ and $\mu _2$ such that they are segmented into $r_1$ and $r_2$ number of segments respectively. Let $\mathcal {R}1$ ($\mathcal {R}2$) be the set of $r_1$ ($r_2$) number of segments and let $\mathcal {R}1'$ ($\mathcal {R}2'$) be the set of segments excluding the central segment $S_{c_1}$ ($S_{c_2}$). $A_1$ and $A_2$ are said to be similar in shape if the following conditions are satisfied.

• (i.) $\varDelta r = |r_1 - r_2| \leqslant \xi _1$

• (ii.) $\varDelta s = |(\frac{|\alpha (S_{c_1}) - \mu _1|}{\alpha (S_{c_1})}) - (\frac{|\alpha (S_{c_2}) - \mu _2|}{\alpha (S_{c_2})})| \leqslant \xi _2$

• (iii.) $\varDelta t = |(\frac{|\sigma _{\mathcal {R}1} - \sigma _{\mathcal {R}1'}|}{\sigma _{\mathcal {R}1}}) - (\frac{|\sigma _{\mathcal {R}2} - \sigma _{\mathcal {R}2'}|}{\sigma _{\mathcal {R}2}})| \leqslant \xi _3$

The condition (i.) is a preliminary condition for checking the similarity of two objects. Here, the threshold $\xi _1$ ranges from 0 to 1 so that $A_1$ ($A_2$) has at most one segment more than $A_2$ ($A_1$). If condition (i.) is satisfied then only we proceed further with the other conditions. For instance, in Table 1, the object Horse is checked for similarity with Hand, Leopard, and Dinosaur ($\varDelta r \leqslant 1$) and not with Spider and Ant ($\varDelta r > 1$).

For a reasonable range of areas of articulated components, the value of $\frac{|\alpha (S_{c_1}) - \mu _1|}{\alpha (S_{c_1})}$ in condition (ii.) determines the extent of articulation in the topology of $A_1$. Similarly, the value of $\frac{|\alpha (S_{c_2}) - \mu _2|}{\alpha (S_{c_2})}$ determines the extent of articulation in the topology of $A_2$. Similarity of $A_1$ and $A_2$ w.r.t. their articulations is measured by the value of $\varDelta s$. Two objects are more similar if the value of $\varDelta s$ is lower. As a convention, the threshold $\xi _2$ varies between 0 and 0.05 for the objects considered in this paper. That is, we allow a variation of 5 % in the shapes of $A_1$ and $A_2$. For instance, in Table 2, the objects Table and Human are not comparable because $\varDelta s = 0.39$ or $\varDelta s = 0.31$. It is evident that the variation of the central section area from the mean of the articulated segment areas is much larger (larger than the threshold) in case of Table than that in case of Human. On the other hand, Horse and Leopard are comparable as $\varDelta s = 0.01$. The value of $\xi _2$ depends on the object size, nature of triangulation, and density of triangulation.

Fig. 3.

Shape matching results with search object Horse (top left). According to the values of $\frac{|\alpha (S_c) - \mu |}{\alpha (S_c)}$ and $\frac{|\sigma _{\mathcal {R}} - \sigma _{\mathcal {R}'}|}{\sigma _{\mathcal {R}}}$ terms the objects Leopard (top middle) and Dog (top right) are more similar to Horse than Octopus, Chair, and Human (bottom left, middle, and right).

Fig. 4.

Top: Shape matching results with search object Spider (left) which is more similar to Ant (middle) than Hand (right). Bottom: Shape matching results with search object Airplane (left) which is more similar to Bird (middle) than Teddy (right).

The contribution of articulated components and the central component in the total surface area of $A_1$ is determined by the value of $\frac{|\sigma _{\mathcal {R}1} - \sigma _{\mathcal {R}1'}|}{\sigma _{\mathcal {R}1}}$ in condition (iii.). The contribution of articulated components and the central component in the total surface area of $A_2$ is determined by the value of $\frac{|\sigma _{\mathcal {R}2} - \sigma _{\mathcal {R}2'}|}{\sigma _{\mathcal {R}2}}$. Similarity of $A_1$ and $A_2$ w.r.t. the relation between the articulated and non-articulated sections of the objects is determined by the value of $\varDelta t$. Two objects are more similar if the value of $\varDelta t$ is lower. For instance, in Table 2, the central section in the object Horse has a much greater contribution in its total surface area than the articulated segments. Similar is the case in Dinosaur; hence Dinosaur may be compared to Horse ($\varDelta t = 0.14$). In Human, the contribution of the central section and the articulated components to the total surface area are comparable; hence Human is not comparable to Horse ($\varDelta t = 0.68$). The value of threshold $\xi _3$ also depends on the object size, nature of triangulation, and density of triangulation. If $A_1$ and $A_2$ satisfy all the three conditions, then they are considered as similar.

Table 1. Shape matching results for the objects Hand, Spider, Ant, Animal (Horse and Leopard), and Dinosaur at different postures. Each object in the database is matched against all other objects in the database. $\varDelta r$ denotes the difference in the number of segments of the two objects. $\varDelta s$ gives the difference when the term $\frac{|\alpha (S_c) - \mu |}{\alpha (S_c)}$ is evaluated for the two objects. $\varDelta t$ denotes the difference when the term $\frac{|\sigma _{\mathcal {R}} - \sigma _{\mathcal {R}'}|}{\sigma _{\mathcal {R}}}$ is evaluated for the two objects. Same objects at different postures are the most similar.

Table 2. Shape matching results for the objects Animal (Horse and Leopard), Dinosaur, Table, Chair, and Human at different postures. Each object in the database is matched against all other objects in the database. $\varDelta r$ denotes the difference in the number of segments of the two objects. $\varDelta s$ gives the difference when the term $\frac{|\alpha (S_c) - \mu |}{\alpha (S_c)}$ is evaluated for the two objects. $\varDelta t$ denotes the difference when the term $\frac{|\sigma _{\mathcal {R}} - \sigma _{\mathcal {R}'}|}{\sigma _{\mathcal {R}}}$ is evaluated for the two objects. Same objects at different postures are the most similar.

3 Results and Conclusion

Shape matching results for some topologically segmented objects are shown in Figs. 3 and 4 and Tables 1 and 2 where the articulated components have been separated from the central section. Figure 3 shows graph-theoretic representation of the objects Horse, Leopard, Dog, Octopus, Chair, and Human. Comparing the values of r, $\frac{|\alpha (S_c) - \mu |}{\alpha (S_c)}$, and $\frac{|\sigma _{\mathcal {R}} - \sigma _{\mathcal {R}'}|}{\sigma _{\mathcal {R}}}$ shows that the search object Horse is more similar with Leopard and Dog, than with Octopus, Chair, and Human. Similarly, in Fig. 4, the search object Spider is more similar to Ant than Hand, and the search object Airplane is more similar to Bird than Teddy. Databases for shape matching of different segmented objects and at various postures are presented in Tables 1 and 2. The values of $\varDelta r$, $\varDelta s$, and $\varDelta t$ show that the same object at different postures are the most similar (diagonal entries in the table). The similarity of Horse with Leopard and Dinosaur, or Ant with Spider, and the dissimilarity of Human with Table and Horse are also substantiated by the values of $\varDelta r$, $\varDelta s$, and $\varDelta t$ in Tables 1 and 2. The segmentation method is rotation-invariant preserving the areas of the segments during rotation. Hence, shape matching of two objects is preserved under rotation. Shape matching w.r.t. scaling can be taken care of by normalizing the sizes of the two objects before segmentation. The objects are segmented by topological means and the topological variations on the object surface are incorporated in the graph-theoretic representations. The scheme of shape matching in this paper, however, is independent of the topological equivalence of objects. That is why we attempt to compare the shape of an object like Chair, which is homeomorphic to a torus, to that of a spherical object Human.