Fuzzy Directional Enlacement Landscapes

Abstract

Spatial relations between objects represented in images are of high importance in various application domains related to pattern recognition and computer vision. By definition, most relations are vague, ambiguous and difficult to formalize precisely by humans. The issue of describing complex spatial configurations, where objects can be imbricated in each other, is addressed in this article. A novel spatial relation, called enlacement, is presented and designed using a directional fuzzy landscape approach. We propose a generic fuzzy model that allows to visualize and evaluate complex enlacement configurations between crisp objects, with directional granularity. The interest and the behavior of this approach is highlighted on several characteristic examples.

1 Introduction

The spatial organization of objects is fundamental to increase the understanding of the perception of similarity between scenes or situations. Despite the fact that humans seem capable of apprehending spatial configurations, in many cases it is exceedingly difficult to quantitatively define these relations, mainly because they are highly prone to subjectivity. Standard all-or-nothing mathematical relations are clearly not suitable, and the interest of fuzzy relations was initially suggested by Freeman in the 70s [9], since they allow to take imprecision into account. Over the last few decades, numerous works were proposed on the analysis of spatial relationships in various domains, ranging from shape recognition to computer vision, with the main purpose of describing the relative positioning of objects in images [3]. These approaches provide a set of interesting features able to describe efficiently most of spatial situations. However, some configurations remain challenging to describe without ambiguities, especially when the objects are imbricated, or composed of multiple connected components. In this context, we propose to study new relations dedicated to the imbrication of objects.

This article is organized as follows. Section 2 presents related works to our approach. Section 3 recalls the model of directional enlacement, proposed in [6] for the description of complex spatial configurations. From the latter, we propose in Sect. 4 a generic model relying on fuzzy landscapes that allows to evaluate relative enlacement configurations between crisp objects, with directional granularity. Section 5 presents experimental results on different illustrative examples that allow to highlight the behavior and the interest of this model. Section 6 provides conclusions and perspectives.

2 Related Work

In the domain of spatial relations, two major research axes can be distinguished in the literature, based on two dual concepts: the one of spatial relationship and that of relative position. On the one hand, it is possible to formulate a fuzzy evaluation of a spatial relation (for example “to the left of”) for two objects, in order to describe their relative position. The fuzzy landscape model is a widely used method for providing this type of assessments [2]. This approach relies on the fuzzy modeling of a given spatial relation, directly in the image space, using morphological operators. Applications of this model can be found in various domains such as spatial reasoning in medical images [7] or the recognition of handwriting [8]. On the other hand, the location of an object with regards to another can be modeled by a quantitative representation, in the form of a relative position descriptor. Different spatial relations can be assessed from this representation and the associated descriptors can be integrated in pattern recognition systems to match similar spatial configurations. Among the various relative position descriptors, the histograms of forces [14] are widely used due to their ability to process pairwise information following a set of directions. They are applied in different works, such as the linguistic description of spatial relations [11] or image retrieval [5]. To summarize, fuzzy landscapes consist in determining the region of space matching a specific spatial relation, and relative position descriptors consist in characterizing the position of an object with regards to another, by combining different spatial features into a standalone descriptor.

Although these two types of approaches allow to interpret many spatial relations between objects, they usually fail at properly describing more complex spatial configurations, in particular when objects are concave, or composed of multiple connected components [4]. A typical complex spatial relation is the “surrounded by” relation, which was first studied by Rosenfeld [15] and deepened by Vanegas [16] with a dedicated approach based on fuzzy landscapes. Another specific spatial relation is “between”. This relation has been studied in details in [4], involving definitions based on convex hulls and specific morphological operators. Applications of this spatial configuration for the analysis of histological images have been proposed by [10]. Work has also been done to characterize the “alignment” and “parallelism” of objects in satellite images [17]. Recent works introduced the $\phi$-descriptor [12, 13], which is a powerful generic framework to assess any spatial relation from a set of specific operators, inspired by Allen intervals [1]. This descriptor can determine if two objects are imbricated or not, but it is not able to measure the depth of imbrication (such as, for instance, when two spirals are interlaced).

In this context, recent works [6] introduced both enlacement and interlacement descriptors, from the relative position point of view, in order to obtain a robust modeling of the imbricated parts of objects. Based on this model, in this article we propose to tackle the dual point of view, by considering fuzzy enlacement landscapes instead of enlacement descriptors. The goal of fuzzy enlacement landscapes is to visualize and evaluate these spatial configurations directly in the image space, by considering the concavities of the objects in a directional fashion.

3 Directional Enlacement Model

In this section, we present the model used to describe the relative enlacement of objects. This model was initially introduced in [6], mostly from the point of view of the relative position descriptors. Here, we recall the intuitive idea behind what is intended with the term enlacement, and we provide some useful definitions and notations for this model.

A two-dimensional object A of the Euclidean space is defined by its characteristic function $f_A : \mathbb {R}^2 \rightarrow \mathbb {R}$. This generic definition allows to handle both crisp and fuzzy objects. Let $\theta \in \mathbb {R}$ be an orientation angle, and $\rho \in \mathbb {R}$ a distance from the origin. We define the oriented line of angle $\theta$ at the altitude $\rho$ by the non-finite set $\varDelta ^{(\theta , \rho )} = \{e^{i\theta }(t + i\rho ), t \in \mathbb {R}\}$. The subset $A \cap \varDelta ^{(\theta , \rho )}$ represents a one-dimensional slice of the object A, also called a longitudinal cut. In the case of crisp objects, such a longitudinal cut of A is either empty (the oriented line does not cross the object) or composed of a finite number of segments. In the general case, a longitudinal cut of A along the line $\varDelta ^{(\theta , \rho )}$ can be defined as:

$$\begin{aligned} \begin{array}{lrcl} f_A^{(\theta , \rho )} : &{} \mathbb {R}&{} \longrightarrow &{} \mathbb {R}\\ &{} t &{} \longmapsto &{} f_A(e^{i\theta }(t + i\rho )). \end{array} \end{aligned}$$

(1)

Let (A, B) be a couple of objects. The goal is to describe how A is enlaced by B. The intuitive idea is therefore to capture the occurrences of points of A being between points of B. In order to determine such occurrences, objects are handled in a one-dimensional case, using longitudinal cuts along oriented lines. For a given oriented line $\varDelta ^{(\theta , \rho )}$, the idea is to combine the quantity of object A (represented by $f_A^{(\theta , \rho )}$) located simultaneously before and after object B (represented by $f_B^{(\theta , \rho )}$). Let f and g be two bounded measurable functions with compact support from $\mathbb {R}$ to $\mathbb {R}$. The enlacement of f with regards to g is defined as:

$$\begin{aligned} E(f, g) = \int _{-\infty }^{+\infty } g(x) \int _{x}^{+\infty } f(y)\int _{y}^{+\infty } g(z) \; \mathrm {d}z \, \mathrm {d}y \, \mathrm {d}x. \end{aligned}$$

(2)

The scalar value $E(f_A^{(\theta , \rho )}, f_B^{(\theta , \rho )})$ represents the enlacement of A by B along the oriented line $\varDelta ^{(\theta , \rho )}$. For crisp objects (i.e., each point is either 0 or 1), it corresponds to the total number of ordered triplets of points on the oriented line, which can be seen as arguments to put in favor of the proposition “A is enlaced by B” in the direction $\theta$. Algorithmically, this value can be derived by an appropriate distribution of segments lengths along the longitudinal cuts of both objects (see [6] for more details).

The set of all parallel lines $\{\varDelta ^{(\theta , \rho )}, \rho \in \mathbb {R}\}$ in the direction $\theta$ slices the objects into sets of longitudinal cut functions. To measure the global enlacement of an object with regards to another in this direction, we aggregate the one-dimensional enlacement values obtained for each of these longitudinal cuts.

The enlacement of A by B in direction $\theta$ is defined by:

$$\begin{aligned} \mathcal {E}_{AB}(\theta ) = \frac{1}{||A ||_1 ||B ||_1} \, \int _{-\infty }^{+\infty } E(f_A^{(\theta , \rho )}, f_B^{(\theta , \rho )}) \, \mathrm {d}\rho , \end{aligned}$$

(3)

where $||A ||_1$ and $||B ||_1$ denote the areas of A and B. This normalization allows to achieve scale invariance. In the binary case, this definition corresponds to a number of triplets of points to put in favor of “A is enlaced by B” along the longitudinal cuts in this direction. Intuitively, it can be interpreted as the quantity of B traversed while sliding the object A in the direction $\theta$, with regards to the quantity of B located on the opposite direction.

In [6] the enlacement model $\mathcal {E}_{AB}$ was considered from the point of view of the relative position descriptors by building a directional enlacement histogram, allowing to characterize how an object A is enlaced by another object B. In the next section, we involve this model in a novel evaluation point of view based on a fuzzy approach that allows to evaluate enlacement configurations directly in the image space, with directional granularity.

4 Fuzzy Enlacement Landscapes

We present here how to extend the directional enlacement model to evaluate the enlacement of objects in the image space from a local point of view, inspired by the works of Bloch [2] on fuzzy landscapes for classical spatial relations.

4.1 Definition

A fuzzy enlacement landscape of an object A should be a representation of the region of space that is enlaced by A. Since the initial enlacement model is essentially directional, we also propose to define directional enlacement landscapes. Let A be a crisp object (i.e., represented as $f_A : \mathbb {R}^2 \rightarrow \{0,1\}$). In a given direction $\theta$, for a point outside of A located at $(\rho , t)$ coordinates in the rotated frame, its local enlacement value can be defined as:

$$\begin{aligned} \mathcal {E}_A(\theta )(\rho , t) = \frac{1}{{||A ||}_1} \, \int _{t}^{+\infty } f_{A}^{(\theta , \rho )}(x) \, \mathrm {d}x \int _{-\infty }^{t} f_{A}^{(\theta , \rho )}(x) \, \mathrm {d}x. \end{aligned}$$

(4)

Therefore, $\mathcal {E}_A(\theta )$ can be seen as a landscape representing the local enlacement values of the points outside of the object A. This image can be normalized into the [0, 1] range of values in order to be interpreted as a fuzzy set, which we call a Fuzzy Directional Enlacement Landscape (Fuzz-DEL) of the object:

$$\begin{aligned} \mu _{\mathcal {E}}^{A}(\theta )(\rho , t) = \frac{\mathcal {E}_A(\theta )(\rho , t)}{\underset{\rho , t}{\max } \, \mathcal {E}_A(\theta )(\rho , t)}. \end{aligned}$$

(5)

Such a landscape allows to assess and visualize to which degree each point is enlaced by the object A in a fixed direction $\theta$. It is interesting to note that the non-zero values of this landscape are necessarily located inside the object’s concavities. This is particularly interesting from an algorithmic point of view, since it allows to restrict the computation to points located in the convex hull of A (and outside of A). Another point to highlight is that enlacement landscapes are symmetric, with period $\pi$ (i.e., $\mu _{\mathcal {E}}^{A}(\theta + \pi ) = \mu _{\mathcal {E}}^{A}(\theta )$).

Since $\mu _{\mathcal {E}}^{A}(\theta )$ is focused on a single direction, we propose to aggregate such fuzzy landscapes across multiple orientation angles. Let $\theta \in [0, \pi ]$ be an orientation angle and $\omega \in [0, \pi ]$ a width parameter. The Fuzz-DEL on the interval $[\theta - \frac{\omega }{2}, \theta + \frac{\omega }{2}]$ is defined as follows:

$$\begin{aligned} \mu _{\mathcal {E}}^{A}(\theta , \omega )(\rho , t) = \frac{1}{\omega } \, \int _{\theta - \frac{\omega }{2}}^{\theta + \frac{\omega }{2}} \mu _\mathcal {E}^{A} (\alpha )(\rho , t) \, \mathrm {d}\alpha , \end{aligned}$$

(6)

where $\theta$ represents the direction on which the fuzzy landscape is focused, while $\omega$ controls the width of the interval, allowing to measure either a narrow direction or a more global one. In particular, the landscape that aggregates all directions is denoted by $\widetilde{\mu }_{\mathcal {E}}^{A} = \mu _{\mathcal {E}}^{A}(\frac{\pi }{2}, \pi )$.

Fig. 1.

Fuzzy directional enlacement landscapes of a crisp object A, with a fixed direction $\theta$ and an increasing width $\omega$. In (b, c and d), A is outlined in white.

In order to illustrate such definitions, Figs. 1 and 2 show the Fuzz-DELs obtained for two different objects. On the one hand, Fig. 1 illustrates the impact of the width parameter $\omega$ for a given vertical direction ($\theta = \frac{\pi }{2}$). Note that the landscape would be identical for the opposite vertical direction ($\theta = \frac{3\pi }{2}$) because of symmetry. We can observe the zero-valued points in the center of the object for $\omega = 0$, representing the fact that these points are not enlaced vertically by A. This can be interpreted by the idea that if another object was located here, it would be able to move in the vertical direction without crossing the other object (i.e., the object could slide downwards). We also observe that when $\omega$ increases, the fuzzy landscape progressively gets smoother, taking into account a wider range of directions. On the other hand, Fig. 2 shows enlacement landscapes on another object for different directions $\theta$ (with a fixed width $\omega = \frac{\pi }{3}$). From these examples, one can note how a Fuzz-DEL allows to capture the object directional concavities. In the horizontal direction ($\theta = 0$), the local enlacement values are relatively high, and the values are higher the deeper we get inside the “snaked” shape. In the vertical direction ($\theta = \frac{\pi }{2}$), the Fuzz-DEL is mostly empty, except on some small concavities.

Fig. 2.

Fuzzy directional enlacement landscapes of a crisp object A, with a fixed width $\omega$ and for different directions $\theta$. In (b, c and d), A is outlined in white.

4.2 Fuzzy Evaluation

In the previous definitions, a reference object A is considered, and different Fuzz-DELs can be derived from it. These fuzzy landscapes allow to visualize the interaction area of A. In the following, we show how to exploit these landscapes to evaluate to which degree a target object B is enlaced by the reference object A, using classical fuzzy operators.

Let $\mu _{A}$ and $\mu _{B}$ be two fuzzy sets over $\mathbb {R}^2$. A typical way to evaluate how $\mu _{B}$ matches with $\mu _{A}$ is the necessity-possibility measure. The necessity N and possibility $\varPi$ can be respectively defined as follows:

$$\begin{aligned} \varPi (\mu _{A}, \mu _{B})&= \sup _{x, y} \, t(\mu _{A}(x, y), \mu _B(x, y)), \end{aligned}$$

(7)

$$\begin{aligned} N(\mu _{A}, \mu _{B})&= \inf _{x, y} \, T(\mu _{A}(x, y), 1 - \mu _B(x, y)), \end{aligned}$$

(8)

where t is a fuzzy intersection (t-norm) and T is a fuzzy union (t-conorm). For the rest of this article, the $\min$ and $\max$ operators are chosen for t-norm and t-conorm respectively, but other fuzzy operators could be considered.

In our context, this fuzzy matching measure can be applied to evaluate how a target object B (represented by its membership function $\mu _{B}$) matches with a Fuzz-DEL $\mu _\mathcal {E}^A(\theta , \omega )$ of a reference object A. The necessity-possibility interval $[N(\mu _\mathcal {E}^{A}(\theta , \omega ), \mu _B), \varPi (\mu _\mathcal {E}^{A}(\theta , \omega ), \mu _B)]$ constitutes a fuzzy evaluation of how B is enlaced by A in direction $\theta$, with the necessity being a pessimist point of view, while the possibility represents an optimist point of view. The mean value $M(\mu _\mathcal {E}^{A}(\theta , \omega ), \mu _B)$ can also be considered. This evaluation strategy will be further studied in the upcoming experiments.

5 Experimental Results

We present different illustrative examples to highlight the interest of our approach. These experiments are organized around two main applications. The first one is to evaluate the specific relation “surrounded by”. As mentioned previously, this relation can be considered as a particular case that can be derived from the directional enlacement model. The second application is to evaluate the spatial relation “enlaced by” in a more generic sense, in particular when the reference object has multiple degrees of concavities. We also propose some preliminary results on interlacement landscapes.

Fig. 3.

Examples of typical surrounding configurations (gray: reference object A; white: target object B).

5.1 Surrounding

The “surrounded by” relation is easily apprehended by human perception, but is particularly challenging to evaluate quantitatively. It is usually modeled by the “all directions” point of view, i.e., an object surrounds another object if it is located in all directions. In the following, we adopt the same insight, but we adapt it to the enlacement model: an object is surrounded if it is enlaced by the other object in all directions.

Figure 3 presents characteristic examples of surrounding configurations that we assessed using the proposed fuzzy evaluation strategy. In each image, the reference object A is in gray and the target object B is the white circle.

For this application, we propose a specific way to apply our approach. The target object B is projected into a Fuzz-DEL of A, and further normalized as a fuzzy set. Such a projection is defined as:

$$\begin{aligned} \mu _\mathcal {E}^{AB}(\theta , \omega ) = \frac{\underset{\rho , t}{\min } \, (\mu _\mathcal {E}^A(\theta , \omega )(\rho , t), \mu _B(\rho , t))}{\underset{\rho , t}{\max } \, \mu _\mathcal {E}^A(\theta , \omega )(\rho , t)}. \end{aligned}$$

(9)

Then, the necessity $N(\mu _\mathcal {E}^{AB}(\theta , \omega ), \mu _B)$ and possibility $\varPi (\mu _\mathcal {E}^{AB}(\theta , \omega ), \mu _B)$ evaluations are performed for different values of $\theta \in [0, \pi ]$. This results in informative directional necessity and possibility profiles, which can be also then exploited to derive a global evaluation of how B is surrounded by A. For the rest of this study, we fixed $\omega$ to a low value of $\frac{\pi }{36}$ (5$^\circ$) to take into account different directions individually, while smoothing out some discretization issues. Following the “all directions” point of view, this global evaluation can be obtained with the following:

$$\begin{aligned} N^{AB}_{\mathcal {S}}&= \frac{1}{\pi } \int _0^{\pi } N(\mu _{\mathcal {E}}^{AB}(\theta , \omega ), \mu _{B}) \, \mathrm {d}\theta , \end{aligned}$$

(10)

$$\begin{aligned} \varPi ^{AB}_{\mathcal {S}}&= \frac{1}{\pi } \int _0^{\pi } \varPi (\mu _{\mathcal {E}}^{AB}(\theta , \omega ), \mu _{B}) \, \mathrm {d}\theta . \end{aligned}$$

(11)

Fig. 4.

Directional necessity, possibility and mean profiles measuring the surrounding configurations of Fig. 3.

Table 1. Fuzzy surrounding evaluations (necessity-possibility intervals and mean values) obtained for the configurations of Figs. 3 and 5.

Figure 4 shows the directional necessity and possibility profiles obtained for the configurations of Fig. 3. In situation (a), the object is only partially surrounded. Both the pessimist and possibility evaluations agree that the reference object A is surrounded in the vertical directions, but not in the horizontal directions. The gradual transition of the situation is captured along the diagonal directions. The object is also partially surrounded in situation (b), where half of the surrounding circle has been cut out. In situation (c), small parts were added preventing the object to leave without crossing the surrounding object, and therefore the optimist point of view is 1 while the pessimist one oscillates but is never zero. Finally, evaluations tend to agree that the object is surrounded in situation (d). The optimist point of view is always 1, yet it takes into account that the object could escape by crossing a small portion of the surrounding object, resulting in low pessimist evaluations for the vertical directions.

Fig. 5.

Applicative example of a complex surrounding configuration. The satellite image represents the Bassin d’Arcachon (France). (b) Object A is gray and object B is white. (c–f) A is outlined in white and B is outlined in red. (Color figure online)

Fig. 6.

Directional necessity, possibility and mean profiles measuring the surrounding of the island by the bay in Fig. 5.

To complement these results, Table 1 presents the surrounding necessity-possibility intervals $[N_{\mathcal {S}}^{AB}, \varPi _{\mathcal {S}}^{AB}]$, to evaluate the global surrounding of the target objects of Fig. 3. For comparison purposes, we also present the results of the approach of [16], which is also based on a fuzzy landscape framework, but dedicated to surrounding relation. It is based on a specific fuzzy landscape, considering only the visible concavities of the reference object. We also present the results obtained by [6] with the initial enlacement descriptors. Considering the fact that surrounding evaluations are highly subjective, our goal here is not to argue that an approach is better than another, but to illustrate that the proposed Fuzz-DELs can provide interesting point of views regarding this surrounding spatial relation.

To show the potential of our approach on real data, we evaluated the “surrounded by” relation on geographical objects extracted from a satellite image (Fig. 5(a)). This image¹ represents the Bassin d’Arcachon (France) and has been acquired by the Formosat-2 satellite. The image was segmented to produce a 3-class image (Fig. 5(b)) composed of an island enclosed into the bay (reference object A) and the land coast (target object B). For illustrative purposes, Fig. 5(c–f) present the Fuzz-DELs of the bay object for different directions $\theta$ and widths $\omega$. In particular, (c) shows the overall landscape $\widetilde{\mu }_{\mathcal {E}}^{A}$ that aggregates all directions, and (e) shows the direction where the target object is the least enlaced (i.e., for $\theta = \frac{\pi }{3})$. The related directional necessity and possibility profiles are shown in Fig. 6, and the respective fuzzy surrounding evaluations $[N_{\mathcal {S}}^{AB}, \varPi _{\mathcal {S}}^{AB}]$ are reported in Table 1.

Fig. 7.

Fuzzy enlacement landscape of a spiral (reference object A) and evaluation for different target objects inside the spiral (represented in different colors). (Color figure online)

5.2 Global Enlacement

To pursue our study and to go further the surrounding spatial relation, we consider in a more generic sense the spatial relation “enlaced by”, in particular when the reference object has multiple degrees of concavities. Figure 7 (a) presents a complex spatial configuration involving a spiral and different target objects enclosed into it, from the center of the spiral to its “tail”. The spiral is the reference object A, and we consider here its Fuzz-DEL $\widetilde{\mu }_{\mathcal {E}}^{A}$ that aggregates all directions. From this landscape (Fig. 7 (b)), we can observe the decreasing pattern (from white pixels to dark gray pixels) as we shift away from the center of the spiral. To assess this behavior, Fig. 7 (c) presents the intervals $[N(\widetilde{\mu }_\mathcal {E}^{A}, \mu _B), \varPi (\widetilde{\mu }_\mathcal {E}^{A}, \mu _B)]$ measuring the global enlacement for the different target objects inside the spiral. Note that other surrounding approaches cannot take into account the depth within the spiral. For instance, the approach of [16] provides the same evaluations for the green, blue, yellow and pink objects (i.e., around 0.50), because it does not consider the reference object as a whole, but only looks at the visible concavities from the target object.

Fig. 8.

Examples of fuzzy interlacement landscapes (mapped into a “heat” color scale) obtained for different images (in (b), white: object A; gray: object B). (Color figure online)

5.3 Towards Fuzzy Interlacement Landscapes

We also propose some preliminary results on interlacement landscapes. The term interlacement is intended as a mutual enlacement of two objects. If we aggregate all directions, a fuzzy interlacement landscape between two objects A and B can be obtained by: $\widetilde{\mu }_{\mathcal {I}}^{AB}= \widetilde{\mu }_{\mathcal {E}}^{A} + \widetilde{\mu }_{\mathcal {E}}^{B}.$ Fig. 8 shows the fuzzy interlacement landscapes obtained for two illustrative images, which have been respectively segmented into 3 classes. The first landscape is obtained from an image of a zebra whose coat features an alternating stripes pattern. We can observe the high interlacement values concentrated in the center of the animal’s coat. The second landscape is obtained from a ASTER satellite image² covering a large delta river. Notice that the interlacement is mainly located around the ramifications between the river and the mangrove. Such interlacement visualization could be useful for instance for ecological landscape monitoring.

6 Conclusion

We introduced a generic fuzzy model for the evaluation of complex spatial configurations of binary objects represented in images. In particular, we focused on the enlacement spatial relation, which can be considered as a generalization of the notions of surrounding and imbrication of objects. Based on the directional enlacement model [6], our proposed evaluation approach exploits the concept of fuzzy landscapes to assess the enlacement of objects in the image space from a local point of view. An experimental study carried out on different illustrative examples highlighted the interest of this model to evaluate complex spatial relations. In future works, we plan to further study how to exploit fuzzy interlacement landscapes, in particular with overlapping objects. We also plan to extend the model by integrating a measure of spacing in interlacement configurations, allowing to better take into account the distance between the objects.