Group spatiotemporal pattern queries

Abstract

Group spatiotemporal patterns are certain formations, in space and time, shown by groups of moving objects, such as flocks, concurrence, encounter, etc. A large number of recent applications focus on the collective behavior of moving objects, rather than the individual movements. Therefore finding such groups in moving object databases is crucial. There exist, in the literature, smart algorithms for matching some of these patterns. These solutions, however, address specific patterns and require specialized data representation and indexes. They share too little to be integrated into a single system. There is a need for a generic query method that allows users to fill in pattern descriptions, and retrieve the set of matches. In this paper, we propose generic query operators that can consistently express and match a wide range of group spatiotemporal patterns. We formally define these operators, illustrate the evaluation algorithms, and discuss the issues of their integration with moving object database (MOD) systems. These operators have been implemented in the context of Secondo MOD system, and the implementation is available online as open source. Several examples are given to showcase the expressive power of the operators. We have made available scripts that can be invoked from the Secondo interface to automatically repeat some of the experiments in this paper.

Appendix A: The crosspattern evaluation Algorithm

We discuss the evaluation algorithm of the crosspattern operator in the appendix, because of the many details it incorporates. Abstractly speaking, the function of the crosspattern operator is to search for large connected components within a time-dependent graph (i.e., the pattern graph), which is constructed from the evaluations of the time-dependent predicate on pairs of moving objects. As discussed before, the general form of the crosspattern operator is:

$$U~\mathbf{crosspattern}[id_{1},~id_{2},~\alpha,~d,~n, \mathit{subgraph-type}]. $$

The function of the crosspattern operator can be divided into three sub-functions:

• 1. Constructing the pattern graph PG(S ^′, α).

• 2. Searching for the large connected components of type subgraph-type within the pattern graph. Large means that they must fulfill the minimum duration d constraint, and the minimum group cardinality n constraint.

• 3. Representing the found connected components, and yielding them.

1.1 A.1 Related work (REVIEW)

The problem of the crosspattern operator is indeed a generic one, and it has applications in other fields. Generally it helps analyzing the temporal behavior of evolving networks. Social network users and the evolution of their relationships can be modeled by pattern graphs. The α predicate in such a case might be representing whether a pair of users are friends, or whether they contact one another on a daily basis. Other applications are the modeling of the link availability in communication networks, especially wireless and ad-hoc networks, and the link analysis in the evolving web. Such topics are in the focus of several recent workshops.

Up to our knowledge, this problem has not yet been studied in the context of moving object databases. It seems natural however to have a type for time-dependent graphs (moving graphs) and relevant operations defined on top of it. In the context of communication networks, there are works on the so called Evolving Graphs [7] to model network dynamics. An evolving graph is a sequence of snapshots, each of them representing the graph at a single time instant. A recent work [30] proposed a compression technique, so that the space requirements of evolving graphs are affordable. We see two properties of evolving graphs that makes them unsuitable for representing pattern graphs:

• 1. The pattern graph is a continuous mapping from time to graph. The evolving graph is a discrete representation. Clearly the two models are not equivalent.

• 2. An evolving graph requires O(n) storage space, even after applying the compression technique in [30], where n is the number of changes occurring in the graph. In the case of the crosspattern operator, the number of changes normally ranges from 4 to 6 order-of-magnitude. This would easily exceed several GB of memory space.

We are also not informed of works on finding connected components in evolving graphs.

Connectivity queries are studied in the context of the dynamic graphs. A dynamic graph is a standard graph that undergoes a sequence of edge additions or deletions. If both additions and deletions are allowed, it is called a fully dynamic graph. Dynamic graph techniques focus on online graph updates and queries. Only the most recent graph is represented, and no history is kept. A dynamic graph is different from a pattern graph, because the latter needs to represent the history. On the other hand, a dynamic graph can be derived from a pattern graph by means of a temporal scan.

There are smart algorithms for maintaining the minimum spanning forest in fully dynamic graphs [11]. That is, instead of re-evaluating the minimum spanning forest after every graph update (i.e., edge addition or deletion), the algorithm does this evaluation only once in the beginning, and tries to efficiently update the forest after every graph update. Such algorithms can easily be modified to maintain connected components instead of the minimum spanning forest.

There are two main differences that make the problem of the crosspattern operator more complex than the problem of maintaining connected components in fully dynamic graphs:

• 1. The crosspattern operator searches for large connected components in terms of the two constraints d, n as we explained before.

• 2. The crosspattern operator is required to store/represent the result (i.e., the large connected components) as time-dependent graphs. The algorithms for maintaining minimum spanning forests allow only for online queries, and keep no history.

Asur et al. [5] proposed a method for identifying the evolution of groups/communities in time-dependent interaction graphs, such as the merging of several groups, the split of a group, the formation of a new group, or the dissolve of an existing group. The continuous mapping from time to graph was modeled in [22]. The model was used to discover and track the groups on social web applications that appear for short times. This work used the localization characteristics of such groups to restrict the search space within the graph to smaller sets of nodes and smaller time intervals.

The problem of the crosspattern operator combines several sub problems that were separately addressed in the works reviewed in this section. It requires: continuous mapping from time to graph, finding connected components and maintaining them through the graph updates, handling multiple graph updates and simultaneous edge additions and deletions, identifying group evolution events (e.g., split, merge, dissolve), representing the history of the group, and the novel problem of finding large connected components. We describe in the following section an algorithm that solves this problem. It shares concepts from these works. But it implements these concepts in a way that reduces the redundant computations among them and that accommodates them in a common data model. We think that there should be a future study towards implementing an algebra of time-dependent graphs in database systems. Such an algebra should propose an abstract data type, and a set of generic operations.

1.2 A.2 The crosspattern evaluation Algorithm

Algorithm 7 illustrates the evaluation of crosspattern. The first argument is a stream of tuples that corresponds to the set U, defined in Section 6. Every tuple in U has a pair of moving objects, whose identifiers can be queried using the two mappings IDFun ₁, IDFun ₂. The first part of the algorithm (Lines 3–8) constructs the pattern graph, which is represented by the mset instance Accumulator. The Accumulator contains edge identifiers, such that an edge identifier belongs to the Accumulator whenever its corresponding edge exists in the pattern graph. For each tuple in U, the identifiers of the two contained moving objects are used to compute a unique edge identifier by the function NodesToEdge. This function has an inverse EdgeToNodes. The two functions compute the edge identifier given its two nodes’ identifiers, and vice versa in a constant time. The edge itself is computed by evaluating the time-dependent predicate α for the current tuple. Note that the definition time of the edge corresponds to the time periods in which α(u) is true. Finally the edge is constructed using the mbool2mset function (see Algorithm 2), and inserted into the Accumulator.

In the experiments, we have seen that the number of units in the Accumulator, and the number of edge identifiers in every unit are very large. The straightforward implementation of the mset type in Secondo stores the edge identifiers of every unit. Knowing that for a medium input size (e.g., U has 4 order-of-magnitude tuples), the number of units in the Accumulator is 5 order-of-magnitude, and every unit has 2-3 order-of-magnitude edge identifiers. Under such conditions, the Accumulator would require several GB of disk space.⁴ We have implemented a second variant of the mset, that stores in every unit only the changes from the previous unit (i.e., edges that are added or removed). This variant considerably reduces the space requirement, because in most of the time the change is adding or removing a single edge.

We have also implemented the union operation in Line 8 in a different way that works efficiently with this mset implementation. Inside the loop (Lines 3–8), we use a sorted list to buffer events in the form of at time t ₁ the edge e _j starts to be contained in Accumulator, and at time t ₂ the edge e _j ends to be contained in Accumulator. There are two such events for every time interval in which α is fulfilled: one event for joining, and the other for leaving the Accumulator. These events are sorted inside the list by increasing time. Buffering an event is O(log(n)) in the buffer size. After the loop ends, the Accumulator is constructed from the buffer in a single linear scan.

We expect that the most expensive part in constructing the pattern graph will be the evaluation of α(u) (Line 7). This has been experimentally confirmed in Section 10.2. The cost varies according to the associated time-dependent predicate α. The predicate distance(. , .) < . between two mpoint objects, for instance, is O( n + m), where n and m are the number of units of the two arguments. More about time-dependent predicates, and their evaluation algorithms can be found in [8].

The rest of the crosspattern algorithm finds the sub graphs that fulfill the user criteria n, d, q. First the Accumulator is split at the locations where it has definition gaps, if any. The result is a list of mset instances that belong to MSetPart. The search proceeds considering each of these parts separately as a pattern graph, finding the sub graphs in each, and concatenating the results into a single output stream R. We have invested a considerable time in developing a fast algorithm for finding large connected components in the pattern graph. Since the possible values of q are all special kinds of connected components (e.g., clique), we start first by finding the large connected components, then search for sub graphs of kind q within them. At the time of writing this paper, only the algorithm for finding connected components is implemented. So queries looking for clique, walk, etc are not yet available in Secondo.

Finding large connected components is done in two steps as shown in Algorithm 8. The first step finds the connected components (must not be large). The second step filters the nodes and edges of each found component according to the thresholds n, d, and keep only the large components. Note that applying the thresholds can only result in edges being removed from the connected component. It is not possible that edges that do not belong to a connected component be there in the large connected component. Thus the function ApplyThresholds does not need to know about the pattern graph. Rather, it is a function in the connected component, and the two thresholds n and d, in order to find the large connected components. While applying the thresholds, a connected component might split into several, or it might disappear completely if too many nodes and edges are filtered out. Thus, the result of ApplyThresholds is a stream of large components, which is possibly empty.

The function FindConnectedComponents is described in Algorithm 9. It iterates over the units of the mset representing the pattern graph. This iteration is equivalent to a temporal scan of the pattern graph. The result of such a scan is a dynamic graph, as mentioned in Section A.1. Every uset in the pattern graph contains two sets that describe the changes to the graph edges: addedEdges, and removedEdges. The algorithm starts with an empty standard graph g (used to maintain the most recent snapshot of the time-dependent graph), and an empty list of connected components Components. While iterating over the units, the algorithm tries to efficiently maintain g and Components, so that they will be up to date with the last visited unit. The Components list holds only the connected components in g whose number of nodes ≥ n. The nodes of g know the labels of their components if they belong to any, otherwise null. In the beginning of an iteration, g is a snapshot of the pattern graph at the end instant of the unit in the previous iteration, and Components holds the connected components in g. During the iteration, the goal is to update g and Components to reflect the recent changes to the pattern graph, represented by the two sets addedEdges and removedEdges.

Most of the time, an update to the pattern graph is an addition or a removal of a single edge. It occurs less often that an update involves multiple edge additions and/or removals. In the experiment in Section 10.2, 27.3 % of the updates involved multiple edges, while 72.7 % were single edge updates. Algorithm 9 is designed to handle multiple edge updates, because it is the general case.

Newly added edges are first inserted into g. Then the algorithm inspects their effect on the existing components, which can be any of the following:

• 1. Some of the new edges might connect to one another and to other edges in g forming a new component having n nodes or more. This new component is then added to the list Components and marked as NewlyAdded (Line 9 of the algorithm).

• 2. Some of the new edges might connect to one of the existing components causing it to grow up its number of edges (and probably nodes). Such a component is marked as GotMoreEdges (Line 10).

• 3. Some of the new edges might connect several existing components together. These components are marked as Merged, as a preparation to merge them together into a single larger component. This large component will consist of the Merged components and the newly added edges.

• 4. The rest of edges stay in g, and have no effect on the Components list.

These four cases are checked in the same way. A breadth first search that begins from the newly added edges is started to find their entire connected components in g. A search path is terminated if it reaches a node that already belongs to a component in Components. This search is done in linear time in terms of the number of traversed nodes and edges. It traverses at most the newly added edges plus the edges of two components having n − 1 nodes. This bound is reached if it happens that the newly added edge connect two components in the graph having n − 1 nodes each. Notice that the algorithm will not traverse edges of the already maintained components having at least n nodes. Every newly added edge contributes to exactly one of the above cases. After this inspection, every component in the list Components has one of the states: NotChanged, NewlyAdded, GotMoreEdges, or Merged.

The algorithm proceeds to handle edge removals. It iterates over the edges to be removed, and finds in g whether they belong to components. Looking up the component of an edge is O(1) by looking up the component of any of its nodes. The removal of edges might affect the existing components in one of the following ways:

• 1. Edges that belong to no component are removed from g, and have no further effects.

• 2. Edges that belong to components are removed from them. This might result in the dissolve or the split of these components.

The loop in Lines 14–16 collects all the components that are affected by edge removals into the set affectedComponents. The connectivity of each of the affected components is checked, to see whether it is still connected. This is done in O(nodes + edges) of the component.

Now we need to update the states of the affected components. Note that these components might have already states assigned to them in the part handling added edges (this is rather rare). Table 6 illustrates the state composition. Column headers list the possible states after handling added edges, and the row headers list the possible states after handling removed edges. The cells of the table show the result of the composition. In the experiment in Section 10.2, we have counted the occurrences of each of these component states. The percentages are listed in Table 7. These states are required for updating.

Table 6 Composing component states

Table 7 Component states occurrence frequencies

Figure 5 illustrates an example of a graph update that causes the state NewlyAdded followed by RemoveNow. The minimum component size in this example is n = 5. Figure 5 (A) displays the graph before the update. The update adds the edges {(3,1), (6,9)}, and removes the edges {(2,5), (2,6)}. The algorithm handles the added edges first, and the result is a NewlyAdded component having 6 nodes (Fig. 5 (B)). Then the removed edges are handled, and the result is the removal of this component (RemoveNow), because it is split into two small components. Composing NewlyAdded with RemoveNow results in the state RemoveNow for the two small components in Fig. 5 (C). Similarly, Fig. 6 illustrates NewlyAdded followed by Split, and Fig. 7 illustrates Merged followed by Split.

Fig. 5

State NewlyAdded followed by state RemoveNow

Fig. 6

State NewlyAdded followed by state Split

Fig. 7

State Merged followed by state Split

Clearly, situations like these examples hardly occur in real world applications. Consider for examples the application of finding moving clusters within a group of animals. The example in Fig. 5 happens when the two pairs of animals (3,1), (6,9) come close to one another (i.e., below the distance threshold), and the pairs (2,5), (2,6) come apart from one another at exactly the same time instant. It is unlikely that the movement is synchronized in this way.

Now as the algorithm knows the updates that occurred to every component, it propagates these updates to the result stream R ^′. Every mset element in R ^′ represents the history of a connected component in the given pattern graph. At the end of every iteration, the function Finalize appends a unit (uset) to every element in R ^′representing the change that happened to this component in this iteration.

We do not list here the algorithm of Finalize, but we briefly describe it in the following. It maintains two lists of mset. The list R holds the connected components that do not receive updates anymore. These are connected components that used to exist in the past and are currently dissolved. The list R ^′holds the active components that still receive updates. Elements of R ^′ are linked to the elements of the Components list that propagate changes to them. Finalize receives the Components list and iterates over it. According to the state of every component it does one of the following:(FOOTNOTE5)

• 1. A NewlyAdded component results in creating and adding a new mset instance to R ^′. This reflects that a new connected component started to appear in the pattern graph, and that Finalize has to track it. This new mset instance has a single uset having the same time interval as the unit of the pattern graph that is currently being processed, and the set of edges of the new component.

• 2. A component whose state is one of GotMoreEdges, LostEdges, or AddRemoveMix results in extending the associated active component in R ^′by an additional uset having the same time interval as the unit of the pattern graph that is being currently processed, and the set edges in the component after the update.

• 3. A non changed component (i.e., state is NotChanged) results in extending the associated active component in R ^′by an additional uset having the same time interval as the unit of the pattern graph that is being currently processed, and the same set of edges as its last uset instance.⁵

• 4. If the state of the component is RemoveNow, the associated active component is moved from R ^′to R. This is done under the condition that the duration of this active component be ≥ the minimum duration threshold d, otherwise it is ignored by being removed from R ^′, and not put into R.

• 5. If several components merge together (i.e., their state is Merged), this means that they have been separate in the past, but starting from now they share the same set of edges. Finalize extends each of the associated active components by an additional uset having the same time interval as the unit of the pattern graph that is being currently processed, and the set of edges that consists of all the edges in the merged components, plus the newly added edges that caused them to merge.

• 6. If a component is marked as Split, this means that an active component in R ^′splits at this time instant into several smaller components. Finalize makes as many copies of this active component as the number of the split components, and extends every copy with an mset whose time interval is the same as the unit of the pattern graph that is being currently processed, and whose set of edges is the set of edges of one of the split components.⁶

• 7. If a component is marked as ReDistribute, this means that Finalize does not really understand the update. It has no clue which active components should be updated, and how. This state happens only if a component is marked as Merged then as Split in the same iteration, as illustrated in Fig. 7. In such a case, the links between this component and the associated active components might be invalid. Finalize needs to re-investigate the update. It collects in one hand all the components that are marked as ReDistribute, and in the other hand all their associated active components, and it matches the two hands. A component matches an active component if they have common graph nodes. After the matching is done, Finalize is able to assign to every component one of the states above (e.g., Split, GotMoreEdges), and process it accordingly. The ReDistribute state was never observed in the experiment in Section 10.2.

At the end of Finalize, it resets the state of all components in Components to NotChanged, preparing for the next iteration.

Back to Algorithm 8, the next step after finding the connected components is to apply the thresholds n, d to them, in order to yield the large connected components. Please note that these thresholds apply to the nodes of the connected components, and note that the connected components are represented as mset instances that store the edges. Hence, the ApplyThresholds function needs to transform the component it receives from a representation of graph edges into a representation of graph nodes. Using a linear scan of the received component, it builds a hash table in the form shown in Fig. 8 (A), where every node has one entry. An i n t e r v a l _i entry represents one of the intervals on which this NodeID belongs to the connected component. It stores also pointers to the units of the connected components that fall within this interval. ApplyThresholds also builds a map in the form shown in Fig. 8 (B), where Unit is a pointer to one of the units (uset) of the connected component. This map stores the number of nodes in every unit, so that the threshold n can be quickly applied.

Fig. 8

Helper indexes for representing the nodes of the pattern graph

The hash table and the map are linked to the units of the mset representing the connected component, in such a way that the three of them can be quickly synchronized if any of them is changed. ApplyThresholds makes sure that the three structures are always synchronized. To apply the d threshold, it scans the hash table and filters out all the intervals whose duration is less than d. Such a change affects the node count of the units, and it is reflected in the map by means of synchronization. To apply the n threshold, ApplyThresholds scans the map and removes the units that have less than n nodes. Such a change affects the hash table, and it inserts definition gaps within the connected component. ApplyThresholds keeps applying the two thresholds iteratively until no more changes occur. It splits the connected component at the locations where it has definition gaps. The pieces that come out of this split are all large connected components, so they are appended to the result stream.

Back to Algorithm 7 Line 12, the function FindSubGraphs receives this stream of large connected components, and searches for the sub graphs of type q within them. Up to the time of writing this paper, this function has not been implemented. Naively, one would search every unit in the large connected components for sub graphs of type q, and concatenate the results to construct the whole history of the sub graph. Smarter algorithms are still missing, and require further research.

Algorithm 7 has one more missing part. It deals only with undirected pattern graphs. This means that the time-dependent predicate α must be commutative. So far, the group patterns that we have seen in the literature can be expressed using commutative predicates. Most of the patterns are expressed based on the distance between objects, or on a derivative of this distance. Still, the crosspattern operator would be more expressive if it allowed for non-commutative predicates as well. This completes our illustration of the crosspattern algorithm.