Cost-effective conceptual design using taxonomies

Abstract

It is known that annotating entities in unstructured and semi-structured datasets by their concepts improves the effectiveness of answering queries over these datasets. Ideally, one would like to annotate entities of all relevant concepts in a dataset. However, it takes substantial time and computational resources to annotate concepts in large datasets, and an organization may have sufficient resources to annotate only a subset of relevant concepts. Clearly, it would like to annotate a subset of concepts that provides the most effective answers to queries over the dataset. We propose a formal framework that quantifies the amount by which annotating entities of concepts from a taxonomy in a dataset improves the effectiveness of answering queries over the dataset. Because the problem is $\mathbf {NP}$-hard, we propose efficient approximation and pseudo-polynomial time algorithms for several cases of the problem. Our extensive empirical studies validate our framework and show accuracy and efficiency of our algorithms.

Appendices

Appendix

Proofs

Proof of Theorem 1

The problem of CECD can be reduced to the problem of choosing a cost-effective design from a set of concepts by creating a taxonomy $\mathcal {X}=(R,\mathcal {C},\mathcal {R})$ where all nodes except for R are leaf concepts. Since the problem of choosing cost-effective concepts from a set of concepts is $\mathbf {NP}$-hard [50], CECD is also $\mathbf {NP}$-hard. $\square$

Proof of Lemma 1

We have ${\sum \limits _{C\in \texttt {free}(\mathcal {S})} u(C)d(C) \le u(C_{\max })} \sum \limits _{C\in \texttt {free}(\mathcal {S})}d(C)$. Since the frequencies of leaf concepts in $\mathcal {X}$ follow a power-law distribution, we have that $\sum \limits _{C \in \texttt {leaf}(\mathcal {C})} d(C) \le 1+\log (|\texttt {leaf}(\mathcal {C})|)$ where $\texttt {leaf}(\mathcal {C})$ is the set leaf concepts in $\mathcal {C}$ and $|\texttt {leaf}(\mathcal {C})|$ is the number of such concepts. Since $|\texttt {leaf}(\mathcal {C})|$ $\le |\mathcal {C}|$, we have that $QU(\texttt {free}(\mathcal {S})) \le \sum \limits _{C\in \texttt {free}(\mathcal {S})} u(C)d(C) \le (1 + \log |\mathcal {C}|)\ u(C_{\max }) \le 2u(\mathcal {C}_{\max })\log |\mathcal {C}|$. $\square$

Proof of Theorem 2

Let $\mathcal {S}^*$ be a disjoint design over $\mathcal {X}$ with total cost at most B that maximizes QU function. Let $\mathcal {S}^*[i]$ be the set of concepts in $S^*$ of level i, i.e., $\mathcal {S}^*[i] =\mathcal {S}^*\cap \mathcal {C}[i]$. Since $\mathcal {S}^*$ is a disjoint design, for all $1\le i, j\le h$, $\texttt {part}(\mathcal {S}^*[i]) \cap \texttt {part}(\mathcal {S}^*[j]) =\emptyset$. Thus, $QU(\mathcal {S}^*) =\sum \limits _{1\le i\le h}{QU(\mathcal {S}^*[i])} + QU(\texttt {free}(\mathcal {S}^*))$. Next, we consider the two possible cases. First, assume that $\sum _{i=1}^h$ $QU(\mathcal {S}^*[i])$ $\ge QU(\texttt {free}(\mathcal {S}^{*}))$. It follows that the Level-wise algorithm returns a $(\frac{2h}{\texttt {pr}_{\min }})$-approximate solution of the disjoint CECD defined over $\mathcal {X}$.

In the other case in which $QU(\texttt {free}(\mathcal {S}^{*}))$ is larger than the other term, by Lemma 1, extracting the concept with the maximum u value gives a $(\frac{4\log (|\mathcal {C}|)}{\texttt {pr}_{\min }})$-approximation. These two cases together imply that we have an $O({h+\log |\mathcal {C}|\over \texttt {pr}_{\min }})$-approximation. $\square$

Proof of Theorem 4

Construct an instance $\mathcal {M}$ of SU-Knapsack from the MC-CECD instance as described in the preceding paragraph of Theorem 4.

For each pair of concepts $C_i, C_j$, the total cost of $\mathcal {I}_1 := (\{ {C_i} \},\{ {C_j} \})$ is equal to the total cost of $\mathcal {I}_2 := (\{ {C_i} \},\{ {C_j} \},\{ {C_i,C_j} \})$, and all elements have positive profits. If items $\{ {C_i} \}, \{ {C_j} \}$ are selected in an optimal solution of $\mathcal {M}$, item $\{ {C_i,C_j} \}$ will also be picked. Thus, an optimal solution of $\mathcal {M}$ corresponds to a design $\mathcal {S}$ in MC-CECD: $\mathcal {I}_{\mathcal {S}} =\emptyset \cup \bigcup _{C \in \mathcal {S}} \{ {C} \} \cup \bigcup _{C_1, C_2 \in \mathcal {S}} \{ {C_1, C_2} \}$. Furthermore, the profit of $\mathcal {I}_{\mathcal {S}}$ is: $p(\mathcal {I}_{\mathcal {S}}) =p(\emptyset ) + \sum _{C\in \mathcal {S}} p(\{ {C} \}) + \sum _{C_1, C_2\in \mathcal {S}} p(\{ {C_1, C_2} \}) =QU(\mathcal {S})$. Hence, the algorithm of [5] returns a design whose queribility is at least $(1-1/e^{\frac{1}{k+1}})$ times the queribility of an optimal design. $\square$

Proof of Theorem 5

We use the following construction to reduce Densest-k-Subgraph to CECD problem over DAG. Given a graph $G =(V,E)$ and a number k, we build an instance of the CECD over a DAG taxonomy as follows. For each edge $e \in E$, we introduce a leaf concept $a_e$, and for each vertex $v \in V$, we introduce a leaf concept $a_v$ and a non-leaf concept $S_v$ such that $S_v$ is an ancestor of $a_v$ and all $a_e$ corresponding to the incident edges of v in G. Further, we set the budget B to k, the cost of each non-leaf concept to 1 and the cost of leaf concepts to $k+1$. If we select $S_v$ and $S_u$ in the design and $(u,v) \in E$, $a_{e}$ will be a singleton partition. We set the popularities and frequencies of all concepts in the taxonomy, respectively, to the same fixed values u and d. Let m be the number of edges in G and n be the number of vertices in G. For each partition $p\in \texttt {part}(\mathcal {S})$, we set $d(p) =1/\beta$ if p is a singleton edge concept and $d(p) =1$ otherwise. $\beta$ is a parameter which we determine later in the proof. Since leaf concepts are not affordable, there is an optimal design with exactly k non-leaf concepts. In each design $\mathcal {S}$ of size k, the contribution of every leaf concept in a non-singleton edge concept partition is exactly ud. Let $H_{\mathcal {S}}$ be the set of vertices in G whose corresponding non-leaf concepts in $\mathcal {C}$ are in $\mathcal {S}$. $E(H_{\mathcal {S}})$ denotes the set of edges with both endpoints in $H_{\mathcal {S}}$. It corresponds to the set of edge concepts of $\mathcal {C}$ whose both non-leaf concepts associated with their endpoints are in $\mathcal {S}$. Let $\mathcal {S}_{\textsc {OPT}}$ be the solution of CECD corresponding to an optimal solution of the Densest-k-Subgraph. We have $QU(\mathcal {S}_{\textsc {OPT}}) =ud(\beta \cdot t + m + n -t)$, where t denotes the number of edges in $H_{\mathcal {S}_{\textsc {OPT}}}$. Let $\mathcal {A}$ be an $\alpha$-approximation algorithm of CECD and $\mathcal {S}_{\mathcal {A}}$ be the $\alpha$-approximate design returned by $\mathcal {A}$. Let $QU(\mathcal {S}_{\mathcal {A}}) =ud (t'\cdot \beta + m + n -t')$, where $t'$ is the number of edges whose both endpoints are in $\mathcal {S}_{\mathcal {A}}$. Since $\mathcal {A}$ is an $\alpha$-approximation algorithm of CECD, $t\cdot \beta + m + n -t \le \alpha ( t'\cdot \beta + m + n -t')$. Thus, $t \le t'\cdot \alpha + (m+n)\cdot {\alpha -1 \over \beta -1}$. Setting $\beta =(m+n)(\alpha -1) + 1$ leads to $O(\alpha )$-approximation for the Densest-k-Subgraph. $\square$

Proof of Corollary 1

By setting $\beta = {(m+n)(\alpha -1)\over \epsilon } + 1$ in proof of Theorem 5 and using the hardness result of [32], no polynomial time approximation scheme algorithm exists for CECD unless $\mathbf {NP}\subseteq \cap _{\epsilon >0} \mathbf {BPTIME}(2^{n^{\epsilon }})$. $\square$

Proof of Corollary 2

It follows from the hardness result of [36] and Theorem 5. $\square$