The Power of Distance Distributions: Cost Models and Scheduling Policies for Quality-Controlled Similarity Queries

Abstract

Approximate similarity queries are a practical way to obtain good, yet suboptimal, results from large data sets without having to pay high execution costs. In this paper we analyze the problem of understanding how the strategy for searching through an index tree, also called scheduling policy, can influence costs. We consider quality-controlled similarity queries, in which the user sets a quality (distance) threshold $\theta$ and the system halts as soon as it finds k objects in the data set at distance $\le \theta$ from the query object. After providing experimental evidence that the scheduling policy might indeed have a high impact on paid costs, we characterize the policies’ behavior through an analytical cost model, in which a major role is played by parameterized local distance distributions. Such distributions are also the key to derive new scheduling policies, which we show to be optimal in a simplified, yet relevant, scenario.

A Guaranteeing Quality of Results

In this Appendix, we show how the threshold $\theta$ of a quality-controlled query can be chosen so as to provide probabilistic guarantees on the quality of results, extending the results presented in [CP00] to the case $k>1$. The approximate result $\widetilde{\mathcal {R}}$ of a query is a list of $k$ objects that may, however, not be the $k$ closest ones to the query $q$. A possible way to define the quality of $\widetilde{\mathcal {R}}$ is in terms of the relative error $Err$ wrt the exact result $\mathcal {R}$. Let us denote the i-th NN of a query $q$ in a set of objects $X$ as $p^{i}_{X} \left( q \right)$ and with $\widetilde{p}^{i}_{X} \left( q \right)$ the i-th NN of $q$ in $\widetilde{\mathcal {R}}$. In the (simplest) case presented in [CP00], when $k=1$, the error is computed as:

$$\begin{aligned} Err= \frac{d\left( q, \widetilde{p}^{1}_{X} \left( q \right) \right) }{d\left( q, p^{1}_{X} \left( q \right) \right) } - 1 \end{aligned}$$

(11)

This can be extended to the case $k> 1$ by using the error on the $k$-th nearest neighbor (see also [AM+98, ZS+98]):

$$\begin{aligned} \boxed { Err\mathop {=}\limits ^{\text {def}}\frac{d\left( q, \widetilde{p}^{k}_{X} \left( q \right) \right) }{d\left( q, p^{k}_{X} \left( q \right) \right) } - 1 } \end{aligned}$$

(12)

which reduces to Eq. 11 when $k=1$.

The type of guarantee provided on the quality of results in [CP00] has the form: “with probability at least $1 - \delta$ the error does not exceed $\epsilon$”, that is $\Pr \left\{ \mathbf {Err} \le \epsilon \right\} \ge 1 - \delta$, where $\epsilon \ge 0$ is an accuracy parameter, $\delta \in [0,1)$ is a confidence parameter, and $\mathbf {Err}$ is the random variable obtained from Eq. 12 applied to a random query $\mathbf {q}$. In [CP00], this is computed by using $G_{}^{} \left( x \right)$, i.e., the distance distribution of the 1-NN, which can be obtained from the distance distribution $F_{} \left( \cdot \right)$ as:

$$\begin{aligned} G_{}^{} \left( x \right) \mathop {=}\limits ^{\text {def}}\Pr \left\{ d\left( \mathbf {q}, p^{1}_{X} \left( \mathbf {q} \right) \right) \le x \right\} = 1 - \left( 1 - F_{} \left( x \right) \right) ^N\end{aligned}$$

(13)

For $k\ge 1$, this generalizes to $G_{}^{k} \left( x \right)$, i.e., the probability to find at least $k$ objects at a distance $\le x$, which can be computed as:

$$\begin{aligned} G_{}^{k} \left( x \right) \mathop {=}\limits ^{\text {def}}\Pr \left\{ d\left( \mathbf {q}, p^{k}_{X} \left( \mathbf {q} \right) \right) \le x \right\} = 1 - \sum _{j=0}^{k-1} \left( {\begin{array}{c}N\\ j\end{array}}\right) \cdot F_{} \left( x \right) ^j \cdot \left( 1 - F_{} \left( x \right) \right) ^{N- j} \end{aligned}$$

(14)

The probability that the result of an approximate $k$-NN query $\theta$ is correct, i.e., that the approximate k-th NN, $\widetilde{p}^{k}_{X} \left( q \right)$, is indeed the correct one, is given by $G_{}^{k} \left( d\left( q, \widetilde{p}^{k}_{X} \left( q \right) \right) \right)$. Since we want to bound the error with confidence $1-\delta$, we obtain the following guarantee on the error:

$$\begin{aligned} Err\le \epsilon = \frac{d\left( q, \widetilde{p}^{k}_{X} \left( q \right) \right) }{\sup \left\{ x | G_{}^{k} \left( x \right) \le \delta \right\} } - 1 \end{aligned}$$

because the probability that $d\left( q, p^{k}_{X} \left( q \right) \right) < \sup \left\{ x | G_{}^{k} \left( x \right) \le \delta \right\}$ is less than $\delta$. The (one-sided) confidence interval corresponding to $1-\delta$ is therefore $[0,\epsilon ]$. Thus, the threshold $\theta$ for a quality-controlled query can be obtained as $(1+\epsilon ) \sup \left\{ x | G_{}^{k} \left( x \right) \le \delta \right\}$. Clearly, if $G_{}^{k} \left( \right)$ is invertible, it is: $\sup \left\{ x | G_{}^{k} \left( x \right) \le \delta \right\} = (G^{k})^{-1}(\delta )$.