Confidence interval approach to feature re-weighting

Abstract

Relevance feedback is commonly incorporated into content-based image retrieval systems with the objective of improving retrieval accuracy via user feedback. One effective method for improving retrieval performance is to perform feature re-weighting based on the obtained feedback. Previous approaches to feature re-weighting via relevance feedback assume the feature data for images can be represented in fixed-length vectors. However, many approaches are invalidated with the recent development of features that cannot be represented in fixed-length vectors. In addition, previous approaches use only the information from the set of images returned in the latest query result for feature re-weighting. In this paper, we propose a feature re-weighting approach that places no restriction on the representation of feature data and utilizes the aggregate set of images returned over the iterations of retrieval to obtain feature re-weighting information. The approach analyzes the feature distances calculated between the query image and the resulting set of images to approximate the feature distances for the entire set of images in the database. Two-sided confidence intervals are used with the distances to obtain the information for feature re-weighting. There is no restriction on how the distances are calculated for each feature. This provides freedom for how the feature representations are structured. The experimental results show the effectiveness of the proposed approach and in comparisons with other work, it is shown that our approach outperforms previous work.

Appendix

1.1 Appendix 1 Proof of Theorem 1

Since the proof for Eq. 12 is straightforward, we will omit the proof for Eq. 12.

Every possible case for the placement of two confidence intervals in the negative range must be considered. Consider Fig. 10a where both bounds for interval 2 are greater than interval 1. Then shift the lower bound of interval 2 to be in the range of interval 1 to obtain case (b). Next, further shift the lower bound of interval 2 so that it is more negative than interval 1 to obtain case (c). All cases where the upper bound of interval 2 is greater than interval are covered. Thus, shift the upper bound of interval 2 into the range of interval 1. If the lower bound of interval 2 is also placed inside the range of interval 1, case (d) is obtained. Then, shift the lower bound of interval 2 so that it is more negative than interval 1 to obtain case (e). Finally, shift the upper bound of interval 2 to be more negative than interval 1, and this leaves only the possibility of the lower bound of interval 2 being more negative than interval 1 as displayed in case (f). As can be seen, case (c) and (d) are the same, as well as case (b) and (e), and likewise for cases (a) and (f). This leaves three cases that must be checked.

• Case 1: Independent (Fig. 10a and f). Figure 11 illustrates the case where there are two confidence intervals that do not overlap.

Fig. 10

Possible cases for confidence intervals in the negative range

Fig. 11

Confidence intervals with no overlap

Let

• 1. lb₁ + x = lb₂,

• 2. ub₁ + y = ub₂

where

• 3. −1 < lb₁ < ub₁ ≤ 0,

• 4. −1 < lb₂ < ub₂ ≤ 0,

• 5. x > 0,

• 6. y > 0.

Then

$$\begin{array}{*{20}c} {\frac{{lb_1 + ub_1 }}{2} < \frac{{lb_2 + ub_2 }}{2} \Leftrightarrow 1 + \frac{{\left| {ub_1 } \right|}}{{1 - \left| {lb_1 } \right|}} > 1 + \frac{{\left| {ub_2 } \right|}}{{1 - \left| {lb_2 } \right|}}} \\ {\operatorname{ub} _1 + lb_1 < ub_2 + lb_2 \Leftrightarrow 1 + \frac{{\left| {ub_1 } \right|}}{{1 - \left| {lb_1 } \right|}} > 1 + \frac{{\left| {ub_2 } \right|}}{{1 - \left| {lb_2 } \right|}}} \\ {\operatorname{ub} _1 + lb_1 < ub_2 + lb_2 \Leftrightarrow \frac{{\left| {ub_1 } \right|}}{{1 - \left| {lb_1 } \right|}} > \frac{{\left| {ub_2 } \right|}}{{1 - \left| {lb_2 } \right|}}} \\ {\operatorname{ub} _1 + lb_1 < ub_2 + lb_2 \Leftrightarrow \frac{{\left| {ub_1 } \right|}}{{1 + lb_1 }} > \frac{{\left| {ub_2 } \right|}}{{1 + lb_2 }}} \\ {\operatorname{ub} _1 + lb_1 < ub_2 + lb_2 \Leftrightarrow \frac{{ub_1 }}{{1 + lb_1 }} < \frac{{ub_2 }}{{1 + lb_2 }}} \\ \end{array} $$

(16)

$$\operatorname{ub} _1 + ub_1 lb_2 <ub_2 + ub_2 lb_1 $$

$\operatorname{ub} _1 + ub_1 lb_1 + ub_1 x <\operatorname{ub} _2 + ub_2 lb_1$ by substituting (1) $\operatorname{ub} _1 + ub_1 lb_1 + ub_1 x <\operatorname{ub} _1 + y + \operatorname{ub} _1 lb_1 + lb_1 y$ by substituting (2) $\operatorname{ub} _1 x <y + \operatorname{lb} _1 y$

$$0 <y + \operatorname{lb} _1 y - \operatorname{ub} _1 x$$

$0 <y\left( {1 + lb_1 } \right) - \operatorname{ub} _1 x$ is true considering (3), (4), (5), (6).

• Case 2: Overlapping (Fig. 10b and e)

Figure 12 illustrates the case where there are two confidence intervals that have partial overlap.

Fig. 12

Overlapping confidence intervals

Let

• 1. lb₁  +  x  = lb₂,

• 2. ub₁  +  y  = ub₂

where

• 3. −1 < lb₁ < ub₁ ≤ 0,

• 4. −1 < lb₂ < ub₂ ≤ 0,

• 5. x > 0,

• 6. y > 0.

Starting from Eq. 16,

$$\operatorname{ub} _1 + ub_1 lb_2 <ub_2 + ub_2 lb_1 $$

$\operatorname{ub} _1 + ub_1 lb_1 + ub_1 x <ub_2 + ub_2 lb_1$ by substituting (1)$\operatorname{ub} _1 + ub_1 lb_1 + ub_1 x <ub_1 + y + ub_1 lb_1 y$ by substituting (2)

$$\operatorname{ub} _1 x <y + \operatorname{lb} _1 y$$

$$0 <y + \operatorname{lb} _1 y - \operatorname{ub} _1 x$$

$0 <y(1 + \operatorname{lb} _1 ) - \operatorname{ub} _1 x$ is true considering (3), (4), (5), (6).

• Case 3: Contained (Fig. 10c and d)

Figure 13 illustrates the case where one confidence interval is fully contained in another confidence interval.

Fig. 13

Confidence interval contained in another

Let

• 1. ub₁ + x = ub₂

where

• 2. −1 < lb₁ < ub₁ ≤ 0,

• 3. −1 < lb₂ < ub₂ ≤ 0,

• 4. x > 0.

Starting from Eq. 16,

$$\operatorname{ub} _1 + \operatorname{ub} _1 lb_2 <ub_2 + ub_2 lb_1 $$

$\operatorname{ub} _1 + ub_1 lb_2 <ub_1 + x + \operatorname{ub} _1 lb_1 + lb_1 x$ by substituting (1)$0 <x\left( {1 + \operatorname{lb} _1 } \right)$ is true considering (2), (3), (4).

1.2 Appendix 2 MPEG-7 Core Experiment CE-Shape-1 Part B

The following is a subset of the images from the MPEG-7 Core Experiment CE-Shape-1.