Contextualizing Tag Ranking and Saliency Detection for Social Images

Abstract

Tag ranking and saliency detection are two key tasks for image understanding, and have attracted much attention in the past decades. In this paper, we investigate how to iteratively and mutually boost tag ranking and saliency detection by taking the outputs from one task as the context of the other one. Our method first computes an initial saliency value based on fusing multiple feature maps, and then iteratively refines saliency map based on the contextual information from image tag ranking. As a result, an integrated framework for tag saliency ranking which combines both visual attention model and multi-instance learning to investigate the saliency ranking order information. We show that this mutual reinforcement of saliency detection and tag ranking improves the performance by using this combined approach. Experiments conducted on Corel and Flickr image datasets demonstrate the effectiveness of the proposed framework.