Advertisement

RGB-T Saliency Detection Benchmark: Dataset, Baselines, Analysis and a Novel Approach

  • Guizhao Wang
  • Chenglong Li
  • Yunpeng Ma
  • Aihua Zheng
  • Jin Tang
  • Bin Luo
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 875)

Abstract

Despite significant progress, image saliency detection still remains a challenging task in complex scenes and environments. Integrating multiple different but complementary cues, like RGB and Thermal (RGB-T), may be an effective way for boosting saliency detection performance. The current research in this direction, however, is limited by the lack of a comprehensive benchmark. This work contributes such a RGB-T image dataset, which includes 821 spatially aligned RGB-T image pairs and their ground truth annotations for saliency detection purpose. With this benchmark, we propose a novel approach, graph-based multi-task manifold ranking algorithm, for RGB-T saliency detection. Extensive experiments against the baseline methods on the benchmark dataset demonstrate the effectiveness of the proposed approach.

Keywords

RGB-T benchmark Saliency detection Cross-modality consistency Graph-based manifold ranking Multi-task learning 

Notes

Acknowledgement

This work was supported in part by the Natural Science Foundation of Anhui Higher Education Institution of China under Grants KJ2017A017, and in part by the National Natural Science Foundation of China under Grants No. 61602006 and No. 61671018, and in part by the Co-Innovation Center for Information Supply & Assurance Technology, Anhui University under Grant Y01002449.

Supplementary material

470895_1_En_36_MOESM1_ESM.pdf (1.7 mb)
Supplementary material 1 (pdf 1786 KB)
470895_1_En_36_MOESM2_ESM.pdf (8 kb)
Supplementary material 2 (pdf 7 KB)

References

  1. 1.
    Conaire, C.O., O’Connor, N., Cooke, E., Smeaton, A.F.: Comparison of fusion methods for thermo-visual surveillance tracking. In: Proceedings of the International Conference on Information Fusion (2006)Google Scholar
  2. 2.
    Desingh, K., Krishna, K.M., Rajan, D., Jawahar, C.: Depth really matters: improving visual salient region detection with depth. In: BMVC (2013)Google Scholar
  3. 3.
    Erdem, E., Erdem, A.: Visual saliency estimation by nonlinearly integrating features using region covariances. J. Vis. 13(4) (2013)Google Scholar
  4. 4.
    Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(10), 1915–1926 (2012)CrossRefGoogle Scholar
  5. 5.
    Gopalakrishnan, V., Hu, Y., Rajan, D.: Random walks on graphs for salient object detection in images. IEEE Trans. Image Process. 19(12), 3232–3242Google Scholar
  6. 6.
    Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Proceedings of the Advances in Neural Information Processing Systems (2006)Google Scholar
  7. 7.
    Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2007)Google Scholar
  8. 8.
    Itti, L., Koch, C., Niebur, E., et al.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)CrossRefGoogle Scholar
  9. 9.
    Li, C., Yuan, Y., Cai, W., Xia, Y., Dagan Feng, D.: Robust saliency detection via regularized random walks ranking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  10. 10.
    Li, C., Cheng, H., Hu, S., Liu, X., Tang, J., Lin, L.: Learning collaborative sparse representation for grayscale-thermal tracking. IEEE Trans. Image Process. 25(12), 5743–5756 (2016)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Peng, H., Li, B., Xiong, W., Hu, W., Ji, R.: RGBD salient object detection: a benchmark and algorithms. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 92–109. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10578-9_7CrossRefGoogle Scholar
  12. 12.
    Qin, Y., Lu, H., Xu, Y., Wang, H.: Saliency detection via cellular automata. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)Google Scholar
  13. 13.
    Qu, L., He, S., Zhang, J., Tian, J., Tang, Y., Yang, Q.: RGBD salient object detection via deep fusion. IEEE Trans. Image Process. 26(5), 2274–2285 (2017)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Rahtu, E., Kannala, J., Salo, M., Heikkilä, J.: Segmenting salient objects from images and videos. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6315, pp. 366–379. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-15555-0_27CrossRefGoogle Scholar
  15. 15.
    Seo, H.J., Milanfar, P.: Static and space-time visual saliency detection by self-resemblance. J. Vis. 9(12), 15 (2009)CrossRefGoogle Scholar
  16. 16.
    Tavakoli, H.R., Rahtu, E., Heikkilä, J.: Fast and efficient saliency detection using sparse sampling and kernel density estimation. In: Heyden, A., Kahl, F. (eds.) SCIA 2011. LNCS, vol. 6688, pp. 666–675. Springer, Heidelberg (2011).  https://doi.org/10.1007/978-3-642-21227-7_62CrossRefGoogle Scholar
  17. 17.
    Torralba, A., Efros, A.: Unbiased look at dataset bias. In: IEEE Conference on Computer Vision and Pattern Recognition (2011)Google Scholar
  18. 18.
    Tu, W.C., He, S., Yang, Q., Chien, S.Y.: Real-time salient object detection with a minimum spanning tree. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)Google Scholar
  19. 19.
    Wang, Q., Zheng, W., Piramuthu, R.: Grab: visual saliency via novel graph model and background priors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 535–543 (2016)Google Scholar
  20. 20.
    Yan, Q., Xu, L., Shi, J., Jia, J.: Hierarchical saliency detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2013)Google Scholar
  21. 21.
    Yang, C., Zhang, L., Lu, H.: Graph-regularized saliency detection with convex-hull-based center prior. IEEE Sig. Process. Lett. 20(7), 637–640 (2013)CrossRefGoogle Scholar
  22. 22.
    Yang, C., Zhang, L., Lu, H., Ruan, X., Yang, M.H.: Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2013)Google Scholar
  23. 23.
    Zhou, D., Weston, J., Gretton, A., Bousquet, O., Scholkopf, B.: Ranking on data manifolds. In: Proceedings of Neural Information Processing Systems (2004)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  • Guizhao Wang
    • 1
  • Chenglong Li
    • 1
  • Yunpeng Ma
    • 1
  • Aihua Zheng
    • 1
  • Jin Tang
    • 1
  • Bin Luo
    • 1
  1. 1.School of Computer Science and TechnologyAnhui UniversityHefeiChina

Personalised recommendations