Advertisement

A novel multi-graph framework for salient object detection

  • Ye LuEmail author
  • Kedong Zhou
  • Xiyin Wu
  • Penghan Gong
Original Article

Abstract

Graph-based methods have been widely adopted for predicting the most attractive region in an image. Most of the existing graph-based methods only utilize single graph to describe the image information, and thus cannot adapt for complex scenes. In this paper, a novel multi-graph framework for salient object detection is proposed. The proposed method is divided into three steps. Firstly, an image is divided into superpixels and described as a multi-graph, where superpixels are represented as nodes and their information is computed by color space and location space. Secondly, the multiple graphs are combined into a novel multi-graph-based manifold ranking propagation framework to obtain a coarse map. Finally, a map refinement model is developed to improve the quality of the coarse map. Experimental results on four challenging datasets show that the proposed method performs favorably against the state-of-the-art salient object detection methods.

Keywords

Salient object detection Multi-graph framework Map refinement Manifold ranking 

Notes

Acknowledgements

This work was supported by National Natural Science Foundation of China under Grant No. 61602244.

Compliance with ethical standards

Conflict of interest

The authors declared that they have no conflicts of interest to this work.

References

  1. 1.
    Borji, A., Cheng, M.M., Jiang, H., Li, J.: Salient object detection: a benchmark. IEEE Trans. Image Process. 24, 5706–5722 (2015)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Ren, Z., Gao, S., Chia, L.T., Tsang, I.W.H.: Region-based saliency detection and its application in object recognition. IEEE Trans. Circuits Syst. Video Technol. 24, 769–779 (2014)CrossRefGoogle Scholar
  3. 3.
    Guo, C., Zhang, L.: A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE Trans. Image Process. 19, 185–198 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Zhang, J., Feng, S., Li, D., Gao, Y., Chen, Z., Yuan, Y.: Image retrieval using the extended salient region. Inf. Sci. 399, 154–182 (2017)CrossRefGoogle Scholar
  5. 5.
    Zhi, X.-H., Shen, H.-B.: Saliency driven region-edge-based top down level set evolution reveals the asynchronous focus in image segmentation. Pattern Recognit. 80, 241–255 (2018)CrossRefGoogle Scholar
  6. 6.
    Maybank, S.J.: A probabilistic definition of salient regions for image matching. Neurocomputing 120, 4–14 (2013)CrossRefGoogle Scholar
  7. 7.
    Yang, J., Yang, M.-H.: Top-down visual saliency via joint crf and dictionary learning. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 2296–2303 (2012)Google Scholar
  8. 8.
    Hou, X., Zhang, L.: Saliency detection: a spectral residual approach. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)Google Scholar
  9. 9.
    Li, Y., Hou, X., Koch, C., Rehg, J. M.,Yuille, A. L.: The secrets of salient object segmentation. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 280–287 (2014)Google Scholar
  10. 10.
    Kim, J., Han, D., Tai, Y. W., Kim, J.: Salient region detection via high-dimensional color transform. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 883–890 (2014)Google Scholar
  11. 11.
    Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: Advances in Neural Information Processing Systems, pp. 545–552 (2007)Google Scholar
  12. 12.
    Wang, W., Wang, Y., Huang, Q., Gao, W.: Measuring visual saliency by site entropy rate. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 2368–2375 (2010)Google Scholar
  13. 13.
    Mai, L., Niu, Y., Liu, F.: Saliency aggregation: a data-driven approach. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 1131–1138 (2013)Google Scholar
  14. 14.
    Gong, C., Tao, D., Liu, W., Maybank, S. J., Fang, M., Fu, K., Yang, J.: Saliency propagation from simple to difficult. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2531–2539 (2015)Google Scholar
  15. 15.
    Zhang, Q., Lin, J., Li, W., Shi, Y., Cao, G.: Salient object detection via compactness and objectness cues. Vis. Comput. 34, 473–489 (2018)CrossRefGoogle Scholar
  16. 16.
    Qi, W., Han, J., Zhang, Y., Bai, L.: Saliency detection via Boolean and foreground in a dynamic Bayesian framework. Vis. Comput. 33, 209–220 (2017)CrossRefGoogle Scholar
  17. 17.
    Yang, C., Zhang, L., Lu, H., Ruan, X., Yang, M.-H.: Saliency detection via graph-based manifold ranking. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 3166–3173 (2013)Google Scholar
  18. 18.
    Mao, M., Lu, J., Zhang, G., Zhang, J.: Multirelational social recommendations via multigraph ranking. IEEE Trans. Cybern. 47(12), 4049–4061 (2017)CrossRefGoogle Scholar
  19. 19.
    Wu, J., Pan, S., Zhu, X., Zhang, C., Wu, X.: Positive and unlabeled multi-graph learning. IEEE Trans. Cybern. 47(4), 818–829 (2017)CrossRefGoogle Scholar
  20. 20.
    Yang, X., Wang, M., Tao, D.: Robust visual tracking via multi-graph ranking. Neurocomputing 159, 35–43 (2015)CrossRefGoogle Scholar
  21. 21.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20, 1254–1259 (1998)CrossRefGoogle Scholar
  22. 22.
    Liu, T., Yuan, Z., Sun, J., Wang, J., Zheng, N., Tang, X., Shum, H.Y.: Learning to detect a salient object. IEEE Trans. Pattern Anal. Mach. Intell. 33, 353–367 (2011)CrossRefGoogle Scholar
  23. 23.
    Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1597–1604 (2009)Google Scholar
  24. 24.
    Shen, X., Wu, Y.: A unified approach to salient object detection via low rank matrix recovery. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 853–860 (2012)Google Scholar
  25. 25.
    Cheng, M.M., Mitra, N.J., Huang, X., Torr, P.H.S., Hu, S.M.: Global contrast based salient region detection. IEEE Trans. Pattern Anal. Mach. Intell. 37, 569–582 (2015)CrossRefGoogle Scholar
  26. 26.
    Li, X., Zhao, L., Wei, L., Yang, M.-H., Wu, F., Zhuang, Y., Ling, H., Wang, J.: Deepsaliency: multi-task deep neural network model for salient object detection. IEEE Trans. Image Process. 25(8), 3919–3930 (2016)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Kuen, J., Wang, Z., Wang, G.: Recurrent attentional networks for saliency detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3668–3677 (2016)Google Scholar
  28. 28.
    Lee, G., Tai, Y.-W., Kim, J.: Deep saliency with encoded low level distance map and high level features. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 660–668 (2016)Google Scholar
  29. 29.
    Hou, Q., Cheng, M.-M., Hu, X., Borji, A., Tu, Z., Torr, P.: Deeply supervised salient object detection with short connections. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 5300–5309 (2017)Google Scholar
  30. 30.
    Hu, P., Shuai, B., Liu, J., Wang, G.: Deep level sets for salient object detection. In: CVPR, vol. 1, p. 2 (2017)Google Scholar
  31. 31.
    Wang, Q., Zheng, W., Piramuthu, R.: Grab: visual saliency via novel graph model and background priors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 535–543 (2016)Google Scholar
  32. 32.
    Sun, J., Lu, H., Liu, X.: Saliency region detection based on markov absorption probabilities. IEEE Trans. Image Process. 24(5), 1639–1649 (2015)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Li, C., Yuan, Y., Cai, W., Xia, Y., Feng, D. D. et al.: Robust saliency detection via regularized random walks ranking. In: CVPR, pp. 2710–2717 (2015)Google Scholar
  34. 34.
    Wang, J., Lu, H., Li, X., Tong, N., Liu, W.: Saliency detection via background and foreground seed selection. Neurocomputing 152, 359–368 (2015)CrossRefGoogle Scholar
  35. 35.
    Tao, D., Cheng, J., Song, M., Lin, X.: Manifold ranking-based matrix factorization for saliency detection. IEEE Trans. Neural Netw. Learn. Syst. 27(6), 1122–1134 (2016)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Xia, C., Li, J., Chen, X., Zheng, A., Zhang, Y.: What is and what is not a salient object? learning salient object detector by ensembling linear exemplar regressors. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4321–4329 (2017)Google Scholar
  37. 37.
    Zhou, D., Bousquet, O., Lal, T. N., Weston, J., Schölkopf, B.: Learning with local and global consistency. In: Advances in Neural Information Processing Systems, pp. 321–328 (2004)Google Scholar
  38. 38.
    Zhou, D., Weston, J., Gretton, A., Bousquet, O., Schölkopf, B.: Ranking on data manifolds. In: Advances in Neural Information Processing Systems, pp. 169–176 (2004)Google Scholar
  39. 39.
    Perazzi, F., Krähenbühl, P., Pritch, Y., Hornung, A.: .Saliency filters: contrast based filtering for salient region detection. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp. 733–740 (2012)Google Scholar
  40. 40.
    Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)CrossRefGoogle Scholar
  41. 41.
    Jiang, B., Zhang, L., Lu, H., Yang, C., Yang, M.-H.: Saliency detection via absorbing markov chain. In: 2013 IEEE International Conference on Computer Vision (ICCV), IEEE, pp. 1665–1672 (2013)Google Scholar
  42. 42.
    Zhang, L., Yang, C., Lu, H., Ruan, X., Yang, M.H.: Ranking saliency. IEEE Trans. Pattern Anal. Mach. Intell. 39, 1892–1904 (2017)CrossRefGoogle Scholar
  43. 43.
    Wei, Y., Wen, F., Zhu, W., Sun, J.: Geodesic saliency using background priors. In: Fitzgibbon, A., Lazebnik, S. Perona, P. Sato, Y. Schmid, C. (eds.) Computer Vision– ECCV 2012, pp. 29–42. Springer, Berlin (2012)Google Scholar
  44. 44.
    Wu, X., Jin, Z., Zhou, J., Ma, X.: Saliency propagation with perceptual cues and background-excluded seeds. J. Vis. Commun. Image Represent. 54, 51–62 (2018)CrossRefGoogle Scholar
  45. 45.
    Lu, S., Mahadevan, V., Vasconcelos, N.: Learning optimal seeds for diffusion-based salient object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2790–2797 (2014)Google Scholar
  46. 46.
    Cheng, M.-M., Warrell, J., Lin, W.-Y., Zheng, S., Vineet, V., Crook, N.: Efficient salient region detection with soft image abstraction. In: IEEE ICCV, pp. 1529–1536 (2013)Google Scholar
  47. 47.
    Cheng, M.-M., Mitra, N.J., Huang, X., Hu, S.-M.: Salientshape: group saliency in image collections. Vis. Comput. 30, 443–453 (2014)CrossRefGoogle Scholar
  48. 48.
    Shi, J., Yan, Q., Xu, L., Jia, J.: Hierarchical image saliency detection on extended cssd. IEEE Trans. Pattern Anal. Mach. Intell. 38, 717–729 (2016)CrossRefGoogle Scholar
  49. 49.
    Peng, H., Li, B., Ling, H., Hu, W., Xiong, W., Maybank, S.J.: Salient object detection via structured matrix decomposition. IEEE Trans. Pattern Anal. Mach. Intell. 39, 818–832 (2017)CrossRefGoogle Scholar
  50. 50.
    Margolin, R., Zelnik-Manor, L.,Tal, A.: How to evaluate foreground maps? In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014)Google Scholar
  51. 51.
    Zhai, Y., Shah, M.: Visual attention detection in video sequences using spatiotemporal cues. In: Proceedings of the 14th ACM International Conference on Multimedia, MM ’06, pp. 815–824. ACM, New York (2006)Google Scholar
  52. 52.
    Zhu, W., Liang, S., Wei, Y., Sun, J.: Saliency optimization from robust background detection. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’14, IEEE Computer Society, pp. 2814–2821. Washington (2014)Google Scholar
  53. 53.
    Tang, C., Wang, P., Zhang, C., Li, W.: Salient object detection via weighted low rank matrix recovery. IEEE Signal Process. Lett. 24, 490–494 (2017)CrossRefGoogle Scholar
  54. 54.
    Zhao, R., Ouyang, W., Li, H., Wang, X.: Saliency detection by multi-context deep learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1265–1274 (2015)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of Mechanical EngineeringNanjing University of Science and TechnologyNanjingChina
  2. 2.School of Computer Science and EngineeringNanjing University of Science and TechnologyNanjingChina
  3. 3.Shijiazhuang Campus, AEUShijiazhuangChina

Personalised recommendations