Advertisement

Saliency detection using multiple low-level priors and a propagation mechanism

  • Muwei JianEmail author
  • Jing Wang
  • Junyu Dong
  • Chaoran Cui
  • Xiushan Nie
  • Yilong Yin
Article

Abstract

The majority of existing methods for saliency detection based on low-level features failed to uniformly highlight the salient-object regions. In order to improve the accuracy and consistency of generated saliency maps, we propose a novel and efficient framework by combining low-level saliency priors and local similarity cues for image saliency detection. Firstly, we construct a multiple low-level prior map using location prior, color prior and background prior. Then, the prior maps employ a propagation mechanism based on Cellular Automata to enforce relevance of similar regions as a local similarity cue. Finally, a principle refinement framework by integrating multi-level prior maps and local similarity cue map are used to obtain an ultimate high-quality saliency map. Extensive experiments on publicly available datasets show that our designed approach is capable of producing accurate saliency maps compared with those generated results by the state-of-the-art saliency-detection methods.

Keywords

Multimedia data analysis Background prior Saliency detection Cellular Automata Multimedia data modeling 

Notes

Acknowledgements

We would like to thank Prof. Hui Yu in the School of Creative Technologies, University of Portsmouth, for providing technical editing and proofreading of the manuscript.

This work was supported by National Natural Science Foundation of China (NSFC) (61601427, 61602229, 61771230); Royal Society – K. C. Wong International Fellow; Natural Science Foundation of Shandong Province (ZR2016FM40); Shandong Provincial Key Research and Development Program of China (NO. 2017CXGC0701); Fostering Project of Dominant Discipline and Talent Team of Shandong Province Higher Education Institutions.

References

  1. 1.
    Achanta R, Hemami S, Estrada F et al (2009) Frequency-tuned salient region detection. IEEE Computer Vision and Pattern Recognition:1597–1604Google Scholar
  2. 2.
    Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282CrossRefGoogle Scholar
  3. 3.
    Chang K-Y, Liu T-L, Chen H-T, Lai S-H (2011) Fusing generic objectness and visual saliency for salient object detection. Proc IEEE Int Conf Comput Vis:914–921Google Scholar
  4. 4.
    Cheng MM, Mitra NJ, Huang X et al (2015) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell 37(3):569–582CrossRefGoogle Scholar
  5. 5.
    Cheng M, Warrell J, Lin W, Zheng S, Vineet V, Crook N (2013) Efficient salient region detection with soft image abstraction. Proc IEEE Int Conf Comput Vis:1529–1536Google Scholar
  6. 6.
    Cholakkal H, Rajan D, Johnson J (2015) Top-down saliency with locality-constrained contextual sparse coding. BMVCGoogle Scholar
  7. 7.
    Goferman S, Zelnik-Manor L, Tal A (2012) Context-aware saliency detection. IEEE Trans Pattern Anal Mach Intell 34(10):1915–1926CrossRefGoogle Scholar
  8. 8.
    Harel J, Koch C, Perona P (2006) 'Graph-based visual Saliency', Advances in Neural Information Processing Systems, pp. 545–552Google Scholar
  9. 9.
    He S, Lau RWH, Yang Q (2015) Exemplar-Driven Top-Down Saliency Detection via Deep Association. IJCV 115(3):330–344CrossRefGoogle Scholar
  10. 10.
    Hou X, Harel J, Koch C (2012) Image signature: Highlighting sparse salient regions. IEEE Trans Pattern Anal Mach Intell 34(1):194–201CrossRefGoogle Scholar
  11. 11.
    Hou X, Zhang L (2007) Saliency Detection: A spectral residual approach. IEEE Conference on Computer Vision and Pattern Recognition:1–8Google Scholar
  12. 12.
    Huang F et al (2017) Salient object detection via multiple instance learning. IEEE Trans Image Process 26(4):1911–1922MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Itti L, Koch C, Niebur E (1998) A model of saliency based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259CrossRefGoogle Scholar
  14. 14.
    Jian M, Lam K-M, Dong J, Shen L (2015) Visual-patch-attention-aware Saliency Detection. IEEE Transactions on Cybernetics 45(8):1575–1586CrossRefGoogle Scholar
  15. 15.
    Jian M, Qi Q, Dong J, Yin Y, Lam K-M (2018) Integrating QDWD with Pattern Distinctness and Local Contrast for Underwater Saliency Detection. J Vis Commun Image Represent 53:31–41CrossRefGoogle Scholar
  16. 16.
    Jian M et al (2018) Saliency Detection Using Quaternionic Distance Based Weber Local Descriptor and Level Priors. Multimed Tools Appl 77(11):14343–14360CrossRefGoogle Scholar
  17. 17.
    Jiang H, Wang J, Yuan Z, Liu T, Zheng N, Li S (2011) Automatic salient object segmentation based on context and shape prior. Proc Brit Mach Vis Conf:1–12Google Scholar
  18. 18.
    Jing H, He X, Han Q, El-Latif AAA, Niu X (2014) Saliency detection based on integrated features. Neurocomputing 129(10):114–121CrossRefGoogle Scholar
  19. 19.
    Jung C, Kim C (2012) A unified spectral – domain approach for saliency detection and its application to automatic object segmentation. TIP 21(3):1272–1283MathSciNetzbMATHGoogle Scholar
  20. 20.
    Kim J, Han D, Tai Y, Kim J (2014) Salient region detection via high-dimensional color transform. IEEE Conference on Computer Vision and Pattern Recognition:883–890Google Scholar
  21. 21.
    Li J, Chen H, Li G et al (2015) Salient object detection based on meanshift filtering and fusion of colour information. IET Image Process 9(11):977–985CrossRefGoogle Scholar
  22. 22.
    Li J et al (2010) Probabilistic multi-task learning for visual saliency estimation in video. Int J Comput Vis 90(2):150–165CrossRefGoogle Scholar
  23. 23.
    Liu X, Huet B (2016) Event-based cross media question answering. Multimed Tools Appl 75(3):1495–1508CrossRefGoogle Scholar
  24. 24.
    Liu T, Sun J, Zheng N et al (2011) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353–367CrossRefGoogle Scholar
  25. 25.
    Liu X, Wang M, Yin B, Huet B, Li X (2015) Event-Based Media Enrichment Using an Adaptive Probabilistic Hypergraph Model. IEEE T Cybernetics 45(11):2461–2471CrossRefGoogle Scholar
  26. 26.
    Ma YF, Zhang HJ (2003) Contrast-based image attention analysis by using fuzzy growing', Proceedings of the Eleventh ACM International Conference on Multimedia, pp. 374–381Google Scholar
  27. 27.
    Mahadevan V, Vasconcelos N (2009) Saliency-based discriminant tracking. Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, IEEEGoogle Scholar
  28. 28.
    Mahadevan V, Vasconcelos N (2010) Spatiotemporal saliency in dynamic scenes. IEEE Trans Pattern Anal Mach Intell 32(1):171–177CrossRefGoogle Scholar
  29. 29.
    Movahedi V, Elder JH (2010) Design and perceptual validation of performance measures for salient object segmentation. IEEE Conference on Computer Vision and Pattern Recognition Workshops:49–56Google Scholar
  30. 30.
  31. 31.
    Peng H et al (2017) Salient object detection via structured matrix decomposition. IEEE Trans Pattern Anal Mach Intell 39(4):818–832CrossRefGoogle Scholar
  32. 32.
    Qin Y et al (2015) Saliency detection via cellular automata. Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on, IEEEGoogle Scholar
  33. 33.
    Scharfenberger C, Wong A, Fergani K, Zelek JS, Clausi DA (2013) Statistical textural distinctiveness for salient region detection in natural images. Proc IEEE Conf Comput Vis Pattern Recog:979–986Google Scholar
  34. 34.
    Shen X, Wu Y (2012) A unified approach to salient object detection via low rank matrix recovery. Proc IEEE Conf Comput Vis Pattern Recog:2296–2303Google Scholar
  35. 35.
    Shi J et al (2016) Hierarchical image saliency detection on extended CSSD. IEEE Trans Pattern Anal Mach Intell 38(4):717–729CrossRefGoogle Scholar
  36. 36.
    Song H, Liu Z, Xie Y et al (2016) RGBD Co-saliency Detection via Bagging-Based Clustering. IEEE Signal Processing Letters 23(12):1707–1711CrossRefGoogle Scholar
  37. 37.
    Song J, Zhang H, Li X, Gao L, Wang M, Hong R (2018) Self-Supervised Video Hashing With Hierarchical Binary Auto-Encoder. IEEE Trans Image Processing 27(7):3210–3221MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Toshev A, Shi J, Daniilidis K (2007) Image matching via saliency region correspondences. Computer Vision and Pattern Recognition, 2007. CVPR'07. IEEE Conference on, IEEEGoogle Scholar
  39. 39.
    Wei Y, Wen F, Zhu W, Sun J (2012) Geodesic saliency using background priors. Proc Eur Conf Comput Vis:29–42Google Scholar
  40. 40.
    Wolfram S (1983) Statistical mechanics of cellular automata. Rev Mod Phys 55(3):601MathSciNetCrossRefzbMATHGoogle Scholar
  41. 41.
    Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection. Proc IEEE Conf Comput Vis Pattern Recog:1155–1162Google Scholar
  42. 42.
    Yang J, Yang MH (2017) Top-down visual saliency via joint CRF and dictionary learning. IEEE Transactions on PAMI 39(3):576–588CrossRefGoogle Scholar
  43. 43.
    Yang C, Zhang L, Lu H (2013) Graph-regularized saliency detection with convex-hull-based center prior. IEEE Signal Processing Letters 20(7):637–640CrossRefGoogle Scholar
  44. 44.
    Yang C et al (2013) Saliency detection via graph-based manifold ranking. Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, IEEEGoogle Scholar
  45. 45.
    Zhai Y, Shah M (2006) Visual attention detection in video sequences using spatiotemporal cues. In: Proc. 14th ACM Int. Conf. Multimedia, pp 815–824CrossRefGoogle Scholar
  46. 46.
    Zhang H, Kyaw Z, Chang SF, Chua TS (2017) Visual Translation Embedding Network for Visual Relation Detection. CVPR:3107–3115Google Scholar
  47. 47.
    Zhang J, Sclaroff S (2016) Exploiting Surroundedness for Saliency Detection: A Boolean Map Approach. IEEE Trans Pattern Anal Mach Intell 38(5):889–902CrossRefGoogle Scholar
  48. 48.
    Zhu W, Liang S, Wei Y, Sun J (2014) Saliency optimization from robust background detection. Proc IEEE Conf Comput Vis Pattern Recognit:2814–2821Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • Muwei Jian
    • 1
    • 2
    Email author
  • Jing Wang
    • 2
  • Junyu Dong
    • 2
  • Chaoran Cui
    • 1
  • Xiushan Nie
    • 1
  • Yilong Yin
    • 3
  1. 1.School of Computer Science and TechnologyShandong University of Finance and EconomicsJinanChina
  2. 2.Department of Computer Science and TechnologyOcean University of ChinaQingdaoChina
  3. 3.School of Software EngineeringShandong UniversityJinanChina

Personalised recommendations