Advertisement

Deep supervised visual saliency model addressing low-level features

  • Lecheng Zhou
  • Xiaodong GuEmail author
Original Research
  • 6 Downloads

Abstract

Deep neural networks detect visual saliency with semantic information. These high-level features locate salient regions efficiently but pay less attention to structure preservation. In our paper, we emphasize crucial low-level features for deep neural networks in order to preserve local structure and integrity of objects. The proposed framework consists of an image enhancement network and a saliency prediction network. In the first part of our model, we segment the image with a superpixel based unit-linking pulse coupled neural network (PCNN) and generate a weight map representing contrast and spatial properties. With the help of these low-level features, a fully convolutional network (FCN) is employed to compute saliency map in the second part. The weight map enhances the input channels of the FCN, meanwhile refines the output prediction with polished details and contours of salient objects. We demonstrate the superior performance of our model against other state-of-the-art approaches through experimental results on five benchmark datasets.

Keywords

Visual saliency Fully convolutional networks Pulse coupled neural networks Low-level features 

Notes

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grants 61771145 and 61371148.

References

  1. Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 1597–1604Google Scholar
  2. Achanta R, Shaji A, Smith K, Luchhi A, Fua P, Susstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. Proc IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282CrossRefGoogle Scholar
  3. Akram T, Khan MA, Sharif M, Yasmin M (2018) Skin lesion segmentation and recognition using multichannel saliency estimation and M-SVM on selected serially fused features. J Ambient Intell Human Comput.  https://doi.org/10.1007/s12652-018-1051-5 Google Scholar
  4. Alpert S, Galun M, Basri R, Brandt A (2012) Image segmentation by probabilistic bottom–up aggregation and cue integration. IEEE Trans Pattern Anal Mach Intell 34(2):315–327CrossRefGoogle Scholar
  5. Amin-Naji M, Aghagolzadeh A, Ezoji M (2019) CNNs hard voting for multi-focus image fusion. J Ambient Intell Human Comput.  https://doi.org/10.1007/s12652-019-01199-0 Google Scholar
  6. Borji A, Cheng MM, Hou Q, Jiang H, Li J (2015) Salient object detection: a benchmark. IEEE Trans Image Proc 24(12):5706–5722MathSciNetCrossRefzbMATHGoogle Scholar
  7. Chen T, Lin L, Liu L, Luo X, Li X (2016) DISC: deep image saliency computing via progressive representation learning. IEEE Trans Neural Netw Learn Syst 27(6):1135–1149MathSciNetCrossRefGoogle Scholar
  8. Cheng MM, Zhang GX, Mitra NJ, Huang X, Hu SM (2011) Global contrast based salient region detection. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 409–416Google Scholar
  9. Donoser M, Urschler M, Hirzer M, Bischof H (2009) Saliency driven total variation segmentation. In Proceedings of the IEEE International Conference on computer vision, pp 817–824Google Scholar
  10. Eckhorn R, Reitboeck HJ, Arndt M, Dicke P (1990) Feature linking via synchronization among distributed assemblies: simulations of results from cat visual cortex. Neural Comput 2(3):293–307CrossRefGoogle Scholar
  11. Felzenszwalb P, Huttenlocher D (2004) Efficient graph-based image segmentation. Int J Comput Vis 59(2):167–181CrossRefGoogle Scholar
  12. Gao Y, Wang M, Zha ZJ, Shen J, Li X, Wu X (2013) Visualtextual joint relevance learning for tag-based social image search. IEEE Trans Image Process 22(1):363–376MathSciNetCrossRefzbMATHGoogle Scholar
  13. Gordo A, Almazán J, Revaud J, Larlus D (2016) Deep image retrieval: learning global representations for image search. In: Proceedings of the European Conference on computer vision, pp 241–257Google Scholar
  14. Gu X (2008) Feature extraction using unit-linking pulse coupled neural network and its applications. Neural Process Lett 27(1):25–41CrossRefGoogle Scholar
  15. He S, Lau RWH, Liu W, Huang Z, Yang Q (2015) SuperCNN: a superpixelwise convolutional neural network for salient object detection. Int J Comput Vis 115(3):330–344MathSciNetCrossRefGoogle Scholar
  16. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 770–778Google Scholar
  17. Hou Q, Cheng MM, Hu X, Borji A, Tu Z, Torr P (2019) Deeply supervised salient object detection with short connections. IEEE Trans Pattern Anal Mach Intell 41(4):815–828CrossRefGoogle Scholar
  18. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint 1704:04861Google Scholar
  19. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259CrossRefGoogle Scholar
  20. Jia Y, Shelhamer E, Donahue J et al (2014) Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on multimedia, pp 675–678Google Scholar
  21. Jiang B, Zhang L, Lu H, Yang C, Yang MH (2013) Saliency detection via absorbing Markov chain. In: Proceedings of the 2013 IEEE International Conference on computer vision, pp 1665–1672Google Scholar
  22. Jiang H, Wang J, Yuan Z, Wu Y, Zheng N, Li S (2013) Salient object detection: a discriminative regional feature integration approach. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 2083–2090Google Scholar
  23. Johnson JL, Ritter D (1993) Observation of periodic waves in a pulse-coupled neural network. Opt Lett 18:1253–1255CrossRefGoogle Scholar
  24. Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRefGoogle Scholar
  25. Lee G, Tai YW, Kim J (2016) Deep saliency with encoded low level distance map and high level features. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 660–668Google Scholar
  26. Li G, Yu Y (2015) Visual saliency based on multiscale deep features. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 5455–5463Google Scholar
  27. Li G, Yu Y (2018) Contrast-oriented deep neural networks for salient object detection. IEEE Trans Neural Netw Learn Syst 29(12):6038–6051CrossRefGoogle Scholar
  28. Li X, Lu H, Zhang L, Xiang R, Yang MH (2013) Saliency detection via dense and sparse reconstruction. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 2976–2983Google Scholar
  29. Li Y, Hou X, Koch C, Rehg JM, Yuille AL (2014) The secrets of salient object segmentation. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 280–287Google Scholar
  30. Li X, Zhao L, Wei L, Yang MH, Wu F, Zhuang Y, Ling H, Wang J (2016) Deepsaliency: multi-task deep neural network model for salient object detection. IEEE Trans Image Proc 25(8):3919–3930MathSciNetCrossRefzbMATHGoogle Scholar
  31. Li H, Chen J, Lu H, Chi Z (2017) CNN for saliency detection with low-level feature integration. Neurocomputing 226:212–220CrossRefGoogle Scholar
  32. Lin L, Wang X, Yang W, Lai JH (2015) Discriminatively trained and-or graph models for object shape detection. IEEE Trans Pattern Anal Mach Intell 37(5):959–972CrossRefGoogle Scholar
  33. Liu N, Han J (2016) DHSnet: deep hierarchical saliency network for salient object detection. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 678–686Google Scholar
  34. Liu T, Sun J, Zheng N, Tang X, Shum H (2011) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353–367CrossRefGoogle Scholar
  35. Luo Z, Mishra A, Achkar A, Eichel J, Li S, Jodoin PM (2017) Non-local deep features for salient object detection. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 6593–6601Google Scholar
  36. Mai L, Niu Y, Liu F (2013) Saliency aggregation: a data-driven approach. In Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 1131–1138Google Scholar
  37. Nie R, He M, Cao J, Zhou D, Liang Z (2018) Pulse coupled neural network based MRI image enhancement using classical visual receptive field for smarter mobile healthcare. J Ambient Intell Human Comput.  https://doi.org/10.1007/s12652-018-1098-3 Google Scholar
  38. Orlando JT, Rui S (2002) Image segmentation by histogram thresholding using fuzzy sets. IEEE Trans Image Proc 11(12):1457–1465CrossRefGoogle Scholar
  39. Reynolds JH, Desimone R (2003) Interacting roles of attention and visual salience in V4. Neuron 37(5):853–863CrossRefGoogle Scholar
  40. Shelhamer E, Long J, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 3431–3440Google Scholar
  41. Shen X, Wu X (2012) A unified approach to salient object detection via low rank matrix recovery. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 853–860Google Scholar
  42. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: Proceedings of the International Conference on learning representation, pp 1–14Google Scholar
  43. Wang L, Lu H, Ruan X, Yang MH (2015) Deep networks for saliency detection via local estimation and global search. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 3183–3192Google Scholar
  44. Wang Z, Chen T, Li G, Xu R, Lin L (2017) Multi-label image recognition by recurrently discovering attentional regions. In: Proceedings of the IEEE International Conference on computer vision, pp 464–472Google Scholar
  45. Wang H, Dai L, Cai Y, Sun X, Chen L (2018) Salient object detection based on multi-scale contrast. Neural Netw 101:47–56CrossRefGoogle Scholar
  46. Wang L, Wang L, Lu H, Zhang P, Ruan X (2019) Salient object detection with recurrent fully convolutional networks. IEEE Trans Pattern Anal Mach Intell 41(7):1734–1746CrossRefGoogle Scholar
  47. Wu H, Li G, Luo X (2014) Weighted attentional blocks for probabilistic object tracking. Vis. Comput 30(2):229–243CrossRefGoogle Scholar
  48. Xiao Z, Shi J, Chang Q (2009) Automatic image segmentation algorithm based on PCNN and fuzzy mutual information. In: Proceedings of the IEEE International Conference on computer and information technology, pp 241–245Google Scholar
  49. Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 1155–1162Google Scholar
  50. Yang C, Zhang L, Lu H, Ruan X, Yang MH (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 3166–3173Google Scholar
  51. Zhang P, Wang D, Lu H (2017) Learning uncertain convolutional features for accurate saliency detection. In: Proceedings of the IEEE International Conference on computer vision, pp 212–221Google Scholar
  52. Zhao R, Ouyang W, Li H, Wang X (2015) Saliency detection by multi-context deep learning. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 1265–1274Google Scholar
  53. Zhong X, Shih FY (2019) An efficient saliency detection model based on wavelet generalized lifting. Int J Pattern Recogn 33(02):1954006CrossRefGoogle Scholar
  54. Zhu W, Liang S, Wei Y, Sun J (2014) Saliency optimization from robust background detection. In: Proceedings of the IEEE Conference on computer vision & pattern recognition, pp 2814–2821Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Electronic EngineeringFudan UniversityShanghaiChina

Personalised recommendations