Applied Intelligence

, Volume 49, Issue 2, pp 764–790 | Cite as

Traffic sign detection based on visual co-saliency in complex scenes

  • Lingli Yu
  • Xumei XiaEmail author
  • Kaijun ZhouEmail author


Co-saliency detection aims at finding the salient regions from multiple images which capture the focus of human visual system. In this paper, a novel visual co-saliency algorithm is proposed, which adopts three human visual attention cues: contrast, center-bias and symmetry. In order to apply co-saliency to the detection of traffic signs, a traffic sign detection framework based on visual co-saliency in complex scenes is devised. The detection process involves two stages. In the first stage, a cluster-based co-saliency model is built to generate the final co-saliency map. In the second stage, a geometric structure constraint model is constructed to discriminate the detected salient objects and then accurately achieve location of traffic signs. The advantage of our approach lies in the integration of bottom-up and top-down visual processing, and no heavy learning tasks. Experiments on a variety of benchmark databases illustrate high precision, high recall and operation efficiency of the proposed approach. Besides, for traffic sign detection it overcomes the interference of complex urbanization backgrounds. Furthermore, the best trade-off between precision and recall on warning signs is achieved, reaching 93.30% and 89.06%, respectively.


Co-saliency detection Visual attention cues Traffic sign detection Complex scenes 



This research was funded by State Key Laboratory of Mechanical Transmissions of Chongqing University (SKLMT-KFKT-201602), State Key Laboratory of Robotics and System (HIT)(SKLRS-2017-KF-13), Major Projects of Science and Technology in Hunan (Grant No.2017GK1010), National Key Research and Development Plan (Grant No. 2018YFB1201602), National Natural Science Foundation of China (Grant No.61403426), Fundamental Research Funds for the Central Universities of Central South University (Grant No.2017zzts490) and National Natural Science Foundation of Hunan (Grant No.2018JJ2531, 2018JJ2197).


  1. 1.
    Zhang Q (2016) The design of salient object detection experiment based on visual attention. Research & Exploration in LaboratoryGoogle Scholar
  2. 2.
    Ramk DM, Sabourin C, Moreno R, Madani K (2014) A machine learning based intelligent vision system for autonomous object detection and recognition. Appl Intell 40(2):358–375CrossRefGoogle Scholar
  3. 3.
    Goferman S, Zelnik-Manor L, Tal A (2012) Context-aware saliency detection. IEEE Trans Pattern Anal Mach Intell 34(10): 1915–1926CrossRefGoogle Scholar
  4. 4.
    Fang Y, Lin W, Sung Lee BS, Lau CT, Chen Z, Lin CW (2012) Bottom-up saliency detection model based on human visual sensitivity and amplitude spectrum. IEEE Trans Multimed 14(1):187–198CrossRefGoogle Scholar
  5. 5.
    Toet A (2011) Computational versus psychophysical bottom-up image saliency: a comparative evaluation study. IEEE Trans Pattern Anal Mach Intell 33(11):2131CrossRefGoogle Scholar
  6. 6.
    Yang MH, Yang J (2012) Top-down visual saliency via joint crf and dictionary learning. In: IEEE conference on computer vision and pattern recognition, pp 2296–2303Google Scholar
  7. 7.
    Xu Y, Li J, Chen J, Shen G, Gao Y (2017) A novel approach for visual saliency detection and segmentation based on objectness and top-down attention. In: International conference on image, vision and computing, pp 361–365Google Scholar
  8. 8.
    Borji A, Cheng M-M, Jiang H, Li J (2015) Salient object detection: a benchmark. IEEE Trans Image Process 24(12):5706– 5722MathSciNetCrossRefGoogle Scholar
  9. 9.
    Arya R, Singh N, Agrawal RK (2017) A novel combination of second-order statistical features and segmentation using multi-layer superpixels for salient object detection. Appl Intell 46(2):1–18CrossRefGoogle Scholar
  10. 10.
    Jiang H, Wang J, Yuan Z, Wu Y (2013) Salient object detection: a discriminative regional feature integration approach. In: Computer vision and pattern recognition, pp 2083–2090Google Scholar
  11. 11.
    Itti Laurent, Koch Christof, Niebur Ernst (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans 20(11):1254–1259Google Scholar
  12. 12.
    Achanta R, Estrada F, Wils P, Süsstrunk S (2008) Salient region detection and segmentation. Springer, BerlinCrossRefGoogle Scholar
  13. 13.
    Cheng MM, Mitra NJ, Huang X, Torr PH, Hu SM (2015) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell 37(3):569CrossRefGoogle Scholar
  14. 14.
    Hou X, Zhang L (2007) Saliency detection: a spectral residual approach. In: IEEE conference on computer vision and pattern recognition, 2007. CVPR ’07, pp 1–8Google Scholar
  15. 15.
    Jacobs DE, Goldman Dan B, Shechtman E (2010) Cosaliency:where people look when comparing images. In: ACM symposium on user interface software and technology. New York, pp 219–228Google Scholar
  16. 16.
    Cao X, Tao Z, Zhang B, Fu H, Feng W (2014) Self-adaptively weighted co-saliency detection via rank constraint. IEEE Trans Image Process: A Publ IEEE Signal Process Soc 23(9): 4175–4186MathSciNetzbMATHGoogle Scholar
  17. 17.
    Li H, Meng F, Ngan KN (2013) Co-salient object detection from multiple images. IEEE Trans Multimed 15(8):1896–1909CrossRefGoogle Scholar
  18. 18.
    Chang KY, Liu TL, Lai SH (2011) From co-saliency to co-segmentation an efficient and fully unsupervised energy minimization model. In: IEEE conference on computer vision and pattern recognition, pp 2129–2136Google Scholar
  19. 19.
    Li H, Ngan KN (2011) A co-saliency model of image pairs. IEEE Trans Image Process: A Publ IEEE Signal Process Soc 20(12):3365MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Chen HT (2010) Preattentive co-saliency detection. In: IEEE international conference on image processing, pp 1117–1120Google Scholar
  21. 21.
    Fu H, Cao X, Tu Z (2013) Cluster-based co-saliency detection. IEEE Trans Image Process: A Publ IEEE Signal Process Soc 22(10):3766MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Liu Z, Zou W, Li L, Shen L, Meur OL (2013) Co-saliency detection based on hierarchical segmentation. Process Lett 21(1):88–92CrossRefGoogle Scholar
  23. 23.
    Hu X, Zhu X, Li D (2012) Traffic sign detection based on biologically visual mechanism. ISPRS - International Archives of the Photogrammetry. Remote sensing and spatial information sciences, XXXIX-B3, pp 217–221Google Scholar
  24. 24.
    Mogelmose A, Trivedi MM, Moeslund TB (2012) Vision-based traffic sign detection and analysis for intelligent driver assistance systems: perspectives and survey. IEEE Trans Intell Transp Syst 13(4):1484–1497CrossRefGoogle Scholar
  25. 25.
    Guo H, Liu H, Bi S (2011) Detection of traffic signs based on visual saliency model. In: International conference on multimedia technology, pp 3035–3038Google Scholar
  26. 26.
    Gudigar A, Chokkadi S, Raghavendra U (2016) A review on automatic detection and recognition of traffic sign. Multimed Tools Appl 75(1):333–364CrossRefGoogle Scholar
  27. 27.
    Greenhalgh J, Mirmehdi M (2012) Real-time detection and recognition of road traffic signs. IEEE Trans Intell Transp Syst 13(4):1498–1506CrossRefGoogle Scholar
  28. 28.
    Romdhane NB, Mliki H, Hammami M (2016) An improved traffic signs recognition and tracking method for driver assistance system. In: Ieee/acis international conference on computer and information science, pp 1–6Google Scholar
  29. 29.
    Prisacariu VA, Timofte R, Zimmermann K, Reid I, Gool LV (2010) Integrating object detection with 3d tracking towards a better driver assistance system. In: International conference on pattern recognition, pp 3344–3347Google Scholar
  30. 30.
    Zang D, Zhang J, Zhang D, Bao M, Cheng J, Tang K (2016) Traffic sign detection based on cascaded convolutional neural networks. In: Ieee/acis international conference on software engineering, artificial intelligence, NETWORKING and parallel/distributed computing, pp 201–206Google Scholar
  31. 31.
    Shi JH, Lin HY (2017) A vision system for traffic sign detection and recognition. In: IEEE international symposium on industrial electronics, pp 1596–1601Google Scholar
  32. 32.
    Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: IEEE conference on computer vision and pattern recognition, 2009. CVPR 2009, pp 1597–1604Google Scholar
  33. 33.
    Duan L, Wu C, Miao J, Qing L (2011) Visual saliency detection by spatially weighted dissimilarity. In: Computer vision and pattern recognition, pp 473–480Google Scholar
  34. 34.
    Tatler BW (2007) The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. J Vis 7(14):4.1CrossRefGoogle Scholar
  35. 35.
    Rothkegel LOM, Trukenbrod HA, Schtt HH, Wichmann FA, Engbert R (2017) Temporal evolution of the central fixation bias in scene viewing. J Vis 17(13):3CrossRefGoogle Scholar
  36. 36.
    Widynski N, Moevus A, Mignotte M (2014) Local symmetry detection in natural images using a particle filtering approach. IEEE Trans Image Process 23(12):5309–22MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Kovesi P (2000) Phase congruency: a low-level image invariant. Psychol Res 64(2):136CrossRefGoogle Scholar
  38. 38.
    Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X, Shum HY (2011) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353CrossRefGoogle Scholar
  39. 39.
    Lang C, Nguyen TV, Katti H, Yadati K, Kankanhalli M, Yan S (2012) Depth matters: influence of depth cues on visual saliency. In: European conference on computer vision, pp 101–115Google Scholar
  40. 40.
    Batra D, Kowdle A, Parikh D, Luo J (2010) icoseg:interactive co-segmentation with intelligent scribble guidance. In: Computer vision and pattern recognition, pp 3169–3176Google Scholar
  41. 41.
    Larsson F, Felsberg M (2011) Using fourier descriptors and spatial models for traffic sign recognition. In: Scandinavian conference on image analysis, pp 238–249Google Scholar
  42. 42.
    Wei W, Yang XL, Zhou B, Feng J, Shen PY (2012) Combined energy minimization for image reconstruction from few views. Math Probl Eng,2012,(2012-10-31) 2012(3):1094–1099zbMATHGoogle Scholar
  43. 43.
    Wei W, Fan X, Song H, Wang H (2017) Video tamper detection based on multi-scale mutual information. Multimed Tools Appl 2017(9):1–18Google Scholar
  44. 44.
    Wei W, Qi Y (2011) Information potential fields navigation in wireless ad-hoc sensor networks. Sensors 11 (5):4794–4807CrossRefGoogle Scholar
  45. 45.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Information Science and EngineeringCentral South UniversityChangshaChina
  2. 2.The State Key Laboratory of Mechanical TransmissionsChongqing UniversityChongqingChina
  3. 3.State Key Laboratory of Robotics and SystemHarbin Institute of TechnologyHaerbinChina
  4. 4.School of Computer and Information EngineeringHunan University of CommerceChangshaChina

Personalised recommendations