Advertisement

Journal of Central South University

, Volume 25, Issue 2, pp 259–276 | Cite as

Exploiting global and local features for image retrieval

  • Li Li (李莉)
  • Lin Feng (冯林)
  • Jun Wu (吴俊)
  • Mu-xin Sun (孙木鑫)
  • Sheng-lan Liu (刘胜蓝)
Article
  • 55 Downloads

Abstract

Two lines of image representation based on multiple features fusion demonstrate excellent performance in image retrieval. However, there are some problems in both of them: 1) the methods defining directly texture in color space put more emphasis on color than texture feature; 2) the methods extract several features respectively and combine them into a vector, in which bad features may lead to worse performance after combining directly good and bad features. To address the problems above, a novel hybrid framework for color image retrieval through combination of local and global features achieves higher retrieval precision. The bag-of-visual words (BoW) models and color intensity-based local difference patterns (CILDP) are exploited to capture local and global features of an image. The proposed fusion framework combines the ranking results of BoW and CILDP through graph-based density method. The performance of our proposed framework in terms of average precision on Corel-1K database is 86.26%, and it improves the average precision by approximately 6.68% and 12.53% over CILDP and BoW, respectively. Extensive experiments on different databases demonstrate the effectiveness of the proposed framework for image retrieval.

Key words

local binary patterns hue saturation value (HSV) color space graph fusion image retrieval 

全局和局部特征的图像检索

摘要

两种基于多特征融合的图像检索方法具有非常好的性能。 但是, 这两种融合方法存在以下问题: 1) 在颜色空间中直接定义纹理结构的方法会增大对颜色特征的描述; 2) 提取多种特征再重新融合为一个向量的方法, 这种方法将有效的特征和无效的特征直接结合后, 无效的特征会降低检索性能。 针对以上问题, 提出一种新的混合框架用于彩色图像检索, 该框架使用词袋模型(bag-of-visual words, BoW)和颜色强度局部差分模式(color intensity-based local difference patterns, CILDP)分别提取图像的不同特征信息。 同时, 提出的融合框架利用 graph density 的方法将 BoW 和 CILDP 的排序结果进行有效融合, 利用该框架能够提高图像检索的精度。 在Corel-1K 数据库上, 返回 10 幅图像时, 提出的框架的平均精度为86.26%, 分别比 CILDP 和 BoW 提高了大约 6.68%和 12.53%。 在不同数据库上的大量实验也验证了该框架在图像检索上的有效性。

关键词

局部二值模式 色调 饱和度 明度 (HSV) 颜色空间 图融合 图像检索 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    LIU Ying, ZHANG Deng-sheng, LU Guo-jun, MA Wei-ying. A survey of content-based image retrieval with high-level semantics [J]. Pattern Recognition, 2007, 40(1): 262–282.CrossRefMATHGoogle Scholar
  2. [2]
    PENATTI O A B, SILVA F B, VALLE E, GOUET-BRUNET V, TORRES R D S. Visual word spatial arrangement for image retrieval and classification [J]. Pattern Recognition, 2014, 47(2): 705–720.CrossRefGoogle Scholar
  3. [3]
    DATTA R, JOSHI D, LI Jia, WANG J Z. Image retrieval: Ideas, influences, and trends of the new age [J]. ACM Computing Surveys (CSUR), 2008, 40, 2: 5.CrossRefGoogle Scholar
  4. [4]
    SWAIN M J, BALLARD D H. Color indexing [J]. International Journal of Computer Vision, 1991, 7(1): 11–32.CrossRefGoogle Scholar
  5. [5]
    STRICKER M A, ORENGO M. Similarity of color images [C]//Proceedings of IS&T/SPIE's Symposium on Electronic Imaging: Science & Technology. Bellingham: SPIE, 1995: 381–392.Google Scholar
  6. [6]
    HUANG Jing, KUMAR S R, MITRA M, ZHU Wei-jing, ZABIH R. Image Indexing Using Color Correlograms [C]//Proceedings of 1997 IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 1997: 762–768.Google Scholar
  7. [7]
    PASS G, ZABIH R, MILLER J. Comparing images using color coherence vectors [C]//Proceedings of the Fourth ACM International Conference on Multimedia. New York: ACM, 1997: 65–73.Google Scholar
  8. [8]
    OJALA T, PIETIKAINEN M, MAENPAA T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7): 971–987.CrossRefMATHGoogle Scholar
  9. [9]
    HUANG Di, SHAN Cai-feng, ARDABILIAN M, WANG Yun-hong, CHEN Li-ming. Local binary patterns and its application to facial image analysis: A survey [J]. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 2011, 41(6): 765–781.CrossRefGoogle Scholar
  10. [10]
    HEIKKILÄ M, PIETIKÄINEN M, SCHMID C. Description of interest regions with local binary patterns [J]. Pattern Recognition, 2009, 42(3): 425–436.CrossRefMATHGoogle Scholar
  11. [11]
    GUO Yi-mo, ZHAO Guo-ying, PIETIKÄINEN M. Texture Classification using a Linear Configuration Model based Descriptor [C]//Proceedings of the British Machine Vision Conference (BMVC). Dundee, United Kingdom: Citeseer, 2011: 1–10.Google Scholar
  12. [12]
    TAN Xiao-yang, TRIGGS B. Enhanced local texture feature sets for face recognition under difficult lighting conditions [J]. IEEE Transactions on Image Processing, 2010, 19(6): 1635–1650.MathSciNetCrossRefMATHGoogle Scholar
  13. [13]
    GUO Zhen-hua, ZHANG Lei, ZHANG D. A completed modeling of local binary pattern operator for texture classification [J]. IEEE Transactions on Image Processing, 2010, 19(6): 1657–1663.MathSciNetCrossRefMATHGoogle Scholar
  14. [14]
    MURALA S, WU Q M. Local mesh patterns versus local binary patterns: Biomedical image indexing and retrieval [J]. IEEE Journal of Biomedical and Health Informatics, 2014, 18(3): 929–938.CrossRefGoogle Scholar
  15. [15]
    ZHANG Gang, MA Zong-min, DENG Li-guo, XU Chang-ming. Novel histogram descriptor for global feature extraction and description [J]. Journal of Central South University of Technology, 2010, 17: 580–586.CrossRefGoogle Scholar
  16. [16]
    LOWE D G. Distinctive image features from scale-invariant keypoints [J]. International Journal of Computer Vision, 2004, 60(2): 91–110.CrossRefGoogle Scholar
  17. [17]
    BAY H, ESS A, TUYTELAARS T, VAN GOOL L. Speeded-up robust features (SURF) [J]. Computer Vision and Image Understanding, 2008, 110(3): 346–359.CrossRefGoogle Scholar
  18. [18]
    RUBLEE E, RABAUD V, KONOLIGE K, BRADSKI G. ORB: An efficient alternative to SIFT or SURF [C]//Proceedings of 2011 IEEE International Conference on Computer Vision (ICCV). Piscataway, NJ: IEEE, 2011: 2564–2571.CrossRefGoogle Scholar
  19. [19]
    SIVIC J, ZISSERMAN A. Video Google: A text retrieval approach to object matching in videos [C]//Proceedings of Ninth IEEE International Conference on Computer Vision. Piscataway, NJ: IEEE, 2003: 1470–1477.CrossRefGoogle Scholar
  20. [20]
    SHEN Guan-lin, WU Xiao-jun. Content based image retrieval by combining color, texture and Centrist [C]//2013 Constantinides International Workshop on Signal Processing (CIWSP 2013). Stevenage, GBR: IET, 2013: 1–4.Google Scholar
  21. [21]
    ELALAMI M E. A novel image retrieval model based on the most relevant features [J]. Knowledge-Based Systems, 2011, 24(1): 23–32.CrossRefGoogle Scholar
  22. [22]
    SUBRAHMANYAM M, WU Q M J, MAHESHWARI R P, BALASUBRAMANIAN R. Modified color motif co-occurrence matrix for image indexing and retrieval [J]. Computers & Electrical Engineering, 2013, 39(3): 762–774.CrossRefGoogle Scholar
  23. [23]
    LIU Guang-hai, YANG Jing-yu. Image retrieval based on the texton co-occurrence matrix [J]. Pattern Recognition, 2008, 41(12): 3521–3527.CrossRefMATHGoogle Scholar
  24. [24]
    LIU Guang-hai, ZHANG Lei, HOU Ying-kun, LI Zuo-yong, YANG Jing-yu. Image retrieval based on multi-texton histogram [J]. Pattern Recognition, 2010, 43(7): 2380–2389.CrossRefMATHGoogle Scholar
  25. [25]
    LIU Guang-hai, YANG Jing-yu. Content-based image retrieval using color difference histogram [J]. Pattern Recognition, 2013, 46(1): 188–198.CrossRefGoogle Scholar
  26. [26]
    ZHANG Shao-ting, YANG Ming, COUR T, YU Kai, METAXAS D N. Query specific rank fusion for image retrieval [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(4): 803–815.CrossRefGoogle Scholar
  27. [27]
    SINGHA M, HEMACHANDRAN K. Performance analysis of color spaces in image retrieval [J]. Assam University Journal of Science and Technology, 2011, 7(2): 94–104.Google Scholar
  28. [28]
    VADIVEL A, SURAL S, MAJUMDAR A K. An integrated color and intensity co-occurrence matrix [J]. Pattern Recognition Letters, 2007, 28(8): 974–983.CrossRefGoogle Scholar
  29. [29]
    LIU Li, ZHAO Ling-jun, LONG Yun-li, KUANG Gang-yao, FIEGUTH P. Extended local binary patterns for texture classification [J]. Image and Vision Computing, 2012, 30(2): 86–99.CrossRefGoogle Scholar
  30. [30]
    CROSS G R, JAIN A K. Markov random field texture models [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,, 1983, 1: 25–39.CrossRefGoogle Scholar
  31. [31]
    GUO Zhen-hua, ZHANG Lei, ZHANG D. Rotation invariant texture classification using LBP variance (LBPV) with global matching [J]. Pattern Recognition, 2010, 43(3): 706–719.CrossRefMATHGoogle Scholar
  32. [32]
    ZHANG Xiao-fan, DOU Hang, JU Tao, ZHANG Shao-ting. Fusing heterogeneous features for the image-guided diagnosis of intraductal breast lesions [C]//Proceedings of 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI). Piscataway, NJ: IEEE, 2015: 1288–1291.CrossRefGoogle Scholar
  33. [33]
    WANG J Z, LI Jia, WIEDERHOLD G. SIMPLIcity: Semantics-sensitive integrated matching for picture libraries [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(9): 947–963.CrossRefGoogle Scholar
  34. [34]
    LI Jia, WANG J Z. Automatic linguistic indexing of pictures by a statistical modeling approach [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(9): 1075–1088.CrossRefGoogle Scholar
  35. [35]
    ANTANI S, KASTURI R, JAIN R. A survey on the use of pattern recognition methods for abstraction, indexing and retrieval of images and video [J]. Pattern Recognition, 2002, 35(4): 945–965.CrossRefMATHGoogle Scholar
  36. [36]
    LIU Guang-hai, LI Zuo-yong, ZHANG Lei, XU Yong. Image retrieval based on micro-structure descriptor [J]. Pattern Recognition, 2011, 44(9): 2123–2133.CrossRefGoogle Scholar
  37. [37]
    BESIRIS D, ZIGOURIS E. Dictionary-based color image retrieval using multiset theory [J]. Journal of Visual Communication and Image Representation, 2013, 24(7): 1155–1167.CrossRefGoogle Scholar
  38. [38]
    LANCE G N, WILLIAMS W T. Mixed-data classificatory programs I-agglomerative systems [J]. Australian Computer Journal, 1967, 1(1): 15–20.Google Scholar
  39. [39]
    ELALAMI M E. A new matching strategy for content based image retrieval system [J]. Applied Soft Computing, 2014, 14(1): 407–418.CrossRefGoogle Scholar
  40. [40]
    JALAB H. Image retrieval system based on color layout descriptor and Gabor filters [C]//Proceedings of 2011 IEEE Conference on Open Systems (ICOS). Piscataway, NJ: IEEE, 2011: 32–36.CrossRefGoogle Scholar
  41. [41]
    HIREMATH P S, PUJARI J. Content based image retrieval using color, texture and shape features [C]//Proceedings of 2007 International Conference on Advanced Computing and Communications (ADCOM 2007). Piscataway, NJ: IEEE, 2007: 780–784.Google Scholar
  42. [42]
    BANERJEE M, KUNDU M K, MAJI P. Content-based image retrieval using visually significant point features [J]. Fuzzy Sets and Systems, 2009, 160(23): 3323–3341.MathSciNetCrossRefGoogle Scholar
  43. [43]
    GUO Jing-ming, PRASETYO H, SU Huai-sheng. Image indexing using the color and bit pattern feature fusion [J]. Journal of Visual Communication and Image Representation, 2013, 24(8): 1360–1379.CrossRefGoogle Scholar
  44. [44]
    IRTAZA A, JAFFAR M A, ALEISA E, CHOI T S. Embedding neural networks for semantic association in content based image retrieval [J]. Multimedia Tools and Applications, 2014, 72(2): 1911–1931.CrossRefGoogle Scholar
  45. [45]
    ZENG Shan, HUANG Rui, WANG Hai-bing, KANG Zhen. Image retrieval using spatiograms of colors quantized by Gaussian Mixture Models [J]. Neurocomputing, 2016, 171(1): 673–684.CrossRefGoogle Scholar

Copyright information

© Central South University Press and Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Computer Science and Technology, Faculty of Electronic Information and Electrical EngineeringDalian University of TechnologyDalianChina
  2. 2.School of Innovation and EntrepreneurshipDalian University of TechnologyDalianChina
  3. 3.School of Control Science and Engineering, Faculty of Electronic Information and Electrical EngineeringDalian University of TechnologyDalianChina

Personalised recommendations