Advertisement

Multimedia Tools and Applications

, Volume 78, Issue 2, pp 2525–2550 | Cite as

Improving image retrieval by integrating shape and texture features

  • Yu-Nan LiuEmail author
  • Shan-Shan Zhang
  • Yu Sang
  • Si-Miao Wang
Article
First Online:
  • 101 Downloads

Abstract

Content-based image retrieval (CBIR) has been an active research topic in the last decade. Multiple feature extraction and representation is one of the most important issues in the CBIR. In this paper, we propose a new CBIR method based on an efficient integration of texture and shape features. The texture features are extracted on the decomposed images processed by the optimal non-subsampled shearlet transform (NSST), and are represented by the high-frequency sub-band coefficients, which can be modeled by Bessel K Form (BKF) distribution; the shape features are represented by low-order quaternion polar harmonic transforms (QPHTs). The two kinds of features are then integrated by a weighted distance measurement, where Kullback-Leibler distance (KLD) and Euclidean distance (ED) are used for texture and shape features respectively. The integration of shape and texture information provides a robust feature set for image retrieval. Experimental results on standard benchmarks show significant improvements on retrieval performance using the proposed method compared with previous state-of-the-art methods.

Keywords

Content-based image retrieval Non-subsampled shearlet transform BKF distribution Quaternion polar harmonic transforms 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

This work was partially supported by the National Science Fund of China under Grant Nos. 61702262, 61602226,U1713208 and 61472187, the 973 Program No. 2014CB349303, Program for Changjiang Scholars, and “the Fundamental Research Funds for the Central Universities” No. 30918011322.

References

  1. 1.
    Alzu'bi A, Amira A, Ramzan N (2017) Content-based image retrieval with compact deep convolutional features. Neurocomputing 249:95–105Google Scholar
  2. 2.
    Anuar FM, Setchi R, Lai Y (2013) Trademark image retrieval using an integrated shape descriptor. Expert Syst Appl 40(1):105–121Google Scholar
  3. 3.
    Aptoula E (2014) Remote sensing image retrieval with global morphological texture descriptors. IEEE Trans Geosci Remote Sensing 52(2):3023–3034Google Scholar
  4. 4.
    Arandjelovi R, Gronat P, Torii A, Pajdla T, Sivic J (2016) NetVLAD: CNN architecture for weakly supervised place recognition. Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5297–5307Google Scholar
  5. 5.
    Arandjelovic R, Zisserman A (2013) All about VLAD. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1578–1585Google Scholar
  6. 6.
    Atto AM, Berthoumieu Y, Bolon P (2013) 2-D wavelet packet spectrum for texture analysis. IEEE Trans Image Process 22(6):2495–2500MathSciNetzbMATHGoogle Scholar
  7. 7.
    Babenko A, Slesarev A, Chigorin A, Lempitsky V (2014) Neural codes for image retrieval. In: European Conference on Computer vision (ECCV). Springer, pp 584–599Google Scholar
  8. 8.
    Chen WT, Liu WC, Chen MS (2010) Adaptive color feature extraction based on image color distributions. IEEE Trans Image Process 19(8):2005–2016MathSciNetzbMATHGoogle Scholar
  9. 9.
    Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection, vol 1. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, pp 886–893Google Scholar
  10. 10.
    Delhumeau J, Gosselin PH, Jégou H, Pérez P (2013) Revisiting the VLAD image representation. Proceedings of the 21st ACM International Conference on Multimedia, Rennes, pp 653–656Google Scholar
  11. 11.
    Fadili JM, Boubchir L (2005) Analytical form for a Bayesian wavelet estimator of images using the Bessel K form densities. IEEE Trans Image Process 14(2):231–240MathSciNetGoogle Scholar
  12. 12.
    Farsi H, Mohamadzadeh S (2013) Colour and texture feature-based image retrieval by using hadamard matrix in discrete wavelet transform. IET Image Process 7(3):212–218MathSciNetGoogle Scholar
  13. 13.
    He Z, You X, Yuan Y (2009) Texture image retrieval based on non-tensor product wavelet filter banks. Signal Process 89(8):1501–1510zbMATHGoogle Scholar
  14. 14.
    Hu W, Xie N, Li L, Zeng X (2011) A survey on visual content-based video indexing and retrieval. IEEE Trans Syst, Man Cybernetics, Part C: Appl Rev 41(6):797–819Google Scholar
  15. 15.
    Huang J, You X, Yuan Y, Yang F, Lin L (2010) Rotation invariant iris feature extraction using Gaussian Markov random fields with non-separable wavelet. Neurocomputing 73(4–6):883–894Google Scholar
  16. 16.
    Imran M, Hashim R, Khalid NEA (2014) Color histogram and first order statistics for content based image retrieval. In: Recent Advances on Soft CompHeuting and Data Mining, Springer, pp: 153–162Google Scholar
  17. 17.
    Jain V, Sahu N (2013) A survey: on content based image retrieval. Int J Eng Res Appl 3(4):1166–1169Google Scholar
  18. 18.
    Jégou H, Douze M, Schmid C (2010) Improving bag-of-features for large scale image search. Int J Comput Vis 87(3):316–336Google Scholar
  19. 19.
    Jian M, Lam KM (2014) Face-image retrieval based on singular values and potential-field representation. Signal Process 100:9–15Google Scholar
  20. 20.
    Khokher A, Talwar R (2017) A fast and effective image retrieval scheme using color-, texture-, and shape-based histograms. Multimed Tools Appl 76(20):21787–21809Google Scholar
  21. 21.
    Lasmar NE, Berthoumieu Y (2014) Gaussian copula multivariate modeling for texture image retrieval using wavelet transforms. IEEE Trans Image Process 23(5):2246–2261MathSciNetzbMATHGoogle Scholar
  22. 22.
    Li X (2003) Image retrieval based on perceptive weighted color blocks. Pattern Recogn Lett 24(12):1935–1941Google Scholar
  23. 23.
    Li S, Lee MC, Pun CM (2009) Complex Zernike moments features for shape-based image retrieval. IEEE Trans Syst, Man Cybern, Part A: Systems Humans 39(1):227–237Google Scholar
  24. 24.
    Lim WQ (2010) The discrete shearlet transform: A new directional transform and compactly supported shearlet frames. IEEE Trans Image Process 19(5):1166–1180MathSciNetzbMATHGoogle Scholar
  25. 25.
    Liu Z, Li H, Zhou W, Rui T, Tian Q (2015) Uniforming residual vector distribution for distinctive image representation. IEEE Trans Circuits Syst Video Technol 99:1Google Scholar
  26. 26.
    Liu M, Vemuti BC, Amari SI, Nielsen F (2012) Shape retrieval using hierarchical total Bregman soft clustering. IEEE Trans Pattern Anal Mach Intell 34(12):2407–2419Google Scholar
  27. 27.
    Liu Z, Wang S, Tian Q (2016) Fine-residual VLAD for image retrieval. Neurocomputing 173:1183–1191Google Scholar
  28. 28.
    Liu GH, Yang JY (2013) Content-based image retrieval using color difference histogram. Pattern Recogn 46(1):188–198Google Scholar
  29. 29.
    Liu Y, Zhang DS, Lu GJ, Ma WY (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recogn 40(1):262–282zbMATHGoogle Scholar
  30. 30.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110MathSciNetGoogle Scholar
  31. 31.
    Ng J, Yang F, Davis L S (2015) Exploiting local features from deep networks for image retrieval. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 53–61Google Scholar
  32. 32.
    Park U, Park J, Jian AK (2014) Robust keypoint detection using higher-order scale space derivatives: application to image retrieval. IEEE Signal Process Lett 21(8):962–965Google Scholar
  33. 33.
    Paulin M, Douze M, Harchaoui Z, Mairal J, Perronnin F, Schmid C (2015) Local convolutional features with unsupervised training for image retrieval. Proceedings of the IEEE International Conference on Computer Vision, pp 91–99Google Scholar
  34. 34.
    Radenović F, Tolias G, Chum O (2017) Fine-tuning CNN Image Retrieval with No Human Annotation. arXiv preprint arXiv:1711.02512Google Scholar
  35. 35.
    Rakvongthai Y, Oraintara S (2013) Statistical texture retrieval in noise using complex wavelet. Signal Process Image Commun 28(10):1494–1505Google Scholar
  36. 36.
    Seetharaman K, Jeyakarthic M (2014) Statistical distributional approach for scale and rotation invariant color image retrieval using multivariate parametric tests and orthogonality condition. J Vis Commun Image Represent 25(5):727–729Google Scholar
  37. 37.
    Shahdoosti HR, Khayat O (2016) Image denoising using sparse representation classification and non-subsampled shearlet transform. Signal, Image Video Process 10(6):1081–1087zbMATHGoogle Scholar
  38. 38.
    Shu X, Wu XJ (2011) A novel contour descriptor for 2D shape matching and its applications to image retrieval. Image Vis Comput 29(4):286–294Google Scholar
  39. 39.
    Singha M, Hemachandran K, Paul A (2012) Content-based image retrieval using the combination of the fast wavelet transformation and the colour histogram. IET Image Process 6(9):1221–1229MathSciNetGoogle Scholar
  40. 40.
    Smeulders AWM, Worring M, Santini S, Gupta A (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–1380Google Scholar
  41. 41.
    Talib A, Mahmuddin M, Husni H (2013) A weighted dominant color descriptor for content-based image retrieval. J Vis Commun Image Represent 24(3):345–360Google Scholar
  42. 42.
  43. 43.
    Van De Sande K, Gevers T, Snoek C (2010) Evaluating color descriptors for object and scene recognition. IEEE Trans Pattern Anal Mach Intell 32(9):1582–1596Google Scholar
  44. 44.
    Varish N, Pradhan J, Pal AK (2017) Image retrieval based on non-uniform bins of color histogram and dual tree complex wavelet transform. Multimedia Tools & Applications 76(14):1–37Google Scholar
  45. 45.
    Vogel J, Schiele B (2006) Performance evaluation and optimization for content-based image retrieval. Pattern Recogn 39(5):897–909zbMATHGoogle Scholar
  46. 46.
    Wan J, Wang D, SCH H, Wu PC (2014) Deep learning for content-based image retrieval: a comprehensive study. Proceedings of the 22nd ACM international Conference on Multimedia, Orlando, pp 157–166Google Scholar
  47. 47.
    Wang XY, Liang LL, Li YW, Yang HY (2016) Image retrieval based on exponent moments descriptor and localized angular phase histogram. Multimed Tools Appl 76(6):7633–7659Google Scholar
  48. 48.
    Wang XY, Liu YN, Li S (2016) Robust image watermarking approach using polar harmonic transforms based geometric correction. Neurocomputing 174:627–642Google Scholar
  49. 49.
    Wang XY, Liu YN, Xu H, Wang AL (2016) Blind optimum detector for robust image watermarking in nonsubsampled shearlet Domain. Inf Sci 372:634–654Google Scholar
  50. 50.
    Wang CP, Wang XY, Li YW, Xia ZQ, Zhang C (2018) Quaternion polar harmonic Fourier moments for color images. Inf Sci 450:141–156MathSciNetGoogle Scholar
  51. 51.
    Wang CP, Wang XY, Zhang C, Xia ZQ (2016) Geometrically invariant image watermarking based on fast Radial Harmonic Fourier Moments. Signal Process-Image Commun 45:10–23Google Scholar
  52. 52.
    Wang CP, Wang XY, Zhang C, Xia ZQ (2017) Geometric correction based color image watermarking using fuzzy least squares support vector machine and Bessel K form distribution. Signal Process 134:197–208Google Scholar
  53. 53.
    Wang XY, Wu JF, Yang HY (2010) Robust image retrieval based on color histogram of local feature regions. Multimed Tools Appl 49(2):323–345Google Scholar
  54. 54.
    Wang XY, Yu YJ, Yang HY (2011) An effective image retrieval scheme using color, texture and shape features. Comput Stand Interfaces 33(1):59–68Google Scholar
  55. 55.
    Xie L, Shen J, Zhu L (2016) Online cross-modal hashing for web image retrieval. In: Proc. AAAI Conf. Artif. Intell, pp. 294–300Google Scholar
  56. 56.
    Yap PT, Jiang X, Kot AC (2010) Two-dimensional polar harmonic transforms for invariant image representation. IEEE Trans Pattern Anal Mach Intell 32(7):1259–1270Google Scholar
  57. 57.
    Yap PT, Paramesran R (2006) Content-based image retrieval using Legendre chromaticity distribution moments. IEE Proc-Vis, Image. Signal Process 153(1):17–24Google Scholar
  58. 58.
    Yu J, Liu DQ, Tao DC, Seah HS (2012) On combining multiple features for cartoon character retrieval and clip synthesis. IEEE Trans Syst, Man Cybern, Part B: Cybern 42(5):1413–1427Google Scholar
  59. 59.
    Zhao ZJ, Tian Q, Sun HD, Guo JX (2016) Content based image retrieval scheme using color, texture and shape features. Int J Signal Process Image Process. Pattern Recogn 9(1):203–212Google Scholar
  60. 60.
    Zhu L, Shen J, Xie L, Cheng Z (2017) Unsupervised topic hypergraph hashing for efficient mobile image retrieval. IEEE Transactions on Cybernetics 47(11):3941–3954Google Scholar
  61. 61.
    Zhu Z, You X, Chen CLP, Tao D, Ou W, Jiang X, Zou J (2015) An adaptive hybrid pattern for noise-robust texture analysis. Pattern Recogn 48:2592–2608Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  • Yu-Nan Liu
    • 1
    Email author
  • Shan-Shan Zhang
    • 1
  • Yu Sang
    • 2
  • Si-Miao Wang
    • 3
  1. 1.The Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, School of Computer Science and EngineeringNanjing University of Science and TechnologyNanjingPeople’s Republic of China
  2. 2.School of Electronic and Information EngineeringLiaoning Technical UniversityHuludaoChina
  3. 3.School of Computer and Information TechnologyLiaoning Normal UniversityDalianChina

Personalised recommendations

Cite article

Buy options