Skip to main content

Advertisement

SpringerLink
Log in

Content-based image retrieval via a hierarchical-local-feature extraction scheme

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Recently, with the development of various camera sensors and internet network, the volume of digital images is becoming big. Content-based image retrieval (CBIR), especially in network big data analysis, has attracted wide attention. CBIR systems normally search the most similar images to the given query example among a wide range of candidate images. However, human psychology suggests that users concern more about regions of their interest and merely want to retrieve images containing relevant regions, while ignoring irrelevant image areas (such as the texture regions or background). Previous CBIR system on user-interested image retrieval generally requires complicated segmentation of the region from the background. In this paper, we propose a novel hierarchical-local-feature extraction scheme for CBIR, whereas complex image segmentation is avoided. In our CBIR system, a perception-based directional patch extraction method and an improved salient patch detection algorithm are proposed for local features extraction. Then, color moments and Gabor texture features are employed to index the salient regions. Extensive experiments have been performed to evaluate the performance of the proposed scheme, and experimental results show that the developed CBIR system produces plausible retrieval results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Alzu’bi A, Amira A, Ramzan N (2015) Semantic content-based image retrieval: a comprehensive study. J Vis Commun Image Represent 32:20–54

    Article  Google Scholar 

  2. Carson C, Belongie S, Greenspan H (2002) Blobworld: image segmentation using expectation-maximization and its application to image querying. IEEE Trans PAMI 24(8):1026–1038

    Article  Google Scholar 

  3. Corel: Image Library University of California, Berkely. http://calphotos.berkeley.edu/use.html#download

  4. Daubechies I (1988) Orthonormal bases of compactly supported wavelets. Commun Pure Appl Math 41:909–996

    Article  MathSciNet  Google Scholar 

  5. ElAlami ME (2014) A new matching strategy for content based image retrieval system. Appl Soft Comput 14:407–418

    Article  Google Scholar 

  6. Elleuch N, Ben Ammar A, Alimi AM (2015) A generic framework for semantic video indexing based on visual concepts/contexts detection. Multimed Tools Appl 74(4):1397–1421

    Article  Google Scholar 

  7. Ester M, Kriegel HP, Sander J, Xu X (1996) A density based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining. Portland: AAAI Press, pp. 226–231

  8. Fauqueur J, Boujemaa N (2002) Region-based retrieval: coarse segmentation with fine signature, IEEE ICIP, Rochester, NY, USA

  9. Fauqueur J, Boujemaa N (2004) Region-based image retrieval: fast coarse segmentation and fine color description. J Vis Lang Comput 15:69–95

    Article  Google Scholar 

  10. Gao L, Guo Z, Zhang H, Xu X, Shen HT (Sep. 2017) Video captioning with attention-based LSTM and semantic consistency. IEEE Trans Multimed 19(9):2045–2055

    Article  Google Scholar 

  11. Gouet V, Boujemaa N (2001) Object-based queries using color points of interest. IEEE Workshop on Content-based Access of Image and Video Labraries, vol. 1, p 30–36

  12. Ground Truth Database: Department of Computer Science and Engineering, University of Washington. http://www.cs.washington.edu/research/imagedatabase/groundtruth/_tars.for.download/

  13. Jain AK, Farroknia F (1991) Unsupervised texture segmentation using Gabor filters. Pattern Recogn 24(12):1167–1186

    Article  Google Scholar 

  14. Jian MW, Dong JY (2007) Wavelet-Based Salient Regions and their Spatial Distribution for Image Retrieval, IEEE International Conference on Multimedia & Expo., p 2194–2197, 2–5 July

  15. Jian M, Lam K-M (2014) Face-image retrieval based on singular values and potential-field representation. Signal Process 100:9–15

    Article  Google Scholar 

  16. Jian M, Guo H, Liu L (2009) Texture classification using visual perceptual texture features and Gabor wavelet features. J Comput 4(8):763–770

    Article  Google Scholar 

  17. Jian M, Dong J, Ma J (2011) Image retrieval using wavelet-based salient regions. Imaging Sci J 59(4):219–231

    Article  Google Scholar 

  18. Jian M, Lam K-M, Dong J (2014) Facial-feature detection and localization based on a hierarchical scheme. Inf Sci 262:1–14

    Article  Google Scholar 

  19. Jian M, Lam K-M, Dong J (2014) Illumination-insensitive texture discrimination based on illumination compensation and enhancement. Inf Sci 269:60–72

    Article  MathSciNet  Google Scholar 

  20. Jian M, Lam K-M, Dong J, Shen L (2015) Visual-patch- attention-aware saliency detection. IEEE Trans Cybern 45(8):1575–1586

    Article  Google Scholar 

  21. Jiji GW, DuraiRaj PJ (2015) Content-based image retrieval techniques for the analysis of dermatological lesions using particle swarm optimization technique. Appl Soft Comput 30:650–662

    Article  Google Scholar 

  22. Lan R, Zhou Y, Tang YY (2016) Quaternionic local ranking binary pattern: a local descriptor of color images. IEEE Trans Image Process 25(2):566–579

    Article  MathSciNet  Google Scholar 

  23. Lau HF, Levine MD (2002) Finding a small number of regions in an image using low-level features. Pattern Recogn 35(11):2323–2339

    Article  Google Scholar 

  24. Liu G, Yang J (2013) Content-based image retrieval using color difference histogram. Pattern Recogn 46(1):188–198

    Article  Google Scholar 

  25. Liu Y, Zhang D, Lu G, Ma WY (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recogn 40(1):262–282

    Article  Google Scholar 

  26. Liu F, Zhang D, Shen L (2015) Study on novel curvature features for 3D fingerprint recognition. Neurocomputing 168(1):599–608

    Article  Google Scholar 

  27. Long F, Zhang HJ, Feng DD (2003) Fundamentals of content-based image retrieval. In: Feng D, Siu WC, Zhang HJ (eds) Multimedia information retrieval and management-technological fundamentals and applications. Springer, Berlin

    MATH  Google Scholar 

  28. Mallat S (1989) A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans PAMI 11(7):674–693

    Article  Google Scholar 

  29. McGill Calibrated Colour Image Database http://tabby.vision.mcgill.ca/html/browsedownload.html

  30. Muwei J, Dong J (2007) Combining color, texture and region with objects of user’s interest for content-based image retrieval. Eighth ACIS International Conference on SNPD, p 713–718

  31. Pavlidis T (2008) Limitations of content-based image retrieval. ICPR. http://www.theopavlidis.com/technology/CBIR/PaperB/vers3.htm

  32. Sander J, Ester M, Kriegel HP et al (1998) Density-based clustering in spatial databases: the algorithm GDBSCAN and its applications. Data Min Knowl Disc 2(2):169–194

    Article  Google Scholar 

  33. Sebe N, Lew MS (2003) Comparing salient points detectors. Pattern Recogn Lett 24(1–3):89–96

    Article  Google Scholar 

  34. Sebe N, Tian Q, Loupias E, Lew MS, Huang TS (2001) Content-based Retrieval using Salient Point Techniques, IEEE Conference on Computer Vision and Pattern Recognition (CVPR'01), Technical Demo, Electronic Proceedings, Kauai, Hawaii

  35. Sebe N, Tian Q, Loupias E, Lew MS, Huang TS (2003) Evaluation of salient point techniques. J Image Vision Comput 21(13–14):1087–1095

    Article  Google Scholar 

  36. Shen L, Bai L (2008) 3D Gabor wavelets for evaluating SPM normalization algorithm. Med Image Anal 12(3):375–383

    Article  Google Scholar 

  37. SIMPLIcity Image Database: http://wang.ist.psu.edu/docs/related/

  38. J. Song, Lianli Gao, Xiaofeng Zhu, Nicu Sebe (2017) Quantization based hashing: a general framework for scalable image and video retrieval. Pattern Recogn

  39. Song J, He T, Gao L, Xu X, Shen H (2018) Deep region hashing for efficient large-scale instance search from images. AAAI

  40. Sudhakar MS, Bagan KB (2014) An effective biomedical image retrieval framework in a fuzzy feature space employing phase congruency and GeoSOM. Appl Soft Comput 22:492–503

    Article  Google Scholar 

  41. Taylor JR (1997) An introduction to error analysis, 2nd edn. University Science Books, Sausolito, California

    Google Scholar 

  42. Tian Q, Sebe N, Loupias E, Lew MS, Huang TS (2001) Image retrieval using wavelet-based salient points. J Electron Imaging 835–849

  43. Tsai HH, Chang BM, Liou SH (2014) Rotation-invariant texture image retrieval using particle swarm optimization and support vector regression. Appl Soft Comput 17:127–139

    Article  Google Scholar 

  44. Vieux R, Benois-Pineau J, Domenger J-P (2012) Content based image retrieval using bag-of-regions, 18th International Conference, MMM 2012, Klagenfurt, Austria, January 4–6, pp. 507–517

  45. Walia E, Pal A (2014) Fusion framework for effective color image retrieval. J Vis Commun Image Represent 25(6):1335–1348

    Article  Google Scholar 

  46. Wang Q, Yuan Y, Yan P, Li X (2013) Visual saliency by selective contrast. IEEE Trans Circ Syst Vid Technol 23(7):1150–1155

    Article  Google Scholar 

  47. Wang Q, Yuan Y, Yan P, Li X (2013) Saliency detection by multiple-instance learning. IEEE Trans Cybern 43(2):660–672

    Article  Google Scholar 

  48. Wang Q, Wan J, Yuan Y (2018) Locality constraint distance metric learning for traffic congestion detection. Pattern Recogn 75:272–281

    Article  Google Scholar 

  49. Wang X, Gao L, Wang P, Sun X, Liu X (2018) Two-stream 3D convNet fusion for action recognition in videos with arbitrary size and length. IEEE Trans Multimed 20(3):634–644

    Article  Google Scholar 

  50. Wang J, Zhang T, Song J, Sebe N, Shen H (2018) A survey on learning to hash. IEEE Trans Pattern Anal Mach Intell 40(4):769–790

    Article  Google Scholar 

  51. Wang Q, Wan J, Yuan Y Deep metric learning for crowdedness regression. IEEE Trans. Circ Syst Vid Technol. https://doi.org/10.1109/TCSVT.2017.2703920

  52. Xu Y-Y (2016) Multiple-instance learning based decision neural networks for image retrieval and classification. Neurocomputing 171:826–836

    Article  Google Scholar 

  53. Yang M, Zhu P, Liu F, Shen L (2015) Joint representation and pattern learning for robust face recognition. Neurocomputing 168(30):70–80

    Article  Google Scholar 

  54. Zhu Z, Jia S, He S, Sun Y, Ji Z, Shen L (2015) Three-dimensional Gabor feature extraction for hyperspectral imagery classification using a memetic framework. Inf Sci 298(1):274–287

    Article  Google Scholar 

  55. Zhu Y, Jiang J, Han W, Ding Y, Tian Q (2017) Interpretation of users’ feedback via swarmed particles for content-based image retrieval. Inf Sci 375:246–257

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by National Natural Science Foundation of China (NSFC) (61601427, 61602229, 61771230); Natural Science Foundation of Shandong Province (ZR2015FQ011, ZR2016FM40); Shandong Provincial Key Research and Development Program of China (NO. 2017CXGC0701); Fostering Project of Dominant Discipline and Talent Team of Shandong Province Higher Education Institutions.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, China

    Muwei Jian

  2. School of Software Engineering, Shandong University, Jinan, 250101, China

    Yilong Yin

  3. Department of Computer Science and Technology, Ocean University of China, Qingdao, China

    Junyu Dong

  4. Centre for Signal Processing, Department of Electronic and Information Engineering, The Hong Kong Polytechnic University, Kowloon, Hong Kong

    Kin-Man Lam

Authors
  1. Muwei Jian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yilong Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junyu Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kin-Man Lam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muwei Jian.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jian, M., Yin, Y., Dong, J. et al. Content-based image retrieval via a hierarchical-local-feature extraction scheme. Multimed Tools Appl 77, 29099–29117 (2018). https://doi.org/10.1007/s11042-018-6122-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6122-2

Keywords

Search

Navigation