Content-based image retrieval using feature weighting and C-means clustering in a multi-label classification framework

Abstract

In this paper, a novel learning algorithm based on feature weighting is proposed to improve the performance of image classification or retrieval systems in a multi-label framework. The goal is to exploit maximally the beneficial properties of each feature in the system. Since each feature can separate more effectively some of the image classes, it is hypothesized that the weights of various features at some states can be traded off against each other. The training phase of the suggested algorithm is performed in two stages: (1) The input images are clustered using a supervised C-means method iteratively; (2) image features are weighted using a local feature weighting method in each cluster. These weights are determined by considering the importance of each feature in minimizing the classification error on each cluster. In the testing phase, the cluster corresponding to the query is found first. Then, the most similar images are retrieved in the multi-label framework using the feature weights assigned to that cluster. Experimental results on three well-known, public and international image datasets demonstrate that our proposed method leads to significant performance gains over existing methods.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3

References

  1. 1.

    Irawan C, Listyaningsih W, Sari CA, Rachmawanto EH (2018) CBIR for herbs root using color histogram and GLCM based on K-nearest neighbor. In: 2018 International seminar on application for technology of information and communication. IEEE, pp 509–514

  2. 2.

    Singh S, Rajput ER (2015) Content based image retrieval using SVM, NN and KNN classification. Int J Adv Res Comput Commun Eng 4(6):549–552

  3. 3.

    Dharani T, Aroquiaraj IL (2013) Content based image retrieval system using feature classification with modified KNN algorithm. arXiv preprint arXiv:1307.4717

  4. 4.

    Zhou W, Li H, Tian Q (2017) Recent advance in content-based image retrieval: a literature survey. arXiv preprint arXiv:1706.06064

  5. 5.

    Tzelepi M, Tefas A (2018) Deep convolutional learning for content based image retrieval. Neurocomputing 275:2467–2478

    Article  Google Scholar 

  6. 6.

    Wan J, Wang D, Hoi SCH, Wu P, Zhu J, Zhang Y, Li J (2014). Deep learning for content-based image retrieval: a comprehensive study. In: Proceedings of the 22nd ACM international conference on multimedia. ACM, pp 157–166

  7. 7.

    Yu J, Yang X, Gao F, Tao D (2016) Deep multimodal distance metric learning using click constraints for image ranking. IEEE Trans Cybern 47(12):4014–4024

    Article  Google Scholar 

  8. 8.

    Yu J, Tao D, Wang M, Rui Y (2014) Learning to rank using user clicks and visual features for image retrieval. IEEE Trans Cybern 45(4):767–779

    Article  Google Scholar 

  9. 9.

    Qayyum A, Anwar SM, Awais M, Majid M (2017) Medical image retrieval using deep convolutional neural network. Neurocomputing 266:8–20

    Article  Google Scholar 

  10. 10.

    Sadeghi-Tehran P, Angelov P, Virlet N, Hawkesford MJ (2019) Scalable database indexing and fast image retrieval based on deep learning and hierarchically nested structure applied to remote sensing and plant biology. J. Imaging 5(3):33

    Article  Google Scholar 

  11. 11.

    Sarwar A, Mehmood Z, Saba T, Qazi KA, Adnan A, Jamal H (2019) A novel method for content-based image retrieval to improve the effectiveness of the bag-of-words model using a support vector machine. J Inf Sci 45(1):117–135

    Article  Google Scholar 

  12. 12.

    Tsai CF (2012) Bag-of-words representation in image annotation: a review. ISRN Artif Intell 2:1–19

  13. 13.

    Xu D, Yan S, Tao D, Lin S, Zhang HJ (2007) Marginal fisher analysis and its variants for human gait recognition and content-based image retrieval. IEEE Trans Image Process 16(11):2811–2821

    MathSciNet  Article  Google Scholar 

  14. 14.

    Da Silva SF, Avalhais LP, Batista MA, Barcelos CA, Traina AJ (2014) Findings on ranking evaluation functions for feature weighting in image retrieval. J Braz Comput Soc 20(1):7

    MathSciNet  Article  Google Scholar 

  15. 15.

    Chathurani NWUD, Geva S, Chandran V, Rajapaksha P (2016) Image retrieval based on multi-feature fusion for heterogeneous image databases. Int J Comput Inf Eng 10(10):1797–1802

    Google Scholar 

  16. 16.

    Cordeiro De Amorim R, Mirkin B (2012) Minkowski metric, feature weighting and anomalous cluster initialisation in K-means clustering. Pattern Recognit 45(3):1061–1075

    Article  Google Scholar 

  17. 17.

    Modha DS, Spangler WS (2003) Feature weighting in k-means clustering. Mach Learn 52(3):217–237

    Article  Google Scholar 

  18. 18.

    Saha A, Das S (2015) Automated feature weighting in clustering with separable distances and inner product induced norms: a theoretical generalization. Pattern Recognit Lett 63:50–58

    Article  Google Scholar 

  19. 19.

    Chen X, Ye Y, Xu X, Huang JZ (2012) A feature group weighting method for subspace clustering of high-dimensional data. Pattern Recognit 45(1):434–446

    Article  Google Scholar 

  20. 20.

    Magesan E, Gambetta JM, Córcoles AD, Chow JM (2015) Machine learning for discriminating quantum measurement trajectories and improving readout. Phys Rev Lett 114(20):200501

    Article  Google Scholar 

  21. 21.

    Ghodratnama S, Boostani R (2015) An efficient strategy to handle complex datasets having multimodal distribution. In: ISCS 2014: interdisciplinary symposium on complex systems. Springer, Cham, pp 153–163

  22. 22.

    Tsoumakas G, Katakis I, Vlahavas I (2010) Random k-labelsets for multilabel classification. IEEE Trans Knowl Data Eng 23(7):1079–1089

    Article  Google Scholar 

  23. 23.

    Younes Z, Abdallah F, Denœux T (2009) An evidence-theoretic k-nearest neighbor rule for multi-label classification. In: International conference on scalable uncertainty management. Springer, Berlin, pp 297–308

  24. 24.

    Yu Y, Pedrycz W, Miao D (2014) Multi-label classification by exploiting label correlations. Expert Syst Appl 41(6):2989–3004

    Article  Google Scholar 

  25. 25.

    Jiang JY, Tsai SC, Lee SJ (2012) FSKNN: multi-label text categorization based on fuzzy similarity and k nearest neighbors. Expert Syst Appl 39(3):2813–2821

    Article  Google Scholar 

  26. 26.

    Vens C, Struyf J, Schietgat L, Džeroski S, Blockeel H (2008) Decision trees for hierarchical multi-label classification. Mach Learn 73(2):185

    Article  Google Scholar 

  27. 27.

    Wang M, Zhou X, Chua TS (2008) Automatic image annotation via local multi-label classification. In: Proceedings of the 2008 international conference on Content-based image and video retrieval. ACM, pp 17–26

  28. 28.

    Lin Z, Ding G, Hu M, Wang J (2014) Multi-label classification via feature-aware implicit label space encoding. In: International conference on machine learning, pp 325–333

  29. 29.

    Zhou ZH, Zhang ML, Huang SJ, Li YF (2012) Multi-instance multi-label learning. Artif Intell 176(1):2291–2320

    MathSciNet  Article  Google Scholar 

  30. 30.

    Duda RO, Hart PE, Stork DG (1973) Pattern classification and scene analysis, vol 3. Wiley, New York

    MATH  Google Scholar 

  31. 31.

    Celebi ME, Kingravi HA, Vela PA (2013) A comparative study of efficient initialization methods for the k-means clustering algorithm. Expert Syst Appl 40(1):200–210

    Article  Google Scholar 

  32. 32.

    Paredes R, Vidal E (2006) Learning weighted metrics to minimize nearest-neighbor classification error. IEEE Trans Pattern Anal Mach Intell 7:1100–1110

    Article  Google Scholar 

  33. 33.

    Sharma A, Dey S (2012) A comparative study of feature selection and machine learning techniques for sentiment analysis. In: Proceedings of the 2012 ACM research in applied computation symposium. ACM, pp 1–7

  34. 34.

    Bouaguel W, Mufti GB, Limam M (2013) A fusion approach based on wrapper and filter feature selection methods using majority vote and feature weighting. In 2013 International conference on computer applications technology (ICCAT). IEEE, pp 1–6

  35. 35.

    Moghaddam HA, Dehaji MN (2013) Enhanced Gabor wavelet correlogram feature for image indexing and retrieval. Pattern Anal Appl 16(2):163–177

    MathSciNet  Article  Google Scholar 

  36. 36.

    Shad SM (2011). Color image indexing and retrieval using wavelet correlogram. M.Sc. thesis in artificial intelligence and robotics, faculty of computer engineering, K.N. Toosi University of Technology, Tehran, Iran

  37. 37.

    Huang J, Ravi Kumar S, Mitra M, Zhu WJ, Zabih R (1997) Image indexing using color correlograms. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 1:762–768

  38. 38.

    Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recognit 37(9):1757–1771

    Article  Google Scholar 

  39. 39.

    Tsoumakas G, Katakis I (2007) Multi-label classification: an overview. Int J Data Warehous Min 3(3):1–13

    Article  Google Scholar 

  40. 40.

    Yang Y (1999) An evaluation of statistical approaches to text categorization. Inf Retrieval 1(1–2):69–90

    Article  Google Scholar 

  41. 41.

    Guldogan E, Gabbouj M (2008) Feature selection for content-based image retrieval. SIViP 2(3):241–250

    Article  Google Scholar 

  42. 42.

    Huang ZC, Chan PP, Ng WW, Yeung DS (2010) Content-based image retrieval using color moment and Gabor texture feature. In: 2010 International conference on machine learning and cybernetics, vol 2. IEEE, pp 719–724

  43. 43.

    Puviarasan N, Bhavani R, Vasanthi A (2014) Image retrieval using combination of texture and shape features. Int J Adv Res Comput Commun Eng 3(3)

  44. 44.

    Singha M, Hemachandran K (2012) Content based image retrieval using color and texture. Signal Image Process 3(1):39–57

    Google Scholar 

  45. 45.

    Lin CH, Chen RT, Chan YK (2009) A smart content-based image retrieval system based on color and texture feature. Image Vis Comput 27(6):658–665

    Article  Google Scholar 

  46. 46.

    Raghupathi G, Anand RS, Dewal ML (2010) Color and texture features for content based image retrieval. In: Second international conference on multimedia and content based image retrieval

  47. 47.

    Hiremath PS, Pujari J (2007) Content based image retrieval based on color, texture and shape features using image and its complement. Int J Comput Sci Secur 1(4):25–35

    Google Scholar 

  48. 48.

    Rao MB, Rao BP, Govardhan A (2011) CTDCIRS: content based image retrieval system based on dominant color and texture features. Int J Comput Appl 18(6):40–46

    Google Scholar 

  49. 49.

    Li L, Feng L, Yu L, Wu J, Liu S (2016) Fusion framework for color image retrieval based on bag-of-words model and color local Haar binary patterns. J Electron Imaging 25(2):023022

    Article  Google Scholar 

  50. 50.

    Ali N, Mazhar DA, Iqbal Z, Ashraf R, Ahmed J, Khan FZ (2017) Content-based image retrieval based on late fusion of binary and local descriptors. arXiv preprint arXiv:1703.08492

  51. 51.

    Ali N, Bajwa KB, Sablatnig R, Chatzichristofis SA, Iqbal Z, Rashid M, Habib HA (2016) A novel image retrieval based on visual words integration of SIFT and SURF. PLoS ONE 11(6):e0157428

    Article  Google Scholar 

  52. 52.

    Montazer GA, Giveki D (2015) An improved radial basis function neural network for object image retrieval. Neurocomputing 168:221–233

    Article  Google Scholar 

  53. 53.

    Tian X, Jiao L, Liu X, Zhang X (2014) Feature integration of EODH and Color-SIFT: application to image retrieval based on codebook. Sig Process Image Commun 29(4):530–545

    Article  Google Scholar 

  54. 54.

    Moghaddam HA, Ghodratnama S (2017) Toward semantic content-based image retrieval using Dempster-Shafer theory in multi-label classification framework. Int J Multimedia Inf Retr 6(4):317–326

    Article  Google Scholar 

  55. 55.

    Wu B, Lyu S, Ghanem B (2016) Constrained submodular minimization for missing labels and class imbalance in multi-label learning. In: Thirtieth AAAI conference on artificial intelligence

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Hamid Abrishami Moghaddam.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ghodratnama, S., Abrishami Moghaddam, H. Content-based image retrieval using feature weighting and C-means clustering in a multi-label classification framework. Pattern Anal Applic 24, 1–10 (2021). https://doi.org/10.1007/s10044-020-00887-4

Download citation

Keywords

  • CBIR
  • Multi-label
  • Feature weighting
  • C-means clustering
  • KNN classification