Multimedia Tools and Applications

, Volume 73, Issue 3, pp 1195–1224 | Cite as

Image classification without segmentation using a hybrid pyramid kernel

  • Wai-Shing Cho
  • Kin-Man LamEmail author


Image classification usually requires complicated segmentation to separate foreground objects from the background scene. However, the statistical content of a background scene can actually provide very useful information for classification. In this paper, we propose a new hybrid pyramid kernel which incorporates local features extracted from both dense regular grids and interest points for image classification, without requiring segmentation. Features extracted from dense regular grids can better capture information about the background scene, while interest points detected at corners and edges can better capture information about the salient objects. In our algorithm, these two local features are combined in both the spatial and the feature-space domains, and are organized into pyramid representations. In order to obtain better classification accuracy, we fine-tune the parameters involved in the similarity measure, and we determine discriminative regions by means of relevance feedback. From the experimental results, we observe that our algorithm can achieve a 6.37 % increase in performance as compared to other pyramid-representation-based methods. To evaluate the applicability of the proposed hybrid kernel to large-scale databases, we have performed a cross-dataset experiment and investigated the effect of foreground/background features on each of the kernels. In particular, the proposed hybrid kernel has been proven to satisfy Mercer’s condition and is efficient in measuring the similarity between image features. For instance, the computational complexity of the proposed hybrid kernel is proportional to the number of features.


Un-segmented image classification Bag-of-features Spatial-pyramid match Feature-space pyramid representation Hybrid kernel Relevance feedback Point-of-Interest 


  1. 1.
    Ardizzoni S, Bartolini I, Patella M, (1999) “Windsurf: Region-based image retrieval using wavelets.” In IWOSS’99, pp. 167Google Scholar
  2. 2.
    Bartolini I, Ciaccia P, Patella M (2010) Query processing issues in region-based image databases. In Knowledge and Information Systems 25(2):389–420CrossRefGoogle Scholar
  3. 3.
    Boughhorbel S, Tarel J-P, and Fleuret F (2004) “Non-mercer kernels for SVM object recognition.” In British Machine Vision Conference, SeptGoogle Scholar
  4. 4.
    Cho WS, Lam KM (2013) “An efficient and effective hybrid pyramid kernel for un-segmented image classification.” ICSAI 2012, pp. 2153–2158Google Scholar
  5. 5.
    Csurka G, Dance C, Fan L, Willamowski J, Bray C (2004) “Visual categorization with bags of keypoints”. In ECCV’04 workshop on Statistical Learning in Computer Vision. pp.59Google Scholar
  6. 6.
    Datta R, Joshi D, Li J, Wang JZ (2008) “Image retrieval: Ideas, influences, and trends of the new age.” ACM Comput. Surv. 40(2): Article 5, AprilGoogle Scholar
  7. 7.
    Fei-Fei L, Perona P (2005) “A Bayesian hierarchical model for learning natural scene categories”. In Proc. CVPRGoogle Scholar
  8. 8.
    Fergus R, Perona P, Zisserman A (2003) “Object class recognition by unsupervised scale-invariant learning”. In Proc. CVPR Google Scholar
  9. 9.
    Flitton G, Breckon T (2010) “Object Recognition using 3D SIFT in Complex CT Volumes.” In Proc. of the British Machine Vision Conference. pp. 11.1–12Google Scholar
  10. 10.
    Gorkani M, Picard R (1994) Texture orientation for sorting photos ‘at a glance’. In IAPR International Conference on Pattern Recognition 1:459–464Google Scholar
  11. 11.
    Grauman K, Darrell T (2005) “Pyramid match kernels: Discriminative classification with sets of image features.” In Proc. ICCVGoogle Scholar
  12. 12.
    Grauman K, Darrell T (2006) “Unsupervised learning of categories from sets of partially matching image features”. In CVPRGoogle Scholar
  13. 13.
    Kondor R, Jebara T (2003) “A kernel between sets of vectors,” In Proccedings of International Conference on Machine Learning, Washington, D.C., AugGoogle Scholar
  14. 14.
    Kovashka A, Parikh D, Grauman K (2013) “WhittleSearch: Image Search with Relative Attribute Feedback.” In Proc. CVPR, JuneGoogle Scholar
  15. 15.
    Lazebnik S, Schmid C, Ponce J, (2003) “Affine-invariant local descriptors and neighborhood statistics for texture recognition”. In Proc. of the IEEE International Conf. on Computer Vision, pp. 649–655Google Scholar
  16. 16.
    Lazebnik S, Schmid C, Ponce J (2006) “Beyond bags of features: spatial pyramid matching for recognizing natural scene categories.” In Proc. of CVPRGoogle Scholar
  17. 17.
    Li F, Carreira J, Sminchisescu C (2010) “Object Recognition as Ranking Holistic Figure-Ground Hypotheses”. In Proc.of CVPR 2010, pp.1712–1719Google Scholar
  18. 18.
    Lin Y, Lv F, Zhu S, Yang M, Cour T, Yu Kai, Cao Liangliang, Huang T (2011) “Large-scale image classification: Fast feature extraction and SVM training.” In Proc. CVPR, pp.1689–1696Google Scholar
  19. 19.
    Lowe D (1999) “Object recognition from local scale-invariant features.” In Proc. of the International Conference on Computer Vision. pp. 1150–1157Google Scholar
  20. 20.
    Lowe D (2000) “Towards a computational model for object recognition in IT cortex.” In Biologically Motivated Computer Vision pp. 20–31Google Scholar
  21. 21.
    Lowe D (2004) Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision 60(2):91–110CrossRefGoogle Scholar
  22. 22.
    Lyu S (2005) “Mercer kernels for object recognition with local features”. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition, JunGoogle Scholar
  23. 23.
    Malik J, Belongie S, Leung T, Shi J (2001) Contour and Texture Analysis for Image Segmentation. International Journal of Computer Vision 43(1):7–27CrossRefzbMATHGoogle Scholar
  24. 24.
    Moreno P, Ho P, Vasconcelos N, (2003) “A Kullback–Leibler divergence based kernel for SVM classification in multimedia applications.” In NIPS, Vancouver, DecGoogle Scholar
  25. 25.
    Niebles JC, Fei-Fei L (2007) “A hierarchical model model of shape and appearance for human action classification.” In Proc. of IEEE Computer Vision and Pattern RecognitionGoogle Scholar
  26. 26.
    Qu Y, Wu S, Liu H, Xie Y, Wang H (2012) “Evaluation of local features and classifiers in BOW model for image classification”. Journal of Multimedia Tools and Applications. doi: 10.1007/s11042-012-1107-z Google Scholar
  27. 27.
    Rubner Y, Tomasi C, Guibas LJ (2000) “The Earth Mover’s Distance as a metric for image retrieval.” International Journal of Computer Vision 40(2)Google Scholar
  28. 28.
    Shawe-Taylor J, Cristianini N (2004)“Kernel Methods for Pattern Analysis”. Cambridge University PressGoogle Scholar
  29. 29.
    Shi J, Malik J (2000) “Normalized Cuts and Image Segmentation”. IEEE Transactions on Pattern Analysis and Machine Intelligence 22: (8) AugustGoogle Scholar
  30. 30.
    Sivic J, Russell B, Efros A, Zisserman A, Freeman W (2005) “Discovering objects and their localization in images.” In Proc. of ICCVGoogle Scholar
  31. 31.
    Squire D, Muller W, Muller H, Raki J (1999) “Content-based query of image databases, inspirations from text retrieval: inverted files, frequency-based weights and relevance feedback”. in Proceedings of the 11th Scandinavian conference on image analysis. pp. 143–149Google Scholar
  32. 32.
    Szummer M, Picard R (1998) “Indoor-outdoor image classification”. In IEEE International Workshop on Content-Based Access of Image and Video Databases pp. 42–51Google Scholar
  33. 33.
    Tong S, Chang E (2001) “Support vector machine active leaning for image retrieval”. In Proc. of the 9th ACM Conference on Multimedia. Ottawa CanadaGoogle Scholar
  34. 34.
    Torralba A, Murphy K, Freeman W, Rubin M (2003) “Context-based vision system for place and object recognition.” In Proc. ICCVGoogle Scholar
  35. 35.
    Wallraven C, Caputo B, Graf A (2003) “Recognition with local features: the Kernel Recipe.” In Proc. IEEE International Conf. on Computer Vision, OctGoogle Scholar
  36. 36.
    Wang X-Y, Zhang B-B, Yang H-Y (2012) “Content-based image retrieval by integrating color and texture features”. Journal of Multimedia Tools and Applications. doi: 10.1007/s11042-012-1055-7 Google Scholar
  37. 37.
    Wolf L, Shashua A (Dec 2003) Learning over sets using kernel principal angles. Journal of Machine Learning Research 4:913–931Google Scholar
  38. 38.
    Yu S, Shi J (2004) “Segmentation Given Partial Grouping Constraints.” IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(2):FebruaryGoogle Scholar
  39. 39.
    Zhang J, Marszalek M, Lazebnik S, Schmid C (2005) “Local features and kernels for classification of texture and object categories: An in-depth study”, Technical Report RR-5737. INRIA, Rhône-AlpesGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.Centre for Signal Processing, Department of Electronic and Information EngineeringThe Hong Kong Polytechnic UniversityHung HomHong Kong

Personalised recommendations