Multimedia Tools and Applications

, Volume 63, Issue 3, pp 875–897 | Cite as

Simple object recognition based on spatial relations and visual features represented using irregular pyramids

  • Annette Morales-González
  • Edel B. García-Reyes


Spatial relations among objects and object parts play a fundamental role in the human perception and understanding of images, thus becoming very relevant in the computational fields of object recognition, scene understanding and content-based image retrieval. In this work we propose a graph matching scheme that involves color, texture and shape features along with spatial descriptors to represent topological and orientation/directional relationships—which are obtained by means of combinatorial pyramids—in order to identify similar objects from a database. We also suggest a method for deciding which are the more useful levels in the hierarchy of segmentation for the recognition process. Our main objective is to prove that the combination of visual and spatial features is a promising road in order to improve the object recognition task. We performed experiments on two well known databases, COIL-100 and ETH-80 image sets, in order to evaluate the expressiveness of the proposed representation. These sets introduce challenges for simple object recognition in terms of view-point changes, and our results were comparable or superior than other state-of-the-art methods.


Object recognition Spatial relations Topological relations Structure matching Graph matching 


  1. 1.
    Arif T, Shaaban Z, Krekor L, Baba S (2009) Object classification via geometrical, zernike and legendre moments. JATIT 7(1):31–37Google Scholar
  2. 2.
    Brun L, Kropatsch W (2001) Introduction to combinatorial pyramids. Digital and image geometry. Springer-Verlag, New York, pp 108–128.
  3. 3.
    Brun L, Kropatsch W (2003) Contraction kernels and combinatorial maps. Pattern Recogn Lett 24(8):1051–1057. doi: 10.1016/S0167-8655(02)00251-9 MATHCrossRefGoogle Scholar
  4. 4.
    Brun L, Kropatsch W (2006) Contains and inside relationships within combinatorial pyramids. Pattern Recogn 39(4):515–526. doi: 10.1016/j.patcog.2005.10.015 MATHCrossRefGoogle Scholar
  5. 5.
    Cheriet M, Kharma N, Liu Cl, Suen C (2007) Character recognition systems: a guide for students and practitioners. Wiley-InterscienceGoogle Scholar
  6. 6.
    Duval MA, Vega-Pons S, Llano EG (2010) Experimental comparison of orthogonal moments as feature extraction methods for character recognition. In: CIARP’10 proceedings, pp 394–401. doi: 10.1007/978-3-642-16687-7_53
  7. 7.
    Egenhofer MJ, Sharma J, Mark DM (1993) A critical comparison of the 4-intersection and 9-intersection models for spatial relations: formal analysis. In: Autocarto 11, pp 1–11Google Scholar
  8. 8.
    Felzenszwalb PF, Girshick RB, McAllester DA, Ramanan D (2010) Object detection with discriminatively trained part-based models. IEEE Trans Pattern Anal Mach Intell 32(9):1627–1645. doi:10.1109/TPAMI.2009.167 CrossRefGoogle Scholar
  9. 9.
    Fischer B, Thies C, Güld MO, Lehmann TM (2004) Content-based image retrieval by matching hierarchical attributed region adjacency graphs. In: Proc. SPIE–medical imaging: image processing, vol 5370, pp 598–606Google Scholar
  10. 10.
    Grauman K, Darrell T (2005) Pyramid match kernels: discriminative classification with sets of image features. Tech. Rep. MIT-CSAIL-TR-2005-017, Massachusetts Institute of Technology, CambridgeGoogle Scholar
  11. 11.
    Grauman K, Darrell T (2007) The pyramid match kernel: efficient learning with sets of features. J Mach Learn Res 8:725–760MATHGoogle Scholar
  12. 12.
    Guting RH, Iv PI, Hagen F (1994) An introduction to spatial database systems. VLDB J 3:357–399CrossRefGoogle Scholar
  13. 13.
    Hadjidemetriou E, Grossberg M, Nayar S (2001) Spatial information in multi-resolution histograms. In: IEEE conference on computer vision and pattern recognition (CVPR), vol I, pp 702–709Google Scholar
  14. 14.
    Haxhimusa Y, Kropatsch WG (2004) Segmentation graph hierarchies. In: Fred A, Caelli T, Duin RP, Campilho A, de Ridder D (eds) Proceedings of joint international workshops on structural, syntactic, and statistical pattern recognition S+SSPR. Springer, Berlin Heidelberg, New YorkGoogle Scholar
  15. 15.
    Heikkilä M, Pietikäinen M, Schmid C (2009) Description of interest regions with local binary patterns. Pattern Recogn 42(3):425–436. doi: 10.1016/j.patcog.2008.08.014 MATHCrossRefGoogle Scholar
  16. 16.
    Hernández-Gracidas C, Sucar LE (2007) Markov random fields and spatial information to improve automatic image annotation. In: PSIVT, pp 879–892.  10.1007/978-3-540-77129-6_74
  17. 17.
    Hodé Y, Deruyver A (2007) Qualitative spatial relationships for image interpretation by using semantic graph. In: GbRPR, pp 240–250.  10.1007/978-3-540-72903-7_22
  18. 18.
    Hsieh JW, Grimson WEL (2003) Spatial template extraction for image retrieval by region matching. IEEE Trans Image Process 12(11):1404–1415. doi: 10.1109/TIP.2003.816013 CrossRefGoogle Scholar
  19. 19.
    Hurtut T, Gousseau Y, Schmitt F (2008) Adaptive image retrieval based on the spatial organization of colors. Comput Vis Image Underst 112(2):101–113. doi: 10.1016/j.cviu.2007.12.006 CrossRefGoogle Scholar
  20. 20.
    Iglesias-Ham M, Bazán-Pereira Y, García-Reyes EB (2007) A multiple substructure matching algorithm for fingerprint verification. In: CIARP’07 proceedings. Springer-Verlag, pp 172–181Google Scholar
  21. 21.
    Illetschko T, Ion A, Haxhimusa Y, Kropatsch WG (2006) Effective programming of combinatorial maps using coma - a c+ + framework for combinatorial maps. Tech. Rep. PRIP-TR-106, PRIP, TU WienGoogle Scholar
  22. 22.
    Kropatsch WG, Haxhimusa Y, Lienhardt P (2004) Cognitive vision systems: sampling the spectrum of approaches, chap 13. Hierarchies relating Topology and Geometry. Lecture Notes in Computer Science. Springer, Berlin Heidelberg, DagstuhlGoogle Scholar
  23. 23.
    Kropatsch WG, Haxhimusa Y, Pizlo Z, Langs G (2005) Vision pyramids that do not grow too high. Pattern Recogn Lett 26:319–337CrossRefGoogle Scholar
  24. 24.
    Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: CVPR ’06: proceedings of the 2006 IEEE computer society conference on computer vision and pattern recognition. IEEE Computer Society, pp 2169–2178. doi: 10.1109/CVPR.2006.68
  25. 25.
    Leibe B, Schiele B (2003) Analyzing appearance and contour based methods for object categorization. In: IEEE conference on computer vision and pattern recognition (CVPR’03), pp 409–415Google Scholar
  26. 26.
    Lin PL, Tan WH (2003) An efficient method for the retrieval of objects by topological relations in spatial database systems. Inf Process Manag 39(4):543–559. doi: 10.1016/S0306-4573(02)00034-1 MATHCrossRefGoogle Scholar
  27. 27.
    Marée, R, Geurts P, Piater J, Wehenkel L (2005) Decision trees and random subwindows for object recognition. In: ICML workshop on machine learning techniques for processing multimedia content (MLMM2005).
  28. 28.
    Markman AB, Gentner D (2000) Structure mapping in the comparison process. Am J Psychol 113(4):501–538CrossRefGoogle Scholar
  29. 29.
    Morales-González A, García-Reyes EB (2010) Assessing the role of spatial relations for the object recognition task. In: CIARP’10 proceedings, pp 549–556. doi: 10.1007/978-3-642-16687-7_72
  30. 30.
    Morioka N (2008) Learning object representations using sequential patterns. In: Proceedings of the 21st Australasian joint conference on artificial intelligence: advances in artificial intelligence, AI ’08, pp 551–561Google Scholar
  31. 31.
    Nene SA, Nayar SK, Murase H (1996) Columbia object image library (COIL-100). Tech repGoogle Scholar
  32. 32.
    Nomiya H, Uehara K (2009) Content-based image classification via visual learning. Data mining and knowledge discovery in real life applications. InTech, pp 141–166Google Scholar
  33. 33.
    Obdrzálek S, Matas J (2002) Object recognition using local affine frames on distinguished regions. In: Rosin PL, Marshall AD (eds) Proceedings of the British machine vision conference 2002. British Machine Vision Association.
  34. 34.
    Ojala T, Pietikainen M, Harwood D (1996) A comparative study of texture measures with classification based on featured distribution. Pattern Recogn 29(1):51–59CrossRefGoogle Scholar
  35. 35.
    Pham TT, Mulhem P, Maisonnasse L, Gaussier E, Lim JH (2010) Visual graph modeling for scene recognition and mobile robot localization. Multimedia Tools and Applications 1–23. doi: 10.1007/s11042-010-0598-8
  36. 36.
    Punitha P, Guru DS (2006) An effective and efficient exact match retrieval scheme for symbolic image database systems based on spatial reasoning: a logarithmic search time approach. IEEE Trans Knowl Data Eng 18(10):1368–1381. doi: 10.1109/TKDE.2006.154 CrossRefGoogle Scholar
  37. 37.
    Rao CS, Kumar SS, Mohan BC (2010) Content based image retrieval using exact legendre moments and support vector machine. Int J Multimed Appl 2(2):69–79CrossRefGoogle Scholar
  38. 38.
    Skiadopoulos S, Koubarakis M (2004) Composing cardinal direction relations. Artif Intell 152(2):143–171. doi: 10.1016/S0004-3702(03)00137-1 MathSciNetMATHCrossRefGoogle Scholar
  39. 39.
    Sokal RR, Michener C (1958) A statistical method for evaluating systematic relationships. Univ Kans Sci Bull 38:1409–1438Google Scholar
  40. 40.
    Song YZ, Arbelaez P, Hall P, Li C, Balikai A (2010) Finding semantic structures in image hierarchies using laplacian graph energy. In: Proceedings of the 11th European conference on Computer vision: part IV, ECCV’10. Springer-Verlag, Berlin, Heidelberg, pp 694–707. URL: Google Scholar
  41. 41.
    Takala V, Ahonen T, Pietikäinen M (2005) Block-based methods for image retrieval using local binary patterns. In: SCIA, Lecture notes in computer science, vol 3540, pp 882–891. doi: 10.1007/11499145_89
  42. 42.
    Thies C, Malik A, Keysers D, Kohnen M, Fischer B, Lehmann TM (2003) Hierarchical feature clustering for content-based retrieval in medical image databases. In: Proc. medical imaging, Proc. SPIE. San Diego, CA, pp 598–608Google Scholar
  43. 43.
    Tsapatsoulis N, Petridis S (2007) Classifying images from athletics based on spatial relations. In: Proceedings of the second international workshop on semantic media adaptation and personalization. IEEE Computer Society, Washington, pp 92–97. doi: 10.1109/SMAP.2007.14
  44. 44.
    Vieux R, Benois-Pineau J, Domenger JP, Braquelaire A (2010) Segmentation-based multi-class semantic object detection. Multimedia Tools and Applications 1–22. doi: 10.1007/s11042-010-0611-2
  45. 45.
    Wang Y, Gong S (2006) Tensor discriminant analysis for view-based object recognition. In: Proceedings of the 18th international conference on pattern recognition, vol 03, ICPR ’06, pp 33–36Google Scholar
  46. 46.
    Zhang B, Srihari SN (2003) Binary vector dissimilarity measures for handwriting identification. In: DRR, SPIE Proceedings, vol 5010, pp 28–38Google Scholar
  47. 47.
    Zhang J, Marszalek M, Lazebnik S, Schmid C (2007) Local features and kernels for classification of texture and object categories: a comprehensive study. Int J Comput Vis 73:213–238CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Annette Morales-González
    • 1
  • Edel B. García-Reyes
    • 1
  1. 1.Pattern Recognition DepartmentAdvanced Technologies Application Center (CENATAV)HavanaCuba

Personalised recommendations