Abstract
Local features and descriptors that perform well in the case of photographic images are often unable to capture the content of binary technical drawings due to their different characteristics. Motivated by this, a new local feature representation, the contextual local primitives, is proposed in this paper. It is based on the detection of the junction and end points, classification of the local primitives to local primitive words and establishment of the geodesic connections of the local primitives. We exploit the granulometric information of the binary patent images to set all the necessary parameters of the involved mathematical morphology operators and window size for the local primitive extraction, which makes the whole framework parameter free. The contextual local primitives and, their spatial areas as a histogram weighting factor are evaluated by performing binary patent image retrieval experiments. It is found that the proposed contextual local primitives perform better than the local primitives only, the SIFT description of the contextual Hessian points, the SIFT description of local primitives and state of the art local content capturing methods. Moreover, an analysis of the approach in the perspective of a general patent image retrieval system reveals of its being efficient in multiple aspects.
Similar content being viewed by others
References
Attali D, Boissonnat JD, Edelsbrunner H (2009) Stability and computation of medial axes-a state-of-the-art report. In: Mathematical foundations of scientific visualization, computer graphics, and massive data exploration, pp 109–125
Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24(4):509–522
Bergevin R, Filiatrault A (2007) Enhancing contour primitives by pairwise grouping and relaxation. In: Proceedings of 4th international conference on image analysis and recognition (ICIAR), pp 222–233
Bhatti NA, Hanbury A (2011) Detection and classification of local primitives in line drawings. In: Proceedings of the AAPR workshop
Bhatti NA, Hanbury A (2011) Granulometry based detection of junction and end points in patent drawings. In: 2011 7th International symposium on image and signal processing and analysis (ISPA), pp 307–312
Bhatti N, Hanbury A (2011) Morphology based spatial relationships between local primitives in line drawings. In: CIARP, pp 165–172
Bhatti N, Hanbury A (2012) Image search in patents: a review. In: International journal on document analysis and recognition (IJDAR), pp 1–21
Castanedo F (2013) A review of data fusion techniques. Sci World J 2013:1–19
Choi MJ, Torralba A, Willsky AS (2012) A tree-based context model for object recognition. IEEE Trans Pattern Anal Mach Intell 34(2):240–252
Csurka G, Dance CR, Fan L, Willamowski J, Bray C (2004) Visual categorization with bags of keypoints. In: Workshop on statistical learning in computer vision (ECCV), pp 1–22
Csurka G, Renders J, Jacquet G (2011) XRCE’s participation at patent image classification and image-based patent retrieval tasks of the Clef-IP 2011. In: V Petras, P Forner, PD Clough (eds) CLEF (Notebook Papers/Labs/Workshop)
Das M, Manmatha R, Riseman EM (1999) Indexing flower patent images using domain knowledge. IEEE Intell Syst 14(5):24–33
Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv 40(2):1–60
Deseilligny MP, Stamon G, Ching YS (1998) Veinerization: a new shape description for flexible skeletonization. IEEE Trans Pattern Anal Mach Intell 20(5):505–521
Desolneux A, Moisan L, Morel JM (2004) Seeing, thinking and knowing. In: Carsetti A (ed). Kluwer Academic Publishers, Norwell
Fonseca MJ, Ferreira A, Jorge JA (2004) Content-based retrieval of technical drawings. In: Special issue of international journal of computer applications in technology (IJCAT)
Fonseca MJ, Ferreira A, Jorge JA (2009) Sketch-based retrieval of complex drawings using hierarchical topology and geometry. Comput Aided Des 41(12):1067–1081
Förstner W (1999) Uncertain neighborhood relations of point sets and fuzzy delaunay triangulation. In: Mustererkennung, 21. DAGM-Symposium, pp 213–222
Galleguillos C, Belongie S (2010) Context based object categorization: a critical survey. Comput Vis Image Underst 114(6):712–722
Hanbury A, Bhatti N, Lupu M, Morzinger R (2011) Patent image retrieval: a survey. In: Proceedings of the patent inforamtion retrieval workshop (PaIR). ACM, pp 3–8
Heikkila M, Pietikainen M, Schmid C (2009) Description of interest regions with local binary patterns. Pattern Recogn 42(3):425–436
Huet B, Guarascio G, Kern NJ, Mérialdo B. (2001) Relational skeletons for retrieval in patent drawings. In: ICIP, pp 737–740
Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of the 2006 IEEE computer society conference on computer vision and pattern recognition - volume 2, CVPR ’06, pp 2169–2178
Leung WH, Chen T (2002) User-independent retrieval of free-form hand-drawn sketches. In: ICASSP, pp 2029–2032
Li FF, Fergus R, Perona P (2007) Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. Comput Vis Image Underst 106(1):59–70
List J (2007) How drawings could enhance retrieval in mechanical and device patent searching. World Patent Inf 29(3):210–218
Liu R, Wang Y, Baba T, Masumoto D (2010) Shape detection from line drawings with local neighborhood structure. Pattern Recogn 43(5):1907–1916
Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis 60(2):91–110
Mahmoudi F, Shanbehzadeh J, Eftekhari-Moghadam A, Soltanian-Zadeh H (2003) Image retrieval based on shape similarity by edge orientation autocorrelogram. Pattern Recogn 1725–1736
Maire M, Arbelaez P, Fowlkes C, Malik J (2008) Using contours to detect and localize junctions in natural images. In: CVPR, pp 1–8
Mikolajczyk K (2002) Scale and affine invariant interest point detectors. PhD Thesis
Mikolajczyk K, Tuytelaars T, Schmid C, Zisserman A, Matas J, Schaffalitzky F, Kadir T, Gool LV (2005) A comparison of affine region detectors. Int J Comput Vis 65:43–47
Newby GB (1997) Context-based statistical sub-spaces. In: TREC, pp 735–745
Olson CF, Hoover SA, Soltman JL, Zhang S (2016) Complementary keypoint descriptors. Springer International Publishing, pp 341–352
Papari G, Petkov N (2011) Edge and line oriented contour detection: state of the art. Image Vis Comput 29(2-3):79–103
Park JH, Um BS (1999) A new approach to similarity retrieval of 2-D graphic objects based on dominant shapes. Pattern Recogn Lett 20(6):591–616
Parker C, Chen T (2003) Hierarchical matching for retrieval of hand-drawn sketches. In: ICME, pp 29–32
Heuel S, WF (1998) A dual, scalable and hierarchical representation for perceptual organization of binary images. In: Workshop on perceptual organization in computer vision. IEEE Computer Society
Santosh KC, Wendling L, Lamiroy B (2010) Unified pairwise spatial relations: an application to graphical symbol retrieval. In: Proceedings of the 8th international conference on graphics recognition: achievements, challenges, and evolution, GREC’09, pp 163–174
Shen J (2009) Stochastic modeling western paintings for effective classification. Pattern Recogn 42(2):293–301
Shen J, Deng RH, Cheng Z, Nie L, Yan S (2015) On robust image spam filtering via comprehensive visual modeling. Pattern Recogn 48(10):3227–3238
Shotton J, Johnson M, Cipolla R (2008) Semantic texton forests for image categorization and segmentation. In: CVPR. IEEE Computer Society
Sidiropoulos P, Vrochidis S, Kompatsiaris I (2010) Adaptive hierarchical density histogram for complex binary image retrieval. In: CBMI, pp 1–6
Smeulders AWM, Worring M, Santini S, Gupta A, Jain R (2000) Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–1380
Soille P (2003) Morphological image analysis: principles and applications, 2nd edn. Springer-Verlag New York, Inc., Secaucus
Tiwari A, Bansal V (2004) Patseek: content based image retrieval system for patent database. In: ICEB, pp 1167–1171
Tuytelaars T, Mikolajczyk K (2008) Local invariant feature detectors - a survey. Found Trends Comput Graph Vis
Vrochidis S, Papadopoulos S, Moumtzidou A, Sidiropoulos P, Pianta E, Kompatsiaris I (2010) Towards content-based patent image retrieval: a framework perspective. World Patent Inf 32(2):94–106
Wong SKM, Ziarko W, Wong PCN (1985) Generalized vector space model in information retrieval. In: SIGIR, pp 18–25
Xie L, Shen J, Zhu L (2016) Online cross-modal hashing for web image retrieval. In: Proceedings of the thirtieth AAAI conference on artificial intelligence, AAAI’16. AAAI Press, pp 294–300
Yang M, Qiu G, Huang J, Elliman D (2006) Near-duplicate image recognition and content-based image retrieval using adaptive hierarchical geometric centroids. In: ICPR, pp 958–961
Zhiyuan Z, Juan Z, Bin X (2007) An outward-appearance patent-image retrieval approach based on the contour-description matrix. In: FCST, pp 86–89
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bhatti, N., Hanbury, A. & Stottinger, J. Contextual local primitives for binary patent image retrieval. Multimed Tools Appl 77, 9111–9151 (2018). https://doi.org/10.1007/s11042-017-4808-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-017-4808-5