Skip to main content
SpringerLink
Log in

Flowchart Recognition in Patent Information Retrieval

  • Chapter
  • First Online:
Current Challenges in Patent Information Retrieval

Part of the book series: The Information Retrieval Series ((INRE,volume 37))

  • 1614 Accesses

Abstract

In this chapter, we will analyse the current technologies available that deal with graphical information in patent retrieval applications and, in particular, with the problem of recognising and understanding information carried by flowcharts. We will review some of the state-of-the-art techniques that have arisen from the graphics recognition community and their application in the intellectual property domain. We will present an overview of the different steps that compound a flowchart recognition system, looking also at the achievements and remaining challenges in such a domain.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adams S (2005) Electronic non-text material in patent applications – some questions for patent offices, applicants and searchers. World Patent Inf 27(2):99–103

    Article  Google Scholar 

  2. Bhatti N, Hanbury A (2013) Image search in patents: a review. Int J Doc Anal Recognit 16(4):309–329

    Article  Google Scholar 

  3. Blostein D (1996) General diagram-recognition methodologies. In: Graphics recognition methods and applications. Lecture notes in computer science, vol 1072. Springer, New York, pp 106–122

    Google Scholar 

  4. Bunke H (1982) Attributed programmed graph grammars and their application to schematic diagram interpretation. IEEE Trans Pattern Anal Mach Intell 4(6):574–582

    Article  MATH  Google Scholar 

  5. Bunke H, Shearer K (1998) A graph distance metric based on the maximal common subgraph. Pattern Recogn Lett 19(3–4):255–259

    Article  MATH  Google Scholar 

  6. Cheng YQ, Cao YL, Yang JY (1990) An automatic recognition system of assembly drawings. In: Workshop on machine vision applications, pp 211—214

    Google Scholar 

  7. Codina J, Pianta E, Vrochidis S, Papadopoulos S (2008) Integration of semantic, metadata and image search engines with a text search engine for patent retrieval. In: Proceedings of the workshop on semantic search at the fifth European semantic web conference, pp 14–28

    Google Scholar 

  8. Csurka G, Renders JM, Jacquet G (2011) XRCE’s participation at patent image classification and image-based patent retrieval tasks of the CLEF-IP 2011. In: CLEF 2011 evaluation labs and workshop, online working notes

    Google Scholar 

  9. Escalera S, Fornés A, Pujol O, Radeva P, Sánchez G, Lladós J (2009) Blurred shape model for binary and grey-level symbol recognition. Pattern Recogn Lett 30(15):1424–1433

    Article  Google Scholar 

  10. European IPR Helpdesk. How to search for patent information. Fact Sheet, Nov 2011

    Google Scholar 

  11. European IPR Helpdesk. Automatic patent analysis. Fact Sheet, Dec 2013

    Google Scholar 

  12. Fafalios P, Salampasis M, Tzitzikas Y (2013) Exploratory patent search with faceted search and configurable entity mining. In: Proceedings of the integrating IR technologies for professional search workshop

    Google Scholar 

  13. Filippov IV, Nicklaus MC (2009) Extracting chemical structure information: optical structure recognition application. In: Proceedings of the IAPR international workshop on graphics recognition

    Google Scholar 

  14. Fletcher LA, Kasturi R (1988) A robust algorithm for text string separation from mixed text/graphics images. IEEE Trans Pattern Anal Mach Intell 10(6):910–918

    Article  Google Scholar 

  15. Hanbury A, Bhatti N, Lupu M, Mörzinger R (2011) Patent image retrieval: a survey. In: Proceedings of the fourth workshop on patent information retrieval, pp 3–8

    Google Scholar 

  16. Hoang TV, Tabbone S (2010) Text extraction from graphical document images using sparse representation. In: Proceedings of the 9th IAPR international workshop on document analysis systems, pp 143–150

    Google Scholar 

  17. Huet B, Kern NJ, Guarascio G, Merialdo B (2001) Relational skeletons for retrieval in patent drawings. In: Proceedings of the international conference on image processing, pp 737–740

    Google Scholar 

  18. Kasturi R, Tombre K (1995) Graphics recognition: methods and applications. Lecture notes in computer science, vol 1072. Springer, New York

    Google Scholar 

  19. Kesidis A, Karatzas D (2014) Logo and trademark recognition. Handbook of document image processing and recognition. Springer, London, pp 591–646

    Google Scholar 

  20. Lemaitre A, Mouchère H, Camillerapp J, Coüasnon B (2013) Interest of syntactic knowledge for on-line flowchart recognition. In: Graphics recognition new trends and challenges. Springer, New York, pp 89–98

    Chapter  Google Scholar 

  21. Lew MS, Sebe N, Djeraba C, Jain R (2006) Content-based multimedia information retrieval: state of the art and challenges. ACM Trans Multimed Comput Commun Appl 2(1):1–19

    Article  Google Scholar 

  22. Lin X, Shimotsuji S, Minoh M, Sakai T (1985) Efficient diagram understanding with characteristic pattern detection. Comput Vision Graphics Image Process 30(1):84–106

    Article  Google Scholar 

  23. List J (2007) How drawings could enhance retrieval in mechanical and device patent searching. World Patent Inf 29(3):210–218

    Article  Google Scholar 

  24. Lladós J, Rusiñol M (2014) Graphics recognition techniques. Handbook of document image processing and recognition. Springer, London, pp 489–521

    Google Scholar 

  25. Lu Z (1998) Detection of text regions from digital engineering drawings. IEEE Trans Pattern Anal Mach Intell 20(4):431–439

    Article  Google Scholar 

  26. Lupu M, Schuster R, Mörzinger R, Piroi F, Schleser T, Hanbury A (2012) Patent images – a glass-encased tool: opening the case. In: Proceedings of the 12th international conference on knowledge management and knowledge technologies

    Google Scholar 

  27. Mahmoudi F, Shanbehzadeh J, Eftekhari-Moghadam AM, Soltanian-Zadeh H (2003) Image retrieval based on shape similarity by edge orientation autocorrelogram. Pattern Recogn 36(8):1725–1736

    Article  Google Scholar 

  28. Mörzinger R, Horti A, Thallinger G, Bhatti N, Hanbury A (2011) Classifying patent images. In: CLEF 2011 evaluation labs and workshop, online working notes

    Google Scholar 

  29. Mörzinger R, Schuster R, Horti A, Thallinger G (2012) Visual structure analysis of flow charts in patent images. In: CLEF 2012 evaluation labs and workshop, online working notes

    Google Scholar 

  30. Piroi F, Lupu M, Hanbury A, Zenz V (2011) CLEF-IP 2011: retrieval in the intellectual property domain. In: CLEF 2011 evaluation labs and workshop, online working notes

    Google Scholar 

  31. Piroi F, Lupu M, Hanbury A, Sexton AP, Magdy W, Filippov IV (2012) CLEF-IP 2012: retrieval experiments in the intellectual property domain. In: CLEF 2012 evaluation labs and workshop, online working notes

    Google Scholar 

  32. Rusiñol M, Lladós J (2010) Efficient logo retrieval through hashing shape context descriptors. In: Proceedings of the ninth IAPR international workshop on document analysis systems, pp 215–222

    MATH  Google Scholar 

  33. Rusiñol M, Lladós J (2010) Symbol spotting in digital libraries: focused retrieval over graphic-rich document collections. Springer, London

    Book  MATH  Google Scholar 

  34. Rusiñol M, de las Heras LP, Mas J, Terrades OR, Karatzas D, Dutta A, Sánchez G, Lladós J (2012) CVC-UAB’s participation in the flowchart recognition task of CLEF-IP 2012. In: CLEF 2012 evaluation labs and workshop, online working notes

    Google Scholar 

  35. Rusiñol M, de las Heras LP, Ramos O (2014) Flowchart recognition for non-textual information retrieval in patent search. Inf Retr 17(4):331–341

    Google Scholar 

  36. Sadawi N (2009) Recognising chemical formulas from molecule depictions. In: Proceedings of the IAPR international workshop on graphics recognition

    Google Scholar 

  37. Samet H, Webber RE (1985) Storing a collection of polygons using quadtrees. ACM Trans Graph 4(3):182–222

    Article  Google Scholar 

  38. Sánchez G, Lladós J (2004) Syntactic models to represent perceptually regular repetitive patterns in graphic documents. In: Graphics recognition. Recent advances and perspectives. Springer, New York, pp 166–175

    Google Scholar 

  39. Sayre KM (1973) Machine recognition of handwritten words: a project report. Pattern Recogn 5(3):213–228

    Article  Google Scholar 

  40. Sidiropoulos P, Vrochidis S, Kompatsiaris I (2011) Content-based binary image retrieval using the adaptive hierarchical density histogram. Pattern Recogn 44(4):739–750

    Article  Google Scholar 

  41. Szwoch W (2007) Recognition, understanding and aestheticization of freehand drawing flowcharts. In: Proceedings of the ninth international conference on document analysis and recognition, pp 1138–1142

    Google Scholar 

  42. Thean A, Deltorn JM, Lopez P, Romary L (2012) Textual summarisation of flowcharts in patent drawings for CLEF-IP2012. In: CLEF 2012 evaluation labs and workshop, online working notes

    Google Scholar 

  43. Tiwari A, Bansal V (2004) PATSEEK: content based image retrieval system for patent database. In: Proceedings of the fourth international conference on electronic business, pp 1167–1171

    Google Scholar 

  44. Tombre K, Tabbone S, Pelissier L, Lamiroy B, Dosch P (2002) Text/graphics separation revisited. In: Document analysis systems V. Lecture notes in computer science, vol 2423. Springer, New York, pp 615–620

    Google Scholar 

  45. Vasudevan BG, Dhanapanichkul S, Balakrishnan R (2008) Flowchart knowledge extraction on image processing. In: Proceedings of the IEEE international joint conference on neural networks, pp 4075–4082

    Google Scholar 

  46. Vrochidis S, Papadopoulos S, Moumtzidou A, Sidiropoulos P, Pianta E, Kompatsiaris I (2010) Towards content-based patent image retrieval: a framework perspective. World Patent Inf 32(2):94–106

    Article  Google Scholar 

  47. Vrochidis S, Moumtzidou A, Kompatsiaris I (2012) Concept-based patent image retrieval. World Patent Inf 34(4):292–303

    Article  Google Scholar 

  48. Wahl F, Wong K, Casey R (1982) Block segmentation and text extraction in mixed text/image documents. Comput Graphics Image Process 20(4):375–390

    Article  Google Scholar 

  49. Wallis WD, Shoubridge P, Kraetz M, Ray D (2001) Graph distances using graph union. Pattern Recogn Lett 22(6–7):701–704

    Article  MATH  Google Scholar 

  50. Yu Y, Samal A, Seth SC (1997) A system for recognizing a large class of engineering drawings. IEEE Trans Pattern Anal Mach Intell 19(8):868–890

    Article  Google Scholar 

  51. Yuan Z, Pan H, Zhang L (2008) A novel pen-based flowchart recognition system for programming teaching. In: Advances in blended learning. Lecture notes in computer science, vol 5328. Springer, Berlin, pp 55–64

    Google Scholar 

  52. Zesheng S, Jing Y, Chunhong J, Yonggui W (1994) Symbol recognition in electronic diagrams using decision tree. In: Proceedings of the IEEE international conference on industrial technology, pp 719–723

    Google Scholar 

  53. Zhang D, Lu G (2002) A comparative study of three region shape descriptors. In: Proceedings of the digital image computing techniques and applications, pp 1–6

    Google Scholar 

  54. Zhang D, Lu G (2004) Review of shape representation and description techniques. Pattern Recogn 37:1–19

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dept. Ciències de la Computació, Computer Vision Center, Edifici O, UAB, 08193, Bellaterra, Spain

    Marçal Rusiñol & Josep Lladós

Authors
  1. Marçal Rusiñol
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Josep Lladós
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marçal Rusiñol .

Editor information

Editors and Affiliations

  1. Institute for Software Engineering & Interactive Systems, Vienna University of Technology, Vienna, Austria

    Mihai Lupu

  2. Research Platform Responsible Research and Innovation in Academic Practice, University of Vienna, Vienna, Austria

    Katja Mayer

  3. Information & Society Research Division, National Institute of Informatics, Tokyo, Japan

    Noriko Kando

  4. Patinformatics, LLC , Dublin, Ohio, USA

    Anthony J. Trippe

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer-Verlag GmbH Germany

About this chapter

Cite this chapter

Rusiñol, M., Lladós, J. (2017). Flowchart Recognition in Patent Information Retrieval. In: Lupu, M., Mayer, K., Kando, N., Trippe, A. (eds) Current Challenges in Patent Information Retrieval. The Information Retrieval Series, vol 37. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-53817-3_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-53817-3_13

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-53816-6

  • Online ISBN: 978-3-662-53817-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation