Info-Graphics Retrieval: A Multi-kernel Distance Based Hashing Scheme

  • Ritu GargEmail author
  • Santanu Chaudhury
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10481)


Information retrieval research has shown significant improvement and provided techniques that retrieve documents in image or text form. However, retrieval of multi-modal documents has been given very less attention. We aim to build a system for retrieval of documents with embedded information graphics (Info-graphics). Info-graphics are images of bar charts and line graphs appearing with textual components in magazines, newspapers, and journals. In this paper, we present multi-modal document image retrieval framework by learning an optimal fusion of information from text and info-graphics regions. The evaluation of the proposed concept is demonstrated on documents collected from various sources such as magazines and journals.


  1. 1.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005), Washington, D.C., vol. 1, pp. 886–893. IEEE Computer Society (2005)Google Scholar
  2. 2.
    Datta, R., Joshi, D., Li, J., Wang, J.Z.: Image retrieval: ideas, influences, and trends of the new age. ACM Comput. Surv. 40(2), 5:1–5:60 (2008)CrossRefGoogle Scholar
  3. 3.
    Demir, S., Carberry, S., McCoy, K.F.: Generating textual summaries of bar charts. In: Proceedings of the Fifth International Natural Language Generation Conference, pp. 7–15 (2008)Google Scholar
  4. 4.
    Elzer, S., Carberry, S., Zukerman, I.: The automated understanding of simple bar charts. Artif. Intell. 175(2), 526–555 (2011)CrossRefMathSciNetGoogle Scholar
  5. 5.
    Garg, R., Hassan, E., Chaudhury, S.: Document indexing framework for retrieval of degraded document images. In: 13th International Conference on Document Analysis and Recognition, ICDAR 2015, Nancy, France, 23–26 August 2015, pp. 1261–1265 (2015)Google Scholar
  6. 6.
    Gaur, V., Hassan, E., Chaudhury, S.: Design of multi-kernel distance based hashing with multiple objectives for image indexing. In: ICPR, pp. 2637–2642 (2014)Google Scholar
  7. 7.
    Hassan, E., Chaudhury, S., Gopal, M.: Feature combination in kernel space for distance based image hashing. IEEE Trans. Multimedia 14(4), 1179–1195 (2012)CrossRefGoogle Scholar
  8. 8.
    Hassan, E., Chaudhury, S., Gopal, M.: Multi-modal information integration for document retrieval. In: ICDAR, pp. 1200–1204 (2013)Google Scholar
  9. 9.
    Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst. 20(4), 422–446 (2002)CrossRefGoogle Scholar
  10. 10.
    Lapata, M.: Image and natural language processing for multimedia information retrieval. In: Proceedings of the 32nd European Conference on Advances in Information Retrieval, p. 12 (2010)Google Scholar
  11. 11.
    Lew, M.S., Sebe, N., Djeraba, C., Jain, R.: Content-based multimedia information retrieval: state of the art and challenges. ACM Trans. Multimedia Comput. Commun. Appl. 2(1), 1–19 (2006)CrossRefGoogle Scholar
  12. 12.
    Li, Z., Carberry, S., Fang, H., McCoy, K.F., Peterson, K.: Infographics retrieval: a new methodology. In: Métais, E., Roche, M., Teisseire, M. (eds.) Natural Language Processing and Information Systems. LNCS, vol. 8455, pp. 101–113. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-07983-7_15 Google Scholar
  13. 13.
    Li, Z., Carberry, S., Fang, H., McCoy, K.F., Peterson, K., Stagitis, M.: A novel methodology for retrieving infographics utilizing structure and message content. Data Knowl. Eng. 100(PB), 191–210 (2015)CrossRefGoogle Scholar
  14. 14.
    Li, Z., Stagitis, M., Carberry, S., McCoy, K.F.: Towards retrieving relevant information graphics. In: Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2013, pp. 789–792 (2013)Google Scholar
  15. 15.
    Lowe, D.G.: Object recognition from local scale-invariant features. In: Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 2, pp. 1150–1157 (1999)Google Scholar
  16. 16.
    Moreno, P.J., Ho, P.P., Vasconcelos, N.: A Kullback-Leibler divergence based kernel for SVM classification in multimedia applications. In: Advances in Neural Information Processing Systems, pp. 1385–1392 (2004)Google Scholar
  17. 17.
    Prasad, V.S.N., Siddiquie, B., Golbeck, J., Davis, L.S.: Classifying computer generated charts. In: 2007 International Workshop on Content-Based Multimedia Indexing, pp. 85–92 (2007)Google Scholar
  18. 18.
    Saleh, B., Dontcheva, M., Hertzmann, A., Liu, Z.: Learning style similarity for searching infographics. In: Proceedings of the 41st Graphics Interface Conference, pp. 59–64. Canadian Information Processing Society (2015)Google Scholar
  19. 19.
    Savva, M., Kong, N., Chhajta, A., Fei-Fei, L., Agrawala, M., Heer, J.: ReVision: automated classification, analysis and redesign of chart images. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, pp. 393–402 (2011)Google Scholar
  20. 20.
    Smith, R.: An overview of the tesseract OCR engine. In: Proceedings of the Ninth International Conference on Document Analysis and Recognition, Washington, D.C., vol. 02, pp. 629–633. IEEE Computer Society (2007)Google Scholar
  21. 21.
    Wei, X., Croft, W.B.: LDA-based document models for ad-hoc retrieval. In: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 178–185 (2006)Google Scholar
  22. 22.
    Wu, P.: Recognizing the intended message of line graphs: methodology and applications. Ph.D. thesis, Newark, DE, USA (2012)Google Scholar
  23. 23.
    Wu, P., Carberry, R.: Toward extractive summarization of multimodal documents. In: Proceedings of the Canadian AI Workshop on Text Summarization, pp. 53–64 (2011)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Department of Electrical EngineeringIndian Institute of Technology DelhiDelhiIndia

Personalised recommendations