Skip to main content
SpringerLink
Log in

Efficient Multimedia Information Retrieval with Query Level Fusion

  • Conference paper
  • First Online:
Flexible Query Answering Systems 2015

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 400))

  • 569 Accesses

Abstract

Multimedia data particularly digital videos that contain various modalities (visual, audio, and text) are complex and time consuming to deal with. Therefore, managing a large volume of multimedia data reveals the necessity for efficient methods for modeling, processing, storing and retrieving such data. In this study, we investigate how to efficiently manage multimedia data, especially video data. In addition, we discuss various flexible query types including the combination of content as well as concept-based queries that provide users with the ability to perform multimodal query. Furthermore, we introduce a fusion-based approach at the query level to improve query retrieval performance of the multimedia database. Our experimental tests show a significant improvement in the query retrieval performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Yilmaz, T., Yildirim, Y., Yazici, A.: A genetic algorithms based classifier for object classification in images. In: ISCIS, London, pp. 519–525 (2011)

    Google Scholar 

  2. Demir, U., Koyuncu, M., Yazici, A., Yilmaz, T., Sert, M.: Flexible content extraction and querying for videos. In: Christiansen, H., De Tré, G., Yazici, A., Zadrozny, S., Andreasen, T., Larsen, H.L. (eds.) FQAS 2011. LNCS, vol. 7022, pp. 460–471. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  3. Jonker, W., Petkovic, M.: An overview of data models and query languages for content-based video retrieval. In: Advances in Infrastructure for E-Business, Science, and Education on the Internet, Italy (2000)

    Google Scholar 

  4. Datta, R., Joshi, D., Li, J., Wang, J.: Image retrieval: Ideas, influences, and trends of the new age. In: ACM Computing Surveys (CSUR), pp. 1–60. ACM (2008)

    Google Scholar 

  5. Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. In: JMLR, vol. 3, pp. 993–1022 (2003)

    Google Scholar 

  6. Csurka, G., Dance, C., Fan, L., Willamowski, J., Bray, C.: Visual categorization with bags of keypoints. In: Workshop on Statistical Learning in Computer Vision, ECCV, vol. 1, p. 22 (2004)

    Google Scholar 

  7. Deerwester, S., Dumais, S., Furnas, G., Landauer, T., Harshman, R.: Indexing by latent semantic analysis. Journal of the American society for Information Science 41(6), 391–404 (1990)

    Article  Google Scholar 

  8. Wei, X.Y., Jiang, Y.G., Ngo, H.: Concept-Driven Multi-Modality Fusion for Video Search. IEEE Transactions on Circuits and Systems for Video Technology 21(1), 62–73 (2011)

    Article  Google Scholar 

  9. Natsev, A., Haubold, A., Tesic, J., Xie, L., Yan., R.: Semantic concept-based query expansion and re-ranking for multimedia retrieval. In: MULTIMEDIA 2007, pp. 991–1000. ACM, NY (2007)

    Google Scholar 

  10. Dunckley, L.: Multimedia Databases – An Object-Relational Approach (2003). ISBN # 0 201 78899 3

    Google Scholar 

  11. Bhatti, M.A., Rashid, U.: Exploration and management of web based multimedia information resources. In: International Conference on Systems, Computing Sciences and Software Engineering. Bridgeport, USA (2007)

    Google Scholar 

  12. Rolleke, T., Tsikrika, T., Kazai, G.: A general matrix framework for modeling information retrieval. In: Proceedings of the ACM SIGIR MF/IR (2003)

    Google Scholar 

  13. Sanchez, J.A., Arias, J.A.: Content-based search and annotations in multimedia digital libraries. In: ENC 2003, Fourth Mexican International Conference on Computer Science, pp. 109–117 (2003)

    Google Scholar 

  14. Kerne, A., Koh, E., Dworaczyk, B., Mistrot, J.M., Choi, H., Smith, S.M., Graeber, R., Caruso, D.: CombinFormation: a mixed-initiative system for representing collections as compositions of image and text. In: JCDL 2005, ACM/IEEECS Joint Conference on Digital Libraries, pp. 11–20 (2006)

    Google Scholar 

  15. Witten, I.H., Bainbridge, D.: Building digital library collections with greenstone. In: JCDL 2005, 5th ACM/IEEECS Joint Conference on Digital Libraries, pp. 425–425 (2005)

    Google Scholar 

  16. Wilson, M.L.: Advanced search interfaces considering information retrieval and human computer interaction. In: Agents and Multimedia, Southampton (2007)

    Google Scholar 

  17. Schraefel, M.C., Wilson, M., Russel, A., Smith, D.A.: MSPACE: improving information access to multimedia domains with multimodal exploratory search. Communication of the ACM 49(4), 47–49 (2006). ACM

    Article  Google Scholar 

  18. Dunn, W.: EVIADA: ethnomusicological video for instruction and analysis digital archive. In: JCDL 2005, 5th ACM/IEEE-CS Joint Conference on Digital Libraries, p. 407 (2005)

    Google Scholar 

  19. Rashid, U., Niaz, I.A., Bhatti, M.A.: Unified multimodal search framework for multimedia information retrieval. In: 4th International Conference on Systems, Computing Sciences and Software Engineering. Springer, Bridgeport (2007)

    Google Scholar 

  20. Yan, R.: Probabilistic Models for Combining Diverse Knowledge Sources in Multimedia Retrieval. Ph.D. Thesis. School of Computer Science, Carnegie Mellon University, USA (2006)

    Google Scholar 

  21. Manmatha, R.: Multimedia indexing and retrieval. In: Workshop on Challenges in Information Retrieval and Language Modeling (2002)

    Google Scholar 

  22. Westerveld, T., Ianeva, T., Boldareva, L., de Vries A.P., Hiemstra, D.: Combining infomation sources for video retrieval. In: NIST TRECVID (2003)

    Google Scholar 

  23. Amir, A., Hsu, W., Iyengar, G., Lin, C.Y., Naphade, M., Natsev, A., Neti, C., Nock, H.J., Smith, J. R., Tseng, B.L., Wu, Y., Zhang, D.: IBM research TRECVID- video retrieval system. In: NIST TRECVID (2003)

    Google Scholar 

  24. Rautiainen, M., Hosio, M., Hanski, I., Varanka, M., Kortelainen, J., Ojala, T., Seppanen, T.: TRECVID 2004 experiments at mediateam oulu. In: Proc. of TRECVID (2004)

    Google Scholar 

  25. Yu, J., Cong, Y., Qin, Z., Wan, T.: Cross-modal topic correlations for multimedia retrieval. In: ICPR, Tsukuba, Japan (2012)

    Google Scholar 

  26. Song, Y., Philippe Morency, L., Davis, R.: Multimodal human behavior analysis: learning correlation and interaction across modalities. In: ICMI, Santa Monica, California, USA, pp. 22–26 (2012)

    Google Scholar 

  27. Jiang, W., Loui, A.C.: Video concept detection by audio-visual grouplets. Multimedia Information Retrieval 1, 223–238 (2012)

    Article  Google Scholar 

  28. Zeng, J.D., Zheng, H.J., Lu, C., Li, T., Ma, W.: ReCoM: reinforcement clustering of multi-type interrelated data objects. In: ACM SIGIR, pp. 274–281. ACM, Canada (2003)

    Google Scholar 

  29. Wang, X.J., Ma, W.Y., Xue, G.R., Xing, L.: Multi-model similarity propagation and its application for web image retrieval. In: ACM Multimedia, pp. 944–951. ACM (2004)

    Google Scholar 

  30. Zhang, H., Zhuang, Y.T., Wu, F.: Cross-modal correlation learning for clustering on image-audio dataset. In: ACM Multimedia, pp. 273–276. ACM (2007)

    Google Scholar 

  31. Yang, Y., Zuang, Y.T., Wu, F., Pan, Y.H.: Harmonizing hierarchical manifolds for multimedia document semantics understanding and cross-media retrieval. IEEE Transactions, 437–446. (2008)

    Google Scholar 

  32. Rasiwasia, N., Pereira, J.C., Coviello, E., Doyle, G., Lanckriet, Gert, R.G., Levy, R., Vasconcelos, N.: A new approach to cross-modal multimedia retrieval. In: Multimedia 2010. ACM, Firenze (2010)

    Google Scholar 

  33. Chuang, C.T., Yang, K.H., Lin, Y.L., Wang, J.H.: Combining query terms extension and weight correlative for expert finding. In: International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT). IEEE (2014)

    Google Scholar 

  34. Safadi, B., Sahuguet, M., Benoi, H.: When textual and visual information join forces for multimedia retrieval. In: ICMR 2014. ACM, Glasgow (2014)

    Google Scholar 

  35. Aydinlilar, M., Yazici, A.: Semi-automatic semantic video annotation tool. In: ISCIS 2012, Paris, pp. 303–310 (2012)

    Google Scholar 

  36. Okuyucu, C., Sert, M., Yazici, A.: Environmental sound classification using spectral. In: IEEE, Turkey (2013)

    Google Scholar 

  37. Kucuk, D., Yazici, A.: Exploiting information extraction techniques for automatic semantic video indexing with an application to Turkish news videos. Knowledge-Based Systems 25(6), 844–857 (2011)

    Article  Google Scholar 

  38. Arslan, S., Yazici, A., Sacan, A., Toroslu, I.H., Acar, E.: Comparison of feature-based and image registration-based retrieval of image data using multidimensional data access methods. Data & Knowledge Engineering 86, 124–145 (2013). Elsevier

    Article  Google Scholar 

  39. Hardoon, D.R., Szedmak, S., Taylor, J.S.: Canonical correlation analysis; An overview with application to learning methods. Technical Report CSD-TR-03-02. University of London (2003)

    Google Scholar 

  40. Kuss, M., Graepel, T.: The Geometry Of Kernel Canonical Correlation Analysis. Technical Report No. 108. Max Planck Institute (2003)

    Google Scholar 

  41. News Channel of Turkey. http://www.ntvmsnbc.com/

  42. Naphade, M., Smith, J.R., Tesic, J., Chang, S.-F., Hsu, W., Kennedy, L., Hauptmann, A., Curtis, J.: Large-scale concept ontology for multimedia. IEEE MultiMedia 13(3), 86–91 (2006). IEEE

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Middle East Technical University, 06531, Ankara, Turkey

    Saeid Sattari & Adnan Yazici

Authors
  1. Saeid Sattari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adnan Yazici
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Saeid Sattari .

Editor information

Editors and Affiliations

  1. Dept. of Comm., Business & Info. Tech., Roskilde University, Roskilde, Denmark

    Troels Andreasen

  2. Dept. Computer Science, Roskilde University, Roskilde, Denmark

    Henning Christiansen

  3. Sys Res Intit of Polish Acad of Science, Intelligent Systems Laboratory, Warsaw, Poland

    Janusz Kacprzyk

  4. Department of Electronic Systems, Aalborg University, Esbjerg, Denmark

    Henrik Larsen

  5. Computer Science Department, University Milano-Biccoca, Milano, Italy

    Gabriella Pasi

  6. IRISA, University of Rennes I, Rennes, France

    Olivier Pivert

  7. Dept. of Tele. & Info. Processing, Ghent University, Gent, Belgium

    Guy De Tré

  8. Dept. of Comp. & Art. Int. Sci., University of Granada, Granada, Spain

    Maria Amparo Vila

  9. Department of Computer Engineering, Middle East Technical University, Ankara, Turkey

    Adnan Yazici

  10. Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland

    Sławomir Zadrożny

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Sattari, S., Yazici, A. (2016). Efficient Multimedia Information Retrieval with Query Level Fusion. In: Andreasen, T., et al. Flexible Query Answering Systems 2015. Advances in Intelligent Systems and Computing, vol 400. Springer, Cham. https://doi.org/10.1007/978-3-319-26154-6_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-26154-6_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-26153-9

  • Online ISBN: 978-3-319-26154-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation