Encyclopedia of Database Systems

2018 Edition
| Editors: Ling Liu, M. Tamer Özsu

Semantic Modeling and Knowledge Representation for Multimedia Data

  • Edward Y. ChangEmail author
Reference work entry
DOI: https://doi.org/10.1007/978-1-4614-8265-9_1038


Image/Video/Music search; Multimedia information retrieval


Semantic modeling and knowledge representation is essential to a multimedia information retrieval system for supporting effective data organization and search. Semantic modeling and knowledge representation for multimedia data (e.g., imagery, video, and music) consists of three steps: feature extraction, semantic labeling, and features-to-semantics mapping. Feature extraction obtains perceptual characteristics such as color, shape, texture, salient-object, and motion features from multimedia data; semantic labeling associates multimedia data with cognitive concepts; and features-to-semantics mapping constructs correspondence between perceptual features and cognitive concepts. Analogically to data representation for text documents, improving semantic modeling and knowledge representation for multimedia data leads to enhanced data organization and query performance.

Historical Background

The principal design...

This is a preview of subscription content, log in to check access.

Recommended Reading

  1. 1.
    Aizerman MA, Braverman EM, Rozonoer LI. Theoretical foundations of the potential function method in pattern recognition learning. Autom Remote Control. 1964;25:821–37.zbMATHGoogle Scholar
  2. 2.
    Barnard K, Forsyth D. Learning the semantics of words and pictures. Int Conf Comput Vision. 2000;2:408–15.CrossRefGoogle Scholar
  3. 3.
    Beyer K, Goldstein J, Ramakrishnan R, Shaft U. When is nearest neighbor meaningful. In: Proceedings of the 7th International Conference on Database Theory; 1999. p. 217–35.Google Scholar
  4. 4.
    Blei DM, Ng A, Jordan M. Latent Dirichlet allocation. J Machine Learning Res. 2003;3(4/5):993–1022.Google Scholar
  5. 5.
    Chang EY, et al. Parallelizing support vector machines on distributed computers. In: Advances in Neural Information Proceedings of the Systems 20, Proceedings of the 21st Annual Conference on Neural Information Proceedings of the Systems; 2007.Google Scholar
  6. 6.
    Datta R, Joshi D, Li J, Wang JZ. Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv. 2008;40(65).CrossRefGoogle Scholar
  7. 7.
    Donoho DL. Aide-Memoire. High-dimensional data analysis: the curses and blessings of dimensionality (American Math. Society Lecture). In: Mathematical Challenges Of The 21st Century Explored at American Mathematical Society Conference; 2000.Google Scholar
  8. 8.
    Flickner M, et al. Query by image and video content: QBIC system. IEEE Comput. 1995;28(9).Google Scholar
  9. 9.
    Goh K, Chang EY, Lai W-C. Concept-dependent multimodal active learning for image retrieval. In: Proceedings of the 12th ACM International Conference on Multimedia; 2004. p. 564–71.Google Scholar
  10. 10.
    Hoi C-H, Lyu MR. A novel log-based relevance feedback technique in content-based image retrieval. In: Proceedings of the 12th ACM International Conference on Multimedia; 2004. p. 24–31.Google Scholar
  11. 11.
    Li B, Chang EY. Discovery of a perceptual distance function for measuring image similarity. ACM Multimedia Syst J. (Special Issue on Content-Based Image Retrieval). 2003;8(6):512–22.CrossRefGoogle Scholar
  12. 12.
    Rocchio JJ. Relevance feedback in information retrieval. In: Salton G, editor. The SMART retrieval system – experiments in automatic document processing. Englewood Cliffs: Prentice-Hall; 1971. p. 313–23. Chapter 14.Google Scholar
  13. 13.
    Rui Y, Huang TS, Chang S-F. Image retrieval: current techniques, promising directions and open issues. J Visual Commn Image Represent. 1999;10(1):39–62.CrossRefGoogle Scholar
  14. 14.
    Tong S, Chang EY. Support vector machine active learning for image retrieval. In: Proceedings of the 9th ACM International Conference on Multimedia; 2001. p. 107–18.Google Scholar
  15. 15.
    Tversky A. Features of similarity. Psychol Rev. 1997;84(4):327–52.CrossRefGoogle Scholar
  16. 16.
    Wu G, Chang EY. KBA: Kernel Boundary Alignment considering imbalanced data distribution. IEEE Trans Knowl Data Eng. 2005;17(6):786–95.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Google ResearchMountain ViewUSA

Section editors and affiliations

  • Vincent Oria
    • 1
  • Shin'ichi Satoh
    • 2
  1. 1.Dept. of Computer ScienceNew Jersey Inst. of TechnologyNewarkUSA
  2. 2.Digital Content and Media Sciences ReseaMultimedia Information Research DivisionNational Institute of InformaticsTokyoJapan