Using Concept Recognition to Annotate a Video Collection

  • Anupama Mallik
  • Santanu Chaudhury
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5909)


In this paper, we propose a scheme based on an ontological framework, to recognize concepts in multimedia data, in order to provide effective content-based access to a closed, domain-specific multimedia collection. The ontology for the domain is constructed from high-level knowledge of the domain lying with the domain experts, and further fine-tuned and refined by learning from multimedia data annotated by them. MOWL, a multimedia extension to OWL, is used to encode the concept to media-feature associations in the ontology as well as the uncertainties linked with observation of the perceptual multimedia data. Media feature classifiers help recognize low-level concepts in the videos, but the novelty of our work lies in discovery of high-level concepts in video content using the power of ontological relations between the concepts. This framework is used to provide rich, conceptual annotations to the video database, which can further be used to create hyperlinks in the video collection, to provide an effective video browsing interface to the user.


Bayesian Network Multimedia Data Video Annotation Concept Recognition Ontology Learning 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Zha, Z.J., Mei, T., Hua, X.S., Qi, G.J., Wang, Z.: Refining video annotation by exploiting pairwise concurrent relation. In: MULTIMEDIA 2007: Proceedings of the 15th international conference on Multimedia, pp. 345–348. ACM, New York (2007)CrossRefGoogle Scholar
  2. 2.
    Xu, D., Chang, S.F.: Video event recognition using kernel methods with multilevel temporal alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1985–1997 (2008)Google Scholar
  3. 3.
    Mallik, A., Pasumarthi, P., Chaudhury, S.: Multimedia ontology learning for automatic annotation and video browsing. In: MIR 2008: Proceeding of the 1st ACM international conference on Multimedia information retrieval, pp. 387–394. ACM, New York (2008)CrossRefGoogle Scholar
  4. 4.
    Ghosh, H., Chaudhury, S., Kashyap, K., Maiti, B.: Ontology specification and integration for multimedia applications. Springer, Heidelberg (2006)Google Scholar
  5. 5.
    Hofmann, T.: Probabilistic latent semantic analysis. In: Proc. of Uncertainty in Artificial Intelligence, UAI 1999, pp. 289–296 (1999)Google Scholar
  6. 6.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 20, 91–110 (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Anupama Mallik
    • 1
  • Santanu Chaudhury
    • 1
  1. 1.Electrical Engineering DepartmentIIT Delhi 

Personalised recommendations