Skip to main content
SpringerLink
Log in

A Visual Multimedia Query Language for Temporal Analysis of Video Data

  • Chapter
Multimedia Database Systems

Abstract

The storage of various media in multimedia databases poses new challenges to query techniques — challenges that exceed the expressive power of traditional text-based query languages. New query interfaces should take advantage of characteristics inherent in multimedia data, such as the dynamic temporal nature of video, the visual and spatial characteristics of images, the pitch of audio, etc. The focus of our research is to exploit the temporal continuity and combined spatio-temporal characteristics of video data for the purpose of video analysis. We do so by integrating a visual query paradigm with a dynamic visual presentation of results into a user-friendly interactive visualization environment. In this chapter, we present our overall approach for identifying trends in video data via querying for relationships between video annotations. Our approach allows users to analyze the video in terms of temporal relationships between events (e.g., events of type A frequently follow events of type B). We present a temporal visual query language (TVQL) for specifying relative temporal queries between sets of annotations. This query language builds on the notion of dynamic query filters and significantly extends them. It is tailored for temporal analysis — allowing users to pose queries, as well as to browse the data in a temporally continuous manner, thereby aiding them in the discovery of temporal trends. The TVQL is augmented with complementary temporal diagrams, which provide intuitive visual feedback for quickly and qualitatively verifying the temporal query specified. This chapter includes the complete specification of our TVQL, a transformation function for generating corresponding temporal diagrams, and the process for mapping TVQL queries to system queries.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 16.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Allen, J.F. (1983). Maintaining knowledge about temporal intervals. Communications of the ACM, 26(11), 832–843.

    Article  MATH  Google Scholar 

  2. Ahlberg, C., Williamson, C., & Shneiderman, B. (1992). Dynamic Queries for Information Exploration: An Implementation and Evaluation. CHI’92 Conference Proceedings, 619–626: ACM Press.

    Google Scholar 

  3. Ahlberg, C., & Shneiderman, B. (1994). Visual Information Seeking: Tight Coupling of Dynamic Query Filters with Starfleld Displays. CHF’94 Conference Proceedings, 313–317: ACM Press.

    Google Scholar 

  4. Chakravarthy, A.S. (1994). Toward Semantic Retrieval of Pictures and Video. AAAI’94 Workshop on Indexing and Reuse in Multimedia Systems, 12–18.

    Google Scholar 

  5. Chu, W.W., Ieong, I.T., Taira, R.K., & Breant, C.M. (1992). A Temporal Evolutionary Object-Oriented Data Model and Its Query Language for Medical Image Management. Proceedings of the 18th VLDB Conference, 53–64: Very Large Data Base Endowment.

    Google Scholar 

  6. Chua, T.-S., Lim, S.-K., & Pung, H.-K. (1994). Content-Based Retrieval of Segmented Images. ACM Multimedia’94 Proceedings: ACM Press.

    Google Scholar 

  7. Davis, M. (1994). Knowledge Representation for Video. Proceedings of the Twelfth National Conference on Artificial Intelligence, 120–127: AAAI Press.

    Google Scholar 

  8. Freksa, C. (1992). Temporal reasoning based on semi-intervals. Artificial Intelligence, 54 (1992), 199–227.

    Article  MathSciNet  Google Scholar 

  9. Fishkin, K. and Stone, M.C. (1995). Enhanced Dynamic Queries via Movable Filters. CHI’95 Conference Proceedings, 415–420. ACM Press.

    Google Scholar 

  10. Gevers, T. and Smeulders, A.W.M. (1992). Indexing of Images by Pictorial Information. Visual Database Systems, II (E. Knuth and L.M. Wegner, Eds.), North Holland: Amsterdam, 93–100.

    Google Scholar 

  11. Goldstein, J. & Roth, S. (1994). Using Aggregation and Dynamic Queries for Exploring Large Data Sets. CEI’ 94 Conference Proceedings, 23–29. ACM Press.

    Google Scholar 

  12. Hampapur, A., Weymouth, T., & Jain, R. (1994). Digital Video Segmentation. ACM Multimedia’94 Proceedings, 357–364: ACM Press.

    Google Scholar 

  13. Harrison, B.L., Owen, R., & Baecker, R.M. (1994). Timelines: An Interactive System for the Collection of Visualization of Temporal Data. Proceedings of Graphics Interface’ 94. Canadian Information Processing Society.

    Google Scholar 

  14. Hibino, S. & Rundensteiner, E. (1995a). A Visual Query Language for Identifying Temporal Trends in Video Data. To appear in Proceedings of the First International Workship on Multimedia Database Management Systems.

    Google Scholar 

  15. Hibino, S. & Rundensteiner, E. (1995b). Interactive Visualizations for Exploration and Spatio-Temporal Analysis of Video Data. To appear in IJCAI’95 Workshop on Intelligent Multimedia Information Retrieval.

    Google Scholar 

  16. Hibino, S. & Rundensteiner, E. (Dec 1994). A Graphical Query Language for Identifying Temporal Trends in Video Data. University of Michigan, EECS Technical Report CSE-TR-225-94.

    Google Scholar 

  17. Lenat, D. & Guha, R.V. (1994). Strongly Semantic Information Retrieval. AAAI’94 Workshop on Indexing and Reuse in Multimedia Systems, 58–68.

    Google Scholar 

  18. Little, T.D.C. & Ghafoor, A. (1993). Interval-Based Conceptual Models for Time-Dependent Multimedia Data. IEEE Transactions on Knowledge and Data Engineering, 5(4), 551–563.

    Article  Google Scholar 

  19. Mackay, W. E. (1989). EVA: An experimental video annotator for symbolic analysis of video data. SIGCHI Bulletin, 21(2), 68–71.

    Article  Google Scholar 

  20. Nagasaka, A. and Tanaka, A. (1992). Automatic Video Indexing and Full-Video Search for Object Appearances. Visual Database Systems, II (E. Knuth and L.M. Wegner, Eds.), 113–127. Elsevier Science Publishers.

    Google Scholar 

  21. Oomoto, E. & Tanaka, K. (1993). OVID: Design and Implementation of a Video-Object Database System. IEEE Transactions on Knowledge and Data Engineering, 5(4), 629–643.

    Article  Google Scholar 

  22. Roschelle, J., Pea, R., & Trigg, R. (1990). VIDEONOTER: A tool for exploratory analysis (Research Rep. No. IRL90-0021). Palo Alto, CA: Institute for Research on Learning.

    Google Scholar 

  23. Santucci, G. & Sottile, P.A. (1993). Query by Diagram: a Visual Environment for Querying Databases. Software — Practice and Experience, 23(3), 317–340.

    Article  Google Scholar 

  24. Snodgrass, R. (1987). The Temporal Query Language TQuel. ACM Transactions on Database Systems, 12(2), 247–298.

    Article  Google Scholar 

  25. Snodgrass, R. (1992). Temporal Databases. Theories and Methods of Spatio-Temporal Reasoning in Geographic Space (A.U. Frank, I. Campari, and U. Formentini, Eds.), Springer-Verlag: New York, 22–64.

    Google Scholar 

  26. Ueda, H., Miyatake, T., Sumino, S., & Nagasaka, A. (1993). Automatic Structure Visualization for Video Editing. InterCEI’93 Proceedings, (pp. 137–141): ACM Press.

    Google Scholar 

  27. Weber, K. & Poon, A. (1994). Marquee: A Tool for Real-Time Video Logging. CHI’94 Conference Proceedings, 58–64: ACM Press.

    Google Scholar 

  28. Whang, K., Malhotra, A., Sockut, G., Burns, L., & Coi, K-S. (1992). Two-Dimensional Specification of Universal Quantification in a Graphical Database Query Language, IEEE Transactions on Software Engineering, 18(3), 216–224.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electrical Engr. and Computer Sc. Dept. Software Systems Research Laboratory, University of Michigan, Ann Arbor, USA

    Stacie Hibino & Elke A. Rundensteiner

Authors
  1. Stacie Hibino
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elke A. Rundensteiner
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. AT&T Bell Laboratories, Whippany, New Jersey, USA

    Kingsley C. Nwosu

  2. The MITRE Corporation, Bedford, Massachusetts, USA

    Bhavani Thuraisingham

  3. CASE Center, Syracuse University, Syracuse, New York, USA

    P. Bruce Berra

Rights and permissions

Reprints and permissions

Copyright information

© 1996 Kluwer Academic Publishers

About this chapter

Cite this chapter

Hibino, S., Rundensteiner, E.A. (1996). A Visual Multimedia Query Language for Temporal Analysis of Video Data. In: Nwosu, K.C., Thuraisingham, B., Berra, P.B. (eds) Multimedia Database Systems. Springer, Boston, MA. https://doi.org/10.1007/978-1-4613-0463-0_5

Download citation

  • DOI: https://doi.org/10.1007/978-1-4613-0463-0_5

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4613-8060-3

  • Online ISBN: 978-1-4613-0463-0

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation