Skip to main content
SpringerLink
Log in
Book cover

Context-Enhanced Information Fusion pp 507–537Cite as

Context Understanding from Query-Based Streaming Video

  • Chapter
  • First Online:

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

Abstract

Context understanding is established from the content, analysis, and guidance from query-based coordination between users and machines. In this chapter, a live video computing (LVC) structure is presented for access of a database management of information for context assessment. Context assessment includes multimedia fusion of query-based text, images, and exploited tracks which can be utilized for content-based image retrieval (CBIR). In this chapter, we explore the developments in database systems to enable context to be utilized in user-based queries (e.g., Level 5 fusion) for information fusion content extraction. Using a common video dataset, we demonstrate time savings in the analysis from user queries to provide a context, privacy, and semantic-aware information fusion.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. E. Blasch, A. Steinberg, S. Das, J. Llinas, C.-Y. Chong, O. Kessler, E. Waltz, F. White, Revisiting the JDL model for information exploitation, in International Conference on Info Fusion (2013)

    Google Scholar 

  2. C.J. Date, An Introduction to Database Systems, 2nd edn. (Addison-Wesley Publishing Company Inc, 1977)

    Google Scholar 

  3. M.A. Tantaoui, K.A. Hua, T.T. Do, BroadCatch: a periodic broadcast technique for heterogeneous video-on-demand. Broadcast. IEEE Trans. 50(3), 289–301 (2004)

    Article  Google Scholar 

  4. Z. Liu, E. Blasch, Z. Xue, E. Langaniere, W. Wu, Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: a comparative survey. IEEE Trans. Pattern Analysis Mach. Intell. 34(1), 94–109 (2012)

    Article  Google Scholar 

  5. E. Blasch, S. Plano, Cognitive fusion analysis based on context, in Proceedings of SPIE, vol. 5434 (2004)

    Google Scholar 

  6. E.P. Blasch, E. Bosse, D.A. Lambert, High-Level Information Fusion Management and Systems Design (Artech House, Norwood, MA, 2012)

    Google Scholar 

  7. S. Ezekiel, M.G. Alford, D. Ferris, E. Jones et al., Multi-scale decomposition tool for content based image retrieval, in IEEE Applied Imagery Pattern Recognition Workshop (2013)

    Google Scholar 

  8. E. Blasch, I. Kadar, K. Hintz, J. Biermann, C. Chong, S. Das, Resource management coordination with level 2/3 fusion issues and challenges. IEEE Aerosp. Electron. Syst. Mag. 23(3), 32–46 (2008)

    Google Scholar 

  9. S. Hoberman, Data Modeling Made Simple: A Practical Guide for Business and Information Technology Professionals (Technics Publications, 2005)

    Google Scholar 

  10. J. Grimes, M. Potel, What is multimedia? Comput. Graph. Appl. IEEE 11(1), 49–52 (1991)

    Article  Google Scholar 

  11. Z. Zhang, R. Zhang, Multimedia Data Mining: A Systematic Introduction to Concepts and Theory (Chapman & Hall/CRC, 2008)

    Google Scholar 

  12. H. Ling, L. Bai, E. Blasch, X. Mei, Robust infrared vehicle tracking across target pose change using L1 regularization, in International Conference on Info Fusion (2010)

    Google Scholar 

  13. Z. Liu, E. Blasch, Z. Xue, R. Langaniere, W. Wu, Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: a comparative survey. IEEE Trans. Pattern Anal. Mach. Intell. 34(1), 94–109 (2012)

    Article  Google Scholar 

  14. M.K. Hu, Visual pattern recognition by moment invariants. Inf. Theory IRE Trans. 8, 179–187 (1962)

    MATH  Google Scholar 

  15. E. Bribiesca, A. Guzman, How to describe pure form and how to measure differences in shapes using shape numbers. Pattern Recogn. 12, 101–112 (1980)

    Article  Google Scholar 

  16. M. Nixon, A.S. Aguado, Feature Extraction and Image Processing for Computer Vision (Academic Press, 2012)

    Google Scholar 

  17. C. Harris, M. Stephens, A combined corner and edge detector, in Alvey Vision Conference, (Manchester, UK, 1988) p. 50

    Google Scholar 

  18. D.G. Lowe, Object recognition from local scale-invariant features. IEEE International Conference on Computer Vision (1999), pp. 1150–1157

    Google Scholar 

  19. Y. Wu, E. Blasch, G. Chen, L. Bai, H. Ling, Multiple source data fusion via sparse representation for robust visual tracking, in International Conference on Info Fusion (2011)

    Google Scholar 

  20. H. Ling, Y. Wu, E. Blasch, G. Chen, L. Bai, Evaluation of visual tracking in extremely low frame rate wide area motion imagery, in International Conference on Info Fusion

    Google Scholar 

  21. C.R. Wren, A. Azarbayejani, T. Darrell, A.P. Pentland, Pfinder: real-time tracking of the human body. IEEE Trans. Pattern Anal. Mach. Intell. 19, 780–785 (1997)

    Article  Google Scholar 

  22. C. Stauffer, W.E.L. Grimson, Adaptive background mixture models for real-time tracking. in IEEE Conferernce on Computer Vision and Pattern Recognition (1999)

    Google Scholar 

  23. K. Kim, T.H. Chalidabhongse, D. Harwood, L. Davis, Real-time foreground–background segmentation using codebook model. Real-Time Imaging 11, 172–185 (2005)

    Article  Google Scholar 

  24. M. Piccardi, Background subtraction techniques: a review, in Presented at the 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE, 2004), pp. 3099–3104

    Google Scholar 

  25. S.C.S. Cheung, C. Kamath, Robust techniques for background subtraction in urban traffic video, in Presented at the Proceedings of SPIE (2004) pp. 881–892

    Google Scholar 

  26. D.H. Parks, S.S. Fels, Evaluation of background subtraction algorithms with post-processing, in Presented at the IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance (IEEE, 2008), pp. 192–199

    Google Scholar 

  27. R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classification and Scene Analysis, 2nd edn. (1995)

    Google Scholar 

  28. H. Samet, The Design and Analysis of Spatial Data Structures (Addison-Wesley Reading MA, 1990)

    Google Scholar 

  29. H. Samet, Foundations of Multidimensional and Metric Data Structures (Morgan Kaufmann, 2006)

    Google Scholar 

  30. R.M. Bolle, B.L. Yeo, M. Yeung, Video query: research directions. IBM J. Res. Dev. 42, 233–252 (1998)

    Article  Google Scholar 

  31. R. Brunelli, O. Mich, C.M. Modena, A survey on the automatic indexing of video data. J. Vis. Commun. Image Represent. 10, 78–112 (1999)

    Article  Google Scholar 

  32. C.G.M. Snoek, M. Worring, Multimodal video indexing: a review of the state-of-the-art. Multimed. Tools Appl. 25, 5–35 (2005)

    Article  Google Scholar 

  33. Y. Wang, Z. Liu, J.C. Huang, Multimedia content analysis-using both audio and visual clues. Signal Process. Mag. IEEE 17, 12–36 (2000)

    Article  Google Scholar 

  34. J.M. Boggs, The Art of Watching Films (ERIC, 1996)

    Google Scholar 

  35. R. Jain, A. Hampapur, Metadata in video databases. ACM Sigmod Rec. 23, 27–33 (1994)

    Article  Google Scholar 

  36. R. Brunelli, O. Mich, C.M. Modena, A survey on the automatic indexing of video data. J. Vis. Commun. Image Represent. 10, 78–112 (1999)

    Article  Google Scholar 

  37. A.K. Jain, R.P.W. Duin, J. Mao, Statistical pattern recognition: a review. Pattern Anal. Mach. Intell. IEEE Trans. 22, 4–37 (2000)

    Article  Google Scholar 

  38. I. Ide, K. Yamamoto, H. Tanaka, Automatic video indexing based on shot classification. Adv. Multimed. Content Process. 87–102 (1999)

    Google Scholar 

  39. J. Foote, An overview of audio information retrieval. Multimed. Syst. 7, 2–10 (1999)

    Article  Google Scholar 

  40. E. Wold, T. Blum, D. Keislar, J. Wheaten, Content-based classification, search, and retrieval of audio. Multimed. IEEE 3, 27–36 (1996)

    Article  Google Scholar 

  41. J. Nievergelt, H. Hinterberger, K.C. Sevcik, The grid file: an adaptable, symmetric multikey file structure. ACM Trans. Database Syst. TODS 9, 38–71 (1984)

    Article  Google Scholar 

  42. A.V. Aho, J.E. Hopcroft, J. Ullman, Data Structures and Algorithms (Addison-Wesley Longman Publishing Co., Inc, 1983)

    Google Scholar 

  43. T.H. Cormen, C.E. Leiserson, R.L. Rivest, C. Stein, Introduction to Algorithms. MIT press

    Google Scholar 

  44. Pieprzyk, J., Sadeghiyan, B. Design of Hashing Algorithms (Springer, New York, 2001)

    Google Scholar 

  45. J.L. Bentley, Multidimensional binary search trees used for associative searching. Commun. ACM 18, 509–517 (1975)

    Article  MATH  Google Scholar 

  46. P. Scheuermann, M. Ouksel, Multidimensional B-trees for associative searching in database systems. Inf. Syst. 7, 123–137 (1982)

    Article  MATH  Google Scholar 

  47. W.G. Aref, I.F. Ilyas, Sp-gist: an extensible database index for supporting space partitioning trees. J. Intell. Inf. Syst. 17, 215–240 (2001)

    Article  MATH  Google Scholar 

  48. A. Guttman, R-trees: a dynamic index structure for spatial searching (ACM, 1984)

    Google Scholar 

  49. G. Hristescu, M. Farach-Colton, Cluster-preserving embedding of proteins. Technical Report 99-50, Computer Science Department, Rutgers University

    Google Scholar 

  50. J.T.L. Wang, X. Wang, K.I. Lin, D. Shasha, B.A. Shapiro, K. Zhang, Evaluating a class of distance-mapping algorithms for data mining and clustering, in Presented at the Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, 1999), pp. 307–311

    Google Scholar 

  51. W. Aref, H. Samet, Uniquely reporting spatial objects: yet another operation for comparing spatial data structures, in Presented at the Proceedings of the Fifth International Symposium on Spatial Data Handling (1992) pp. 178–189

    Google Scholar 

  52. W.G. Aref, H. Samet, Hashing by proximity to process duplicates in spatial databases, in Presented at the Proceedings of the Third International Conference on Information and Knowledge Management (ACM, 1994), pp. 347–354

    Google Scholar 

  53. H. Samet, Spatial data structures. Mod. Database Syst. Object Model Interoperability Beyond, 361–385 (1995)

    Google Scholar 

  54. M. Flickner, H. Sawhney, W. Niblack, J. Ashley, Q. Huang, B. Dom, M. Gorkani, J. Hafner, D. Lee, D. Petkovic, Query by image and video content: the QBIC system. Computer 28, 23–32 (1995)

    Article  Google Scholar 

  55. A. Blaser, Data base techniques for pictorial applications (Springer, Florence, 1979)

    Google Scholar 

  56. N.S. Chang, K.S. Fu, Query-by-pictorial-example. IEEE Trans. Softw. Eng. 519–524 (1980)

    Google Scholar 

  57. S.F. Chang, A. Eleftheriadis, R. McClintock, Next-generation content representation, creation, and searching for new-media applications in education. Proc. IEEE 86, 884–904 (1998)

    Article  Google Scholar 

  58. J.R. Bach, C. Fuller, A. Gupta, A. Hampapur, B. Horowitz, R. Humphrey, R. Jain, C.F. Shu, The virage image search engine: an open framework for image management, in Presented at the SPIE Storage and Retrieval for Image and Video Databases IV (1996) pp. 76–87

    Google Scholar 

  59. A. Pentland, R.W. Picard, S. Sclaroff, Photobook: content-based manipulation of image databases. Int. J. Comput. Vis. 18, 233–254 (1996)

    Article  Google Scholar 

  60. T. Huang, S. Mehrotra, K. Ramchandran, Multimedia analysis and retrieval system (MARS) project

    Google Scholar 

  61. Y. Rui, T.S. Huang, S.F. Chang, Image retrieval: current techniques, promising directions, and open issues. J. Vis. Commun. Image Represent. 10, 39–62 (1999)

    Article  Google Scholar 

  62. R. Zhao, W.I. Grosky, Negotiating the semantic gap: from feature maps to semantic landscapes. Pattern Recognit. 35, 593–600 (2002)

    Article  MATH  Google Scholar 

  63. Y. Liu, D. Zhang, G. Lu, W.Y. Ma, A survey of content-based image retrieval with high-level semantics. Pattern Recognit. 40, 262–282 (2007)

    Article  MATH  Google Scholar 

  64. D. Durkee, Why cloud computing will never be free. Queue 8, 20 (2010)

    Google Scholar 

  65. T. Sato, T. Kanade, E.K. Hughes, M.A. Smith, S. Satoh, Video OCR: indexing digital news libraries by recognition of superimposed captions. Multimed. Syst. 7, 385–395 (1999)

    Article  Google Scholar 

  66. P. Geetha, V. Narayanan, A survey of content-based video retrieval. J. Comput. Sci. 4, 474–486 (2008)

    Article  Google Scholar 

  67. J. Foote, Content-based retrieval of music and audio, in Presented at the Proceedings of SPIE (1997) pp. 138–147

    Google Scholar 

  68. Z. Liu, Q. Huang, Content-based indexing and retrieval-by-example in audio, in Presented at the IEEE International Conference on Multimedia and Expo (IEEE, 2000), pp. 877–880

    Google Scholar 

  69. J. Makhoul, F. Kubala, T. Leek, D. Liu, L. Nguyen, R. Schwartz, A. Srivastava, Speech and language technologies for audio indexing and retrieval. Proc. IEEE 88, 1338–1353 (2000)

    Article  Google Scholar 

  70. G. Bradski, The OpenCV Library. Dr Dobbs J. Software Tools (2000)

    Google Scholar 

  71. S. Taylor, Optimizing Applications for Multi-Core Processors, Using the Intel Integrated Performance Primitives (Intel Press, 2007)

    Google Scholar 

  72. E. Blasch, Z. Wang, H. Ling, K. Palaniappan, G. Chen, D. Shen, A. Aved, G. Seetharaman, Video-based activity analysis using the L1 tracker on VIRAT data, in IEEE Applied Imagery Pattern Recognition Workshop (2013)

    Google Scholar 

  73. A.J. Aved, Scene Understanding for Real Time Processing of Queries over Big Data Streaming Video. Ph.D. dissertation, University of Central Florida, 2013

    Google Scholar 

  74. H. Zha, X. He, C. Ding, H. Simon, M. Gu, Bipartite graph partitioning and data clustering, in Presented at the Proceedings of the Tenth International Conference on Information and Knowledge Management (ACM, 2001), pp. 25–32

    Google Scholar 

  75. H.W. Kuhn, The Hungarian method for the assignment problem. Nav. Res. Logist. Q. 2, 83–97 (1955)

    Article  MathSciNet  MATH  Google Scholar 

  76. J. Munkres, Algorithms for the assignment and transportation problems. J. Soc. Ind. Appl. Math. 5, 32–38 (1957)

    Article  MathSciNet  MATH  Google Scholar 

  77. G.A. Mills-Tettey, A. Stentz, M.B. Dias, The Dynamic Hungarian Algorithm for the Assignment Problem with Changing Costs (No. CMU-RI-TR-07-27) (Robotics Institute, Pittsburgh, PA, 2007)

    Google Scholar 

  78. R. Fisher, CAVIAR: context aware vision using image-based active recognition (WWW Document). URL http://homepages.inf.ed.ac.uk/rbf/CAVIAR/. Accessed 11 Dec 2011

Download references

Acknowledgments

This work is partly supported by the Air Force Office of Scientific Research (AFOSR) under the Dynamic Data Driven Application Systems program and the Air Force Research Lab.

Author information

Authors and Affiliations

  1. Air Force Research Lab, Rome, NY, USA

    Alex J. Aved & Erik Blasch

Authors
  1. Alex J. Aved
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Erik Blasch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Erik Blasch .

Editor information

Editors and Affiliations

  1. Department Mathematics and Computer Science, University of Udine, Udine, Italy

    Lauro Snidaro

  2. University Carlos III, Department of Computer Science and Engg. University Carlos III, Colmenarejo, Spain

    Jesús García

  3. Center Multisource Info. Fusion & Dept, Univ. at Buffalo, of Industrial and Engg Center Multisource Info. Fusion & Dept, Buffalo, New York, USA

    James Llinas

  4. Information Directorate, Air Force Research Laboratory Information Directorate, Rome, New York, USA

    Erik Blasch

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland (outside the USA)

About this chapter

Cite this chapter

Aved, A.J., Blasch, E. (2016). Context Understanding from Query-Based Streaming Video. In: Snidaro, L., García, J., Llinas, J., Blasch, E. (eds) Context-Enhanced Information Fusion. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-319-28971-7_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-28971-7_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-28969-4

  • Online ISBN: 978-3-319-28971-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation