Skip to main content

Object Trajectory Analysis in Video Indexing and Retrieval Applications

  • Chapter

Part of the book series: Studies in Computational Intelligence ((SCI,volume 287))

Abstract

The focus of this chapter is to present a survey on the most recent advances in representation and analysis of video object trajectories, with application to indexing and retrieval systems. We will review the main methodologies for the description of motion trajectories, as well as the indexing techniques and similarity metrics used in the retrieval process. Strengths and weaknesses of different solutions will be discussed through a comparative analysis, taking into account performance and implementation issues. In order to provide a deeper insight on the exploitation of these technologies in real world products, a selection of exampleswill be introduced and examined. The set of possible applications is very wide and includes (but it is not limited to) generic browsing of video databases, as well as more specific and context-dependent scenarios such as indexing and retrieval in visual surveillance, traffic monitoring, sport events analysis, video-on-demand, and video broadcasting.

This is a preview of subscription content, log in via an institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Wang, Z., Lu, L., Bovik, A.C.: Video quality assessment based on structural distortion measurement. Signal Processing: Image Communication 19(2), 121–132 (2004)

    Article  Google Scholar 

  2. Mansouri, A.R., Mitiche, A., El Feghali, R.: Spatio-temporal motion segmentation via level set partial differential equation. In: Fifth IEEE Southwest Symposium on Image Analysis and Interpretation, p. 243. IEEE Computer Society, Los Alamitos (2002)

    Chapter  Google Scholar 

  3. Naftel, A., Khalid, S.: Classifying spatiotemporal object trajectories using unsupervised learning in the coefficient feature space. Multimedia Systems 12(3), 227–238 (2006)

    Article  Google Scholar 

  4. Min, J., Kasturi, R.: Activity recognition based on multiple motion trajectories. In: Int. Conf. on Pattern Recognition, vol. 4, pp. 199–202 (August 2004)

    Google Scholar 

  5. Bashir, F., Khokhar, A., Schonfeld, D.: Object trajectory-based activity classification and recognition using hidden Markov models. IEEE Trans. on Image Processing 16(7), 1912–1919 (2007)

    Article  MathSciNet  Google Scholar 

  6. Morris, B.T., Trivedi, M.M.: A Survey of Vision-Based Trajectory Learning and Analysis for Surveillance. IEEE Trans. on Circuits and Systems for Video Tech. 18(8), 1114–1127 (2008)

    Article  Google Scholar 

  7. Oliver, N.M., Rosario, B., Pentland, A.P.: A bayesian computer vision system for modeling human interactions. IEEE Trans. on Pattern Analysis and Machine Intelligence 22(8), 831–843 (2000)

    Article  Google Scholar 

  8. Parameswaran, V., Chellappa, R.: View invariance for human action recognition. Int. J. Comput. Vision 66, 83–101 (2006)

    Article  Google Scholar 

  9. Ma, X., Bashir, F., Khokhar, A., Schonfeld, D.: Event analysis based on multiple interactive motion trajectories. IEEE Trans. Circuits Syst. Video Techn. 19(3), 397–406 (2009)

    Article  Google Scholar 

  10. Kaneko, T., Okudaira, M.: Encoding of arbitrary curves based on the chain code representation. IEEE Trans. on Communications 33(7), 697–707 (1985)

    Article  Google Scholar 

  11. Pavlidis, T.: Polygonal approximations by newton’s method. IEEE Trans. on Computing 26(8), 800–807 (1977)

    Article  MATH  Google Scholar 

  12. Von Stryk, O., Bulirsch, R.: Direct and indirect methods for trajectory optimization. Annals of Operations Research 37(1), 357–373 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  13. Medioni, G., Yasumoto, Y.: Corner detection and curve representation using cubic B-splines. In: IEEE Int. Conf. on Robotics and Automation, vol. 3 (1986)

    Google Scholar 

  14. Chen, X., Schonfeld, D., Khokhar, A.: Robust null space representation and sampling for view-invariant motion trajectory analysis. In: IEEE Conf. on Computer Vision and Pattern Recognition, pp. 1–6 (June 2008)

    Google Scholar 

  15. Rosin, P.L.: Techniques for assessing polygonal approximations of curves. IEEE Trans. on Pattern Analisys and Machine Intelligence 19(6), 659–666 (1997), doi:10.1109/34.601253

    Article  Google Scholar 

  16. Chan, W.S., Chin, F.: Approximation of polygonal curves with minimum number of line segments. LNCS, p. 378. Springer, Heidelberg (1992)

    Google Scholar 

  17. Sklansky, J., Gonzalez, V.: Fast polygonal approximation of digitized curves. Pattern Recognition 12(5), 327–331 (1980)

    Article  Google Scholar 

  18. Kurozumi, Y., Davis, W.A.: Polygonal approximation by the minimax method. Computer Graphics and Image Processing 19(3), 248–264 (1982)

    Article  MATH  Google Scholar 

  19. Wall, K., Danielsson, P.E.: A fast sequential method for polygonal approximation of digitized curves. Computer Vision Graphics Image Processing 28(3), 220–227 (1984)

    Article  Google Scholar 

  20. Kumar Ray, B., Ray, K.S.: Determination of optimal polygon from digital curve using L1 norm. Pattern Recognition 26(4), 505–509 (1993)

    Article  Google Scholar 

  21. Douglas, D.H., Peucker, T.K.: Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartographica: The Int. J. for Geographic Information and Geovisualization 10(2), 112–122 (1973)

    Article  Google Scholar 

  22. Ballard, D.H.: Strip trees: A hierarchical representation for curves. Communications of the ACM 24(5), 310–321 (1981)

    Article  Google Scholar 

  23. Duda, R.O., Hart, P.E.: Pattern classification and scene analysis, New York (1973)

    Google Scholar 

  24. Leu, J.G., Chen, L.: Polygonal approximation of 2-D shapes through boundary merging. Pattern Recognition Letters 7(4), 231–238 (1988)

    Article  Google Scholar 

  25. Ansari, N., Delp, E.J.: On detecting dominant points. Pattern Recognition 24(5), 441–451 (1991)

    Article  Google Scholar 

  26. Ray, B.K., Ray, K.S.: A new split-and-merge technique for polygonal approximation of chain coded curves. Pattern Recognition Letters 16(2), 161–169 (1995)

    Article  Google Scholar 

  27. Attneave, F.: Some informational aspects of visual perception. Psychological Review 61(3), 183–193 (1954)

    Article  Google Scholar 

  28. Teh, C.H., Chin, R.T.: On the detection of dominant points on digital curves. IEEE Trans. on Pattern Analysis and Machine Intelligence 11(8), 859–872 (1989)

    Article  Google Scholar 

  29. Held, A., Abe, K., Arcelli, C.: Towards a hierarchical contour description via dominant point detection. IEEE Trans. on Systems, Man and Cybernetics 24(6), 942–949 (1994)

    Article  Google Scholar 

  30. Zhu, P., Chirlian, P.M.: On critical point detection of digital shapes. IEEE Trans. on Pattern Analysis and Machine Intelligence 17(8), 737–748 (1995)

    Article  Google Scholar 

  31. Mikheev, A., Vincent, L., Faber, V., Inc, L.T., Seattle, W.A.: High-quality polygonal contour approximation based on relaxation. In: Int. Conf. on Document Analysis and Recognition, pp. 361–365 (2001)

    Google Scholar 

  32. Ho, S.Y., Chen, Y.C.: An efficient evolutionary algorithm for accurate polygonal approximation. Pattern Recognition 34(12), 2305–2317 (2001)

    Article  MATH  Google Scholar 

  33. Yin, P.Y.: Ant colony search algorithms for optimal polygonal approximation of plane curves. Pattern Recognition 36(8), 1783–1797 (2003)

    Article  MATH  Google Scholar 

  34. Yin, P.Y.: A discrete particle swarm algorithm for optimal polygonal approximation of digital curves. J. of Visual Communication and Image Representation 15(2), 241–260 (2004)

    Article  Google Scholar 

  35. Yin, P.Y.: A tabu search approach to polygonal approximation of digital curves. Int. J. of Pattern Recognition and Artificial Intelligence 14(2), 243–255 (2000)

    Article  Google Scholar 

  36. O’connell, K.J., Inc, M., Schaumburg, I.L.: Object-adaptive vertex-based shape coding method. IEEE Trans. on Circuits and Systems for Video Tech. 7(1), 251–255 (1997)

    Article  Google Scholar 

  37. Moore, B.: Principal component analysis in linear systems: Controllability, observability, and model reduction. IEEE Trans. on Automatic Control 26(1), 17–32 (1981)

    Article  MATH  Google Scholar 

  38. Jolliffe, I.T.: Principal Component Analysis. Springer, New York (2002)

    MATH  Google Scholar 

  39. Bashir, F., Khokhar, A., Schonfeld, D.: Segmented trajectory based indexing and retrieval of video data. In: IEEE Int. Conf. on Image Processing, vol. 2 (2003)

    Google Scholar 

  40. Needleman, S.B., Wunsch, C.D.: A general method applicable to the search for similarities in the amino acid sequence of two proteins. J. Molecular Biology 48(3), 443–453 (1970)

    Article  Google Scholar 

  41. Gdalyahu, Y., Weinshall, D.: Flexible syntactic matching of curves and its application to automatic hierarchical classification of silhouettes. IEEE Trans. on Pattern Analysis and Machine Intelligence 21(12), 1312–1328 (1999)

    Article  Google Scholar 

  42. Wu, G., Wu, Y., Jiao, L., Wang, Y.F., Chang, E.Y.: Multi-camera spatio-temporal fusion and biased sequence-data learning for security surveillance. In: ACM Int. Conf. on Multimedia, pp. 528–538. ACM, New York (2003)

    Google Scholar 

  43. Hsieh, J.W., Yu, S.L., Chen, Y.S.: Motion-based video retrieval by trajectory matching. IEEE Trans. on Circuits and Systems for Video Tech. 16(3), 396–409 (2006)

    Article  Google Scholar 

  44. Piotto, N., Conci, N., De Natale, F.G.B.: Syntactic matching of pedestrian trajectories for ambient intelligence applications. IEEE Trans. on Multimedia 11(7) (2009)

    Google Scholar 

  45. Ristad, E.S., Yianilos, P.N.: Learning string-edit distance. IEEE Trans. on Pattern Analysis and Machine Intelligence 20(5), 522–532 (1998)

    Article  Google Scholar 

  46. Schoenberg, I.J.: Contributions to the problem of approximation of equidistant data by analytic functions. Quarterly of Applied Mathematics 4, 45–99 (1946)

    MathSciNet  Google Scholar 

  47. Loncaric, S.: A survey of shape analysis techniques. Pattern Recognition (1998)

    Google Scholar 

  48. Ikebe, Y., Miyamoto, S.: Shape design, representation, and restoration with splines. Picture Engineering, 75–95 (1982)

    Google Scholar 

  49. Chen, W., Chang, S.F.: Motion trajectory matching of video objects. In: IS&T/SPIE, San Jose, CA (2000)

    Google Scholar 

  50. Bashir, F.I., Khokhar, A.A., Schonfeld, D.: Real-time motion trajectory-based indexing and retrieval of video sequences. IEEE Trans. on Multimedia 9(1), 58–65 (2007)

    Article  Google Scholar 

  51. Idris, F., Panchanathan, S.: Review of image and video indexing techniques. J. of Visual Communication and Image Representation 8(2), 146–166 (1997)

    Article  Google Scholar 

  52. Dagtas, S., Al-Khatib, W., Ghafoor, A., Kashyap, R.L., Res, P., Manor, B.: Models for motion-based video indexing and retrieval. IEEE Trans. on Image Processing 9(1), 88–101 (2000)

    Article  Google Scholar 

  53. Sellis, T., Roussopoulos, N., Faloutsos, C.: The R-tree: A dynamic index for multi-dimensional objects. The VLDB Journal, 507–518 (1987)

    Google Scholar 

  54. Theoderidis, Y., Vazirgiannis, M., Sellis, T.: Spatio-temporal indexing for large multimedia applications. In: IEEE Int. Conf. on Multimedia Computing and Systems, pp. 441–448 (1996)

    Google Scholar 

  55. Nascimento, M.A., Silva, J.R.O.: Towards Historical R-trees. In: ACM Symposium on Applied Computing, pp. 235–240. ACM, New York (1998)

    Google Scholar 

  56. Nascimento, M.A., Silva, J.R.O., Theodoridis, Y.: Evaluation of access structures for discretely moving points. LNCS, pp. 171–188. Springer, Heidelberg (1999)

    Google Scholar 

  57. Tao, Y., Papadias, D.: Mv3r-tree: a spatio-temporal access method for timestamp and interval queries. In: Int. Conf. on Very Large Data Bases, pp. 431–440. Morgan Kaufmann Publishers Inc., San Francisco (2001)

    Google Scholar 

  58. Becker, B., Gschwind, S., Ohler, T., Seeger, B., Widmayer, P.: An asymptotically optimal multiversion B-tree. The Int. J. on Very Large Data Bases 5(4), 264–275 (1996)

    Article  Google Scholar 

  59. Pfoser, D., Jensen, C.S., Theodoridis, Y.: Novel approaches in query processing for moving object trajectories. In: Int. Conf. on Very Large Data Bases, pp. 395–406. Morgan Kaufmann Publishers Inc., San Francisco (2000)

    Google Scholar 

  60. Stollnitz, E.J., DeRose, T.D., Salesin, D.H.: Wavelets for computer graphics: theory and applications. Morgan Kaufmann, San Francisco (1996)

    Google Scholar 

  61. Akansu, A.N., Haddad, R.A.: Multiresolution signal decomposition. Academic Press, Boston (1992)

    MATH  Google Scholar 

  62. Chan, K.P., Fu, A.W.C.: Efficient time series matching by wavelets. In: IEEE Int. Conf. on Data Engineering, pp. 126–135. Institute of Electrical and Electronics Engineers (1999)

    Google Scholar 

  63. Korn, F., Jagadish, H.V., Faloutsos, C.: Efficiently supporting ad hoc queries in large datasets of time sequences. In: ACM SIGMOD Int. Conf. on Management of Data, pp. 289–300. ACM, New York (1997)

    Chapter  Google Scholar 

  64. Berndt, D., Clifford, J.: Using dynamic time warping to find patterns in time series. In: AAAI 1994 Workshop on Knowledge Discovery and Databases, pp. 229–248 (1994)

    Google Scholar 

  65. Keogh, E.J., Pazzani, M.J.: Scaling up dynamic time warping for data mining application. In: ACM SIGKDD Int. Conf. on Knowledge discovery and data mining, pp. 285–289 (2000)

    Google Scholar 

  66. Hu, W., Tan, T., Wang, L., Maybank, S.: A survey on visual surveillance of object motion and behaviors. IEEE Trans. on Systems, Man and Cybernetics 34, 334–352 (2004)

    Google Scholar 

  67. Das, G., Gunopoulos, D., Mannila, H.: Finding similar time series. In: Komorowski, J., Żytkow, J.M. (eds.) PKDD 1997. LNCS, vol. 1263, pp. 88–100. Springer, Heidelberg (1997)

    Google Scholar 

  68. Vlachos, M., Hadjieleftheriou, M., Gunopoulos, D., Keogh, E.: Indexing multidimensional time-series with support for multiple distance measures. In: ACM SIGKDD, pp. 216–225 (2003)

    Google Scholar 

  69. Porikli, F.: Trajectory distance metric using hidden Markov model based representation. In: IEEE European Conf. on Computer Vision (2004)

    Google Scholar 

  70. Li, X., Hu, W., Hu, W.: A coarse-to-fine strategy for vehicle motion trajectory clustering. In: IEEE Int. Conf. on Pattern Recognition, vol. 1, pp. 591–594 (2006)

    Google Scholar 

  71. Anjum, N., Cavallaro, A.: Unsupervised fuzzy clustering for trajectory analysis. In: IEEE Int. Conf. on Image Processing, vol. 3, pp. 213–216 (2007)

    Google Scholar 

  72. Piciarelli, C., Foresti, G.L., Snidaro, L.: Trajectory clustering and its application for video surveillance. In: IEEE Conf. on Advanced Video and Signal Based Surveillance, pp. 40–45 (2005)

    Google Scholar 

  73. Johnson, N., Hogg, N.: Learning the distribution of object trajectories for event recognition. Image and Vision Computing 14, 609–615 (1996)

    Article  Google Scholar 

  74. Mecocci, A., Pannozzo, M.: A completely autonomous system that learns anomalous movements in advanced video surveillance applications. In: IEEE Int. Conf. on Image Processing, vol. 2, pp. 586–589 (2005)

    Google Scholar 

  75. Sumpter, N., Bulpitt, A.: Learning spatio-temporal patterns for predicting object behavior. Image and Visio Computing 18, 697–704 (2000)

    Article  Google Scholar 

  76. Owens, J., Hunter, A.: Application of the self-organizing map to trajectory classification. In: IEEE Int. Workshop Visual Surveillance, vol. 18, pp. 77–83 (2000)

    Google Scholar 

  77. Kohonen, T.: Self-organizing maps. Springer, Heidelberg (1995)

    Google Scholar 

  78. Hu, W.M., Xie, D., Tan, T.N.: A hierarchical self-organizing approach for learning the patterns of motion trajectories. IEEE Trans. on Neural Network 15, 135–144 (2004)

    Article  Google Scholar 

  79. Imran, N., Javed, O., Shah, M.: Multi feature path modeling for video surveillance. In: Int. Conf. on Pattern Recognition, pp. 716–719 (2004)

    Google Scholar 

  80. Marzal, A., Vidal, E.: Computation of normalized edit distance and applications. IEEE Trans. on Pattern Analysis and Machine Intelligence 15, 926–932 (1993)

    Article  Google Scholar 

  81. Chen, L., Otsu, M.T., Oria, V.: Symbolic representation and retrieval of moving object trajectories. In: ACM SIGMM Int. Workshop on Multimedia Information Retrieval, pp. 227–234 (2004)

    Google Scholar 

  82. Levenshtein, V.I.: Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady 10, 707–710 (1966)

    MathSciNet  Google Scholar 

  83. Zheng, J.B., Feng, D.D., Zhao, R.C.: Trajectory Matching and Classification of Video Moving Objects. In: IEEE Workshop on Multimedia Signal Processing, vol. 10, pp. 1–4 (2005)

    Google Scholar 

  84. Hu, W., Xie, D., Fu, Z., Zeng, W., Maybank, S.: Semantic-based surveillance video retrieval. IEEE Trans. on Image Processing 16, 1168–1181 (2007)

    Article  MathSciNet  Google Scholar 

  85. Calderara, S., Cucchiara, R., Prati, A.: A Dynamic programming technique for classifying trajectories. In: Int. Conf. on Image Analysis and Processing, pp. 137–142 (2007)

    Google Scholar 

  86. Piotto, N., Conci, N., De Natale, F.G.B.: Syntactic matching of pedestrian trajectories for behavioral analysis. In: Proc. of 10th IEEE Workshop on Multimedia Signal Processing, pp. 877–882 (2008)

    Google Scholar 

  87. Needleman, S.B., Wunsch, C.D.: A general method applicable to the search for similarities in the amino acid sequence of two proteins. J. of Molecular Biology 48, 443–453 (1970)

    Article  Google Scholar 

  88. Smith, T.F., Waterman, M.S.: Identification of Common Molecular Subsequences. J. of Molecular Biology 147, 195–197 (1981)

    Article  Google Scholar 

  89. Vlacos, M., Kollios, G., Gunopulos, D.: Discovering similar multidimensional trajectories. In: Int. Conf. on Data Engineering, pp. 673–684 (2002)

    Google Scholar 

  90. Shan, M.K., Lee, S.Y.: Content-based video retrieval via motion trajectories. In: Society of Photo-Optical Instrumentation Engineers (SPIE) Conf. Series, pp. 52–61 (1998)

    Google Scholar 

  91. Chang, S.F., Chen, W., Meng, H.J., Sundaram, H.: A fully automated content-based video search engine supportingspatiotemporal queries. IEEE Trans. on Circuits and Systems for Video Tech. 8(5), 602–615 (1998)

    Article  Google Scholar 

  92. Sahouria, E., Zakhor, A.: Motion indexing of video. In: IEEE Int. Conf. on Image Processing, vol. 2 (1997)

    Google Scholar 

  93. Jung, Y.K., Lee, K.W., Ho, Y.S.: Content-based event retrieval using semantic scene interpretation for automated traffic surveillance. IEEE Trans. on Intelligent Transportation Systems 2(3), 151–163 (2001)

    Article  Google Scholar 

  94. Basharat, A., Zhai, Y., Shah, M.: Content based video matching using spatiotemporal volumes. Computer Vision and Image Understanding 110(3), 360–377 (2008)

    Article  Google Scholar 

  95. Chang, S.F., Chen, W., Meng, H.J., Sundaram, H.: VideoQ: an automated content based video search system using visual cues. In: ACM Int. Conf. on Multimedia, pp. 313–324. ACM, New York (1997)

    Chapter  Google Scholar 

  96. Yoshitaka, A., Hosoda, Y., Yoshimitsu, M., Hirakawa, M., Ichikawa, T.: Violone: Video retrieval by motion example. J. of Visual Languages and Computing 7(4), 423–443 (1996)

    Article  Google Scholar 

  97. Aghbari, Z., Kaneko, K., Makinouchi, A.: Content-trajectory approach for searching video databases. IEEE Trans. on Multimedia 5(4), 516–531 (2003)

    Article  Google Scholar 

  98. Le, T., Boucher, A., Thonnat, M.: Subtrajectory-based video indexing and retrieval. In: Cham, T.-J., Cai, J., Dorai, C., Rajan, D., Chua, T.-S., Chia, L.-T. (eds.) MMM 2007. LNCS, vol. 4351, p. 418. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  99. Buxton, H., Gong, S.: Visual surveillance in a dynamic and uncertain world. Artificial Intelligence 78(1-2), 431–459 (1995)

    Article  Google Scholar 

  100. Remagnino, P., Tan, T., Baker, K.: Multi-agent visual surveillance of dynamic scenes. Image and Vision Computing 16(8), 529–532 (1998)

    Article  Google Scholar 

  101. Kuijpers, B., Othman, W.: Trajectory databases: Data models, uncertainty and complete query languages. In: Schwentick, T., Suciu, D. (eds.) ICDT 2007. LNCS, vol. 4353, pp. 224–238. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Broilo, M., Piotto, N., Boato, G., Conci, N., De Natale, F.G.B. (2010). Object Trajectory Analysis in Video Indexing and Retrieval Applications. In: Schonfeld, D., Shan, C., Tao, D., Wang, L. (eds) Video Search and Mining. Studies in Computational Intelligence, vol 287. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12900-1_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-12900-1_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-12899-8

  • Online ISBN: 978-3-642-12900-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics