Abstract
The focus of this chapter is to present a survey on the most recent advances in representation and analysis of video object trajectories, with application to indexing and retrieval systems. We will review the main methodologies for the description of motion trajectories, as well as the indexing techniques and similarity metrics used in the retrieval process. Strengths and weaknesses of different solutions will be discussed through a comparative analysis, taking into account performance and implementation issues. In order to provide a deeper insight on the exploitation of these technologies in real world products, a selection of exampleswill be introduced and examined. The set of possible applications is very wide and includes (but it is not limited to) generic browsing of video databases, as well as more specific and context-dependent scenarios such as indexing and retrieval in visual surveillance, traffic monitoring, sport events analysis, video-on-demand, and video broadcasting.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Wang, Z., Lu, L., Bovik, A.C.: Video quality assessment based on structural distortion measurement. Signal Processing: Image Communication 19(2), 121–132 (2004)
Mansouri, A.R., Mitiche, A., El Feghali, R.: Spatio-temporal motion segmentation via level set partial differential equation. In: Fifth IEEE Southwest Symposium on Image Analysis and Interpretation, p. 243. IEEE Computer Society, Los Alamitos (2002)
Naftel, A., Khalid, S.: Classifying spatiotemporal object trajectories using unsupervised learning in the coefficient feature space. Multimedia Systems 12(3), 227–238 (2006)
Min, J., Kasturi, R.: Activity recognition based on multiple motion trajectories. In: Int. Conf. on Pattern Recognition, vol. 4, pp. 199–202 (August 2004)
Bashir, F., Khokhar, A., Schonfeld, D.: Object trajectory-based activity classification and recognition using hidden Markov models. IEEE Trans. on Image Processing 16(7), 1912–1919 (2007)
Morris, B.T., Trivedi, M.M.: A Survey of Vision-Based Trajectory Learning and Analysis for Surveillance. IEEE Trans. on Circuits and Systems for Video Tech. 18(8), 1114–1127 (2008)
Oliver, N.M., Rosario, B., Pentland, A.P.: A bayesian computer vision system for modeling human interactions. IEEE Trans. on Pattern Analysis and Machine Intelligence 22(8), 831–843 (2000)
Parameswaran, V., Chellappa, R.: View invariance for human action recognition. Int. J. Comput. Vision 66, 83–101 (2006)
Ma, X., Bashir, F., Khokhar, A., Schonfeld, D.: Event analysis based on multiple interactive motion trajectories. IEEE Trans. Circuits Syst. Video Techn. 19(3), 397–406 (2009)
Kaneko, T., Okudaira, M.: Encoding of arbitrary curves based on the chain code representation. IEEE Trans. on Communications 33(7), 697–707 (1985)
Pavlidis, T.: Polygonal approximations by newton’s method. IEEE Trans. on Computing 26(8), 800–807 (1977)
Von Stryk, O., Bulirsch, R.: Direct and indirect methods for trajectory optimization. Annals of Operations Research 37(1), 357–373 (1992)
Medioni, G., Yasumoto, Y.: Corner detection and curve representation using cubic B-splines. In: IEEE Int. Conf. on Robotics and Automation, vol. 3 (1986)
Chen, X., Schonfeld, D., Khokhar, A.: Robust null space representation and sampling for view-invariant motion trajectory analysis. In: IEEE Conf. on Computer Vision and Pattern Recognition, pp. 1–6 (June 2008)
Rosin, P.L.: Techniques for assessing polygonal approximations of curves. IEEE Trans. on Pattern Analisys and Machine Intelligence 19(6), 659–666 (1997), doi:10.1109/34.601253
Chan, W.S., Chin, F.: Approximation of polygonal curves with minimum number of line segments. LNCS, p. 378. Springer, Heidelberg (1992)
Sklansky, J., Gonzalez, V.: Fast polygonal approximation of digitized curves. Pattern Recognition 12(5), 327–331 (1980)
Kurozumi, Y., Davis, W.A.: Polygonal approximation by the minimax method. Computer Graphics and Image Processing 19(3), 248–264 (1982)
Wall, K., Danielsson, P.E.: A fast sequential method for polygonal approximation of digitized curves. Computer Vision Graphics Image Processing 28(3), 220–227 (1984)
Kumar Ray, B., Ray, K.S.: Determination of optimal polygon from digital curve using L1 norm. Pattern Recognition 26(4), 505–509 (1993)
Douglas, D.H., Peucker, T.K.: Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartographica: The Int. J. for Geographic Information and Geovisualization 10(2), 112–122 (1973)
Ballard, D.H.: Strip trees: A hierarchical representation for curves. Communications of the ACM 24(5), 310–321 (1981)
Duda, R.O., Hart, P.E.: Pattern classification and scene analysis, New York (1973)
Leu, J.G., Chen, L.: Polygonal approximation of 2-D shapes through boundary merging. Pattern Recognition Letters 7(4), 231–238 (1988)
Ansari, N., Delp, E.J.: On detecting dominant points. Pattern Recognition 24(5), 441–451 (1991)
Ray, B.K., Ray, K.S.: A new split-and-merge technique for polygonal approximation of chain coded curves. Pattern Recognition Letters 16(2), 161–169 (1995)
Attneave, F.: Some informational aspects of visual perception. Psychological Review 61(3), 183–193 (1954)
Teh, C.H., Chin, R.T.: On the detection of dominant points on digital curves. IEEE Trans. on Pattern Analysis and Machine Intelligence 11(8), 859–872 (1989)
Held, A., Abe, K., Arcelli, C.: Towards a hierarchical contour description via dominant point detection. IEEE Trans. on Systems, Man and Cybernetics 24(6), 942–949 (1994)
Zhu, P., Chirlian, P.M.: On critical point detection of digital shapes. IEEE Trans. on Pattern Analysis and Machine Intelligence 17(8), 737–748 (1995)
Mikheev, A., Vincent, L., Faber, V., Inc, L.T., Seattle, W.A.: High-quality polygonal contour approximation based on relaxation. In: Int. Conf. on Document Analysis and Recognition, pp. 361–365 (2001)
Ho, S.Y., Chen, Y.C.: An efficient evolutionary algorithm for accurate polygonal approximation. Pattern Recognition 34(12), 2305–2317 (2001)
Yin, P.Y.: Ant colony search algorithms for optimal polygonal approximation of plane curves. Pattern Recognition 36(8), 1783–1797 (2003)
Yin, P.Y.: A discrete particle swarm algorithm for optimal polygonal approximation of digital curves. J. of Visual Communication and Image Representation 15(2), 241–260 (2004)
Yin, P.Y.: A tabu search approach to polygonal approximation of digital curves. Int. J. of Pattern Recognition and Artificial Intelligence 14(2), 243–255 (2000)
O’connell, K.J., Inc, M., Schaumburg, I.L.: Object-adaptive vertex-based shape coding method. IEEE Trans. on Circuits and Systems for Video Tech. 7(1), 251–255 (1997)
Moore, B.: Principal component analysis in linear systems: Controllability, observability, and model reduction. IEEE Trans. on Automatic Control 26(1), 17–32 (1981)
Jolliffe, I.T.: Principal Component Analysis. Springer, New York (2002)
Bashir, F., Khokhar, A., Schonfeld, D.: Segmented trajectory based indexing and retrieval of video data. In: IEEE Int. Conf. on Image Processing, vol. 2 (2003)
Needleman, S.B., Wunsch, C.D.: A general method applicable to the search for similarities in the amino acid sequence of two proteins. J. Molecular Biology 48(3), 443–453 (1970)
Gdalyahu, Y., Weinshall, D.: Flexible syntactic matching of curves and its application to automatic hierarchical classification of silhouettes. IEEE Trans. on Pattern Analysis and Machine Intelligence 21(12), 1312–1328 (1999)
Wu, G., Wu, Y., Jiao, L., Wang, Y.F., Chang, E.Y.: Multi-camera spatio-temporal fusion and biased sequence-data learning for security surveillance. In: ACM Int. Conf. on Multimedia, pp. 528–538. ACM, New York (2003)
Hsieh, J.W., Yu, S.L., Chen, Y.S.: Motion-based video retrieval by trajectory matching. IEEE Trans. on Circuits and Systems for Video Tech. 16(3), 396–409 (2006)
Piotto, N., Conci, N., De Natale, F.G.B.: Syntactic matching of pedestrian trajectories for ambient intelligence applications. IEEE Trans. on Multimedia 11(7) (2009)
Ristad, E.S., Yianilos, P.N.: Learning string-edit distance. IEEE Trans. on Pattern Analysis and Machine Intelligence 20(5), 522–532 (1998)
Schoenberg, I.J.: Contributions to the problem of approximation of equidistant data by analytic functions. Quarterly of Applied Mathematics 4, 45–99 (1946)
Loncaric, S.: A survey of shape analysis techniques. Pattern Recognition (1998)
Ikebe, Y., Miyamoto, S.: Shape design, representation, and restoration with splines. Picture Engineering, 75–95 (1982)
Chen, W., Chang, S.F.: Motion trajectory matching of video objects. In: IS&T/SPIE, San Jose, CA (2000)
Bashir, F.I., Khokhar, A.A., Schonfeld, D.: Real-time motion trajectory-based indexing and retrieval of video sequences. IEEE Trans. on Multimedia 9(1), 58–65 (2007)
Idris, F., Panchanathan, S.: Review of image and video indexing techniques. J. of Visual Communication and Image Representation 8(2), 146–166 (1997)
Dagtas, S., Al-Khatib, W., Ghafoor, A., Kashyap, R.L., Res, P., Manor, B.: Models for motion-based video indexing and retrieval. IEEE Trans. on Image Processing 9(1), 88–101 (2000)
Sellis, T., Roussopoulos, N., Faloutsos, C.: The R-tree: A dynamic index for multi-dimensional objects. The VLDB Journal, 507–518 (1987)
Theoderidis, Y., Vazirgiannis, M., Sellis, T.: Spatio-temporal indexing for large multimedia applications. In: IEEE Int. Conf. on Multimedia Computing and Systems, pp. 441–448 (1996)
Nascimento, M.A., Silva, J.R.O.: Towards Historical R-trees. In: ACM Symposium on Applied Computing, pp. 235–240. ACM, New York (1998)
Nascimento, M.A., Silva, J.R.O., Theodoridis, Y.: Evaluation of access structures for discretely moving points. LNCS, pp. 171–188. Springer, Heidelberg (1999)
Tao, Y., Papadias, D.: Mv3r-tree: a spatio-temporal access method for timestamp and interval queries. In: Int. Conf. on Very Large Data Bases, pp. 431–440. Morgan Kaufmann Publishers Inc., San Francisco (2001)
Becker, B., Gschwind, S., Ohler, T., Seeger, B., Widmayer, P.: An asymptotically optimal multiversion B-tree. The Int. J. on Very Large Data Bases 5(4), 264–275 (1996)
Pfoser, D., Jensen, C.S., Theodoridis, Y.: Novel approaches in query processing for moving object trajectories. In: Int. Conf. on Very Large Data Bases, pp. 395–406. Morgan Kaufmann Publishers Inc., San Francisco (2000)
Stollnitz, E.J., DeRose, T.D., Salesin, D.H.: Wavelets for computer graphics: theory and applications. Morgan Kaufmann, San Francisco (1996)
Akansu, A.N., Haddad, R.A.: Multiresolution signal decomposition. Academic Press, Boston (1992)
Chan, K.P., Fu, A.W.C.: Efficient time series matching by wavelets. In: IEEE Int. Conf. on Data Engineering, pp. 126–135. Institute of Electrical and Electronics Engineers (1999)
Korn, F., Jagadish, H.V., Faloutsos, C.: Efficiently supporting ad hoc queries in large datasets of time sequences. In: ACM SIGMOD Int. Conf. on Management of Data, pp. 289–300. ACM, New York (1997)
Berndt, D., Clifford, J.: Using dynamic time warping to find patterns in time series. In: AAAI 1994 Workshop on Knowledge Discovery and Databases, pp. 229–248 (1994)
Keogh, E.J., Pazzani, M.J.: Scaling up dynamic time warping for data mining application. In: ACM SIGKDD Int. Conf. on Knowledge discovery and data mining, pp. 285–289 (2000)
Hu, W., Tan, T., Wang, L., Maybank, S.: A survey on visual surveillance of object motion and behaviors. IEEE Trans. on Systems, Man and Cybernetics 34, 334–352 (2004)
Das, G., Gunopoulos, D., Mannila, H.: Finding similar time series. In: Komorowski, J., Żytkow, J.M. (eds.) PKDD 1997. LNCS, vol. 1263, pp. 88–100. Springer, Heidelberg (1997)
Vlachos, M., Hadjieleftheriou, M., Gunopoulos, D., Keogh, E.: Indexing multidimensional time-series with support for multiple distance measures. In: ACM SIGKDD, pp. 216–225 (2003)
Porikli, F.: Trajectory distance metric using hidden Markov model based representation. In: IEEE European Conf. on Computer Vision (2004)
Li, X., Hu, W., Hu, W.: A coarse-to-fine strategy for vehicle motion trajectory clustering. In: IEEE Int. Conf. on Pattern Recognition, vol. 1, pp. 591–594 (2006)
Anjum, N., Cavallaro, A.: Unsupervised fuzzy clustering for trajectory analysis. In: IEEE Int. Conf. on Image Processing, vol. 3, pp. 213–216 (2007)
Piciarelli, C., Foresti, G.L., Snidaro, L.: Trajectory clustering and its application for video surveillance. In: IEEE Conf. on Advanced Video and Signal Based Surveillance, pp. 40–45 (2005)
Johnson, N., Hogg, N.: Learning the distribution of object trajectories for event recognition. Image and Vision Computing 14, 609–615 (1996)
Mecocci, A., Pannozzo, M.: A completely autonomous system that learns anomalous movements in advanced video surveillance applications. In: IEEE Int. Conf. on Image Processing, vol. 2, pp. 586–589 (2005)
Sumpter, N., Bulpitt, A.: Learning spatio-temporal patterns for predicting object behavior. Image and Visio Computing 18, 697–704 (2000)
Owens, J., Hunter, A.: Application of the self-organizing map to trajectory classification. In: IEEE Int. Workshop Visual Surveillance, vol. 18, pp. 77–83 (2000)
Kohonen, T.: Self-organizing maps. Springer, Heidelberg (1995)
Hu, W.M., Xie, D., Tan, T.N.: A hierarchical self-organizing approach for learning the patterns of motion trajectories. IEEE Trans. on Neural Network 15, 135–144 (2004)
Imran, N., Javed, O., Shah, M.: Multi feature path modeling for video surveillance. In: Int. Conf. on Pattern Recognition, pp. 716–719 (2004)
Marzal, A., Vidal, E.: Computation of normalized edit distance and applications. IEEE Trans. on Pattern Analysis and Machine Intelligence 15, 926–932 (1993)
Chen, L., Otsu, M.T., Oria, V.: Symbolic representation and retrieval of moving object trajectories. In: ACM SIGMM Int. Workshop on Multimedia Information Retrieval, pp. 227–234 (2004)
Levenshtein, V.I.: Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady 10, 707–710 (1966)
Zheng, J.B., Feng, D.D., Zhao, R.C.: Trajectory Matching and Classification of Video Moving Objects. In: IEEE Workshop on Multimedia Signal Processing, vol. 10, pp. 1–4 (2005)
Hu, W., Xie, D., Fu, Z., Zeng, W., Maybank, S.: Semantic-based surveillance video retrieval. IEEE Trans. on Image Processing 16, 1168–1181 (2007)
Calderara, S., Cucchiara, R., Prati, A.: A Dynamic programming technique for classifying trajectories. In: Int. Conf. on Image Analysis and Processing, pp. 137–142 (2007)
Piotto, N., Conci, N., De Natale, F.G.B.: Syntactic matching of pedestrian trajectories for behavioral analysis. In: Proc. of 10th IEEE Workshop on Multimedia Signal Processing, pp. 877–882 (2008)
Needleman, S.B., Wunsch, C.D.: A general method applicable to the search for similarities in the amino acid sequence of two proteins. J. of Molecular Biology 48, 443–453 (1970)
Smith, T.F., Waterman, M.S.: Identification of Common Molecular Subsequences. J. of Molecular Biology 147, 195–197 (1981)
Vlacos, M., Kollios, G., Gunopulos, D.: Discovering similar multidimensional trajectories. In: Int. Conf. on Data Engineering, pp. 673–684 (2002)
Shan, M.K., Lee, S.Y.: Content-based video retrieval via motion trajectories. In: Society of Photo-Optical Instrumentation Engineers (SPIE) Conf. Series, pp. 52–61 (1998)
Chang, S.F., Chen, W., Meng, H.J., Sundaram, H.: A fully automated content-based video search engine supportingspatiotemporal queries. IEEE Trans. on Circuits and Systems for Video Tech. 8(5), 602–615 (1998)
Sahouria, E., Zakhor, A.: Motion indexing of video. In: IEEE Int. Conf. on Image Processing, vol. 2 (1997)
Jung, Y.K., Lee, K.W., Ho, Y.S.: Content-based event retrieval using semantic scene interpretation for automated traffic surveillance. IEEE Trans. on Intelligent Transportation Systems 2(3), 151–163 (2001)
Basharat, A., Zhai, Y., Shah, M.: Content based video matching using spatiotemporal volumes. Computer Vision and Image Understanding 110(3), 360–377 (2008)
Chang, S.F., Chen, W., Meng, H.J., Sundaram, H.: VideoQ: an automated content based video search system using visual cues. In: ACM Int. Conf. on Multimedia, pp. 313–324. ACM, New York (1997)
Yoshitaka, A., Hosoda, Y., Yoshimitsu, M., Hirakawa, M., Ichikawa, T.: Violone: Video retrieval by motion example. J. of Visual Languages and Computing 7(4), 423–443 (1996)
Aghbari, Z., Kaneko, K., Makinouchi, A.: Content-trajectory approach for searching video databases. IEEE Trans. on Multimedia 5(4), 516–531 (2003)
Le, T., Boucher, A., Thonnat, M.: Subtrajectory-based video indexing and retrieval. In: Cham, T.-J., Cai, J., Dorai, C., Rajan, D., Chua, T.-S., Chia, L.-T. (eds.) MMM 2007. LNCS, vol. 4351, p. 418. Springer, Heidelberg (2007)
Buxton, H., Gong, S.: Visual surveillance in a dynamic and uncertain world. Artificial Intelligence 78(1-2), 431–459 (1995)
Remagnino, P., Tan, T., Baker, K.: Multi-agent visual surveillance of dynamic scenes. Image and Vision Computing 16(8), 529–532 (1998)
Kuijpers, B., Othman, W.: Trajectory databases: Data models, uncertainty and complete query languages. In: Schwentick, T., Suciu, D. (eds.) ICDT 2007. LNCS, vol. 4353, pp. 224–238. Springer, Heidelberg (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Broilo, M., Piotto, N., Boato, G., Conci, N., De Natale, F.G.B. (2010). Object Trajectory Analysis in Video Indexing and Retrieval Applications. In: Schonfeld, D., Shan, C., Tao, D., Wang, L. (eds) Video Search and Mining. Studies in Computational Intelligence, vol 287. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12900-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-12900-1_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12899-8
Online ISBN: 978-3-642-12900-1
eBook Packages: EngineeringEngineering (R0)