Dense Point Trajectories by GPU-Accelerated Large Displacement Optical Flow

  • Narayanan Sundaram
  • Thomas Brox
  • Kurt Keutzer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6311)


Dense and accurate motion tracking is an important requirement for many video feature extraction algorithms. In this paper we provide a method for computing point trajectories based on a fast parallel implementation of a recent optical flow algorithm that tolerates fast motion. The parallel implementation of large displacement optical flow runs about 78× faster than the serial C++ version. This makes it practical to use in a variety of applications, among them point tracking. In the course of obtaining the fast implementation, we also proved that the fixed point matrix obtained in the optical flow technique is positive semi-definite. We compare the point tracking to the most commonly used motion tracker - the KLT tracker - on a number of sequences with ground truth motion. Our resulting technique tracks up to three orders of magnitude more points and is 46% more accurate than the KLT tracker. It also provides a tracking density of 48% and has an occlusion error of 3% compared to a density of 0.1% and occlusion error of 8% for the KLT tracker. Compared to the Particle Video tracker, we achieve 66% better accuracy while retaining the ability to handle large displacements while running an order of magnitude faster.


Conjugate Gradient Large Displacement Linear Solver Point Tracking Point Iteration 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

978-3-642-15549-9_32_MOESM1_ESM.avi (14.7 mb)
Electronic Supplementary Material (15,092 KB)


  1. 1.
    Sand, P., Teller, S.: Particle video: Long-range motion estimation using point trajectories. International Journal of Computer Vision 80, 72–91 (2008)CrossRefGoogle Scholar
  2. 2.
    Zach, C., Gallup, D., Frahm, J.M.: Fast gain-adaptive KLT tracking on the GPU. In: CVPR Workshop on Visual Computer Vision on GPU’s, CVGPU (2008)Google Scholar
  3. 3.
    Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L1 optical flow. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 214–223. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  4. 4.
    Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence (to appear, 2010)Google Scholar
  5. 5.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)Google Scholar
  6. 6.
    Shi, J., Tomasi, C.: Good features to track. In: CVPR, pp. 593–600 (1994)Google Scholar
  7. 7.
    Sand, P., Teller, S.: Particle video: Long-range motion estimation using point trajectories. In: CVPR (2006)Google Scholar
  8. 8.
    Liu, C., Freeman, W.T., Adelson, E.H., Weiss, Y.: Human-assisted motion annotation. In: CVPR (2008)Google Scholar
  9. 9.
    Bruhn, A., Weickert, J.: Towards ultimate motion estimation: Combining highest accuracy with real-time performance. In: ICCV ’05: Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV’05), Washington, DC, USA, vol. 1, pp. 749–755. IEEE Computer Society, Los Alamitos (2005)CrossRefGoogle Scholar
  10. 10.
    Grossauer, H., Thoman, P.: GPU-based multigrid: Real-time performance in high resolution nonlinear image processing. In: Gasteratos, A., Vincze, M., Tsotsos, J.K. (eds.) ICVS 2008. LNCS, vol. 5008, pp. 141–150. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  11. 11.
    Brox, T., Bregler, C., Malik, J.: Large displacement optical flow. In: CVPR (2009)Google Scholar
  12. 12.
    Gwosdek, P., Bruhn, A., Weickert, J.: High performance parallel optical flow algorithms on the Sony Playstation 3. Vision, Modeling and Visualization, 253–262 (2008)Google Scholar
  13. 13.
    Wedel, A., Pock, T., Zach, C., Bischof, H., Cremers, D.: An improved algorithm for TV-L1 optical flow. In: Statistical and Geometrical Approaches to Visual Motion Analysis: International Dagstuhl Seminar, Dagstuhl Castle, Germany, July 13-18, pp. 23–45 (2009), Revised PapersGoogle Scholar
  14. 14.
    Werlberger, M., Trobin, W., Pock, T., Wedel, A., Cremers, D., Bischof, H.: Anisotropic Huber-L1 optical flow. In: Proc. of the British Machine Vision Conference, BMVC (2009)Google Scholar
  15. 15.
    Lai, S.H., Vemuri, B.C.: Reliable and efficient computation of optical flow. International Journal of Computer Vision 29 (1998)Google Scholar
  16. 16.
    Mitiche, A., Mansouri, A.R.: On convergence of the Horn and Schunck optical-flow estimation method. IEEE Transactions on Image Processing 13, 848–852 (2004)CrossRefGoogle Scholar
  17. 17.
    Bruhn, A., Weickert, J., Schnörr, C.: Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods. Int. J. Comput. Vision 61, 211–231 (2005)CrossRefGoogle Scholar
  18. 18.
    Sinha, S.N., Frahm, J.M., Pollefeys, M., Genc, Y.: Feature tracking and matching in video using programmable graphics hardware. Machine Vision and Applications (2007)Google Scholar
  19. 19.
    Birchfield, S.T., Pundlik, S.J.: Joint tracking of features and edges. In: CVPR (2008)Google Scholar
  20. 20.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: CVPR (2006)Google Scholar
  21. 21.
    Sundaram, N., Brox, T., Keutzer, K.: Dense point trajectories by GPU-accelerated large displacement optical flow. Technical Report UCB/EECS-2010-104, EECS Department, University of California, Berkeley (2010)Google Scholar
  22. 22.
    Feingold, D.G., Varga, R.S.: Block diagonally dominant matrices and generalizations of the Gerschgorin circle theorem. Pacific J. Math. 12, 1241–1250 (1962)zbMATHMathSciNetGoogle Scholar
  23. 23.
    Bruhn, A.: Variational Optic Flow Computation: Accurate Modelling and Efficient Numerics. PhD thesis, Faculty of Mathematics and Computer Science, Saarland University, Germany (2006)Google Scholar
  24. 24.
    Stüben, K., Trottenberg, U.: Multigrid methods: Fundamental algorithms, model problem analysis and applications. Lecture Notes in Mathematics, vol. 960. Springer, Heidelberg (1982)Google Scholar
  25. 25.
    Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., Szeliski, R.: A database and evaluation methodology for optical flow. In: ICCV (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Narayanan Sundaram
    • 1
  • Thomas Brox
    • 1
  • Kurt Keutzer
    • 1
  1. 1.University of California at Berkeley 

Personalised recommendations