Real-Time Estimation of Optical Flow Based on Optimized Haar Wavelet Features

  • Jan Salmen
  • Lukas Caup
  • Christian Igel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6576)


Estimation of optical flow is required in many computer vision applications. These applications often have to deal with strict time constraints. Therefore, flow algorithms with both high accuracy and computational efficiency are desirable. Accordingly, designing such a flow algorithm involves multi-objective optimization. In this work, we build on a popular algorithm developed for real-time applications. It is originally based on the Census transform and benefits from this encoding for table-based matching and tracking of interest points. We propose to use the more universal Haar wavelet features instead of the Census transform within the same framework. The resulting approach is more flexible, in particular it allows for sub-pixel accuracy. For comparison with the original method and another baseline algorithm, we considered both popular benchmark datasets as well as a long synthetic video sequence. We employed evolutionary multi-objective optimization to tune the algorithms. This allows to compare the different approaches in a systematic and unbiased way. Our results show that the overall performance of our method is significantly higher compared to the reference implementation.


Optical Flow Pareto Front Multiobjective Optimization Candidate Solution Interest Point 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Horn, B.K.P., Schunck, B.G.: Determining optical flow. Technical report, Massachusetts Institute of Technology, Cambridge, MA, USA (1980)Google Scholar
  2. 2.
    Stein, F.: Efficient computation of optical flow using the census transform. In: Rasmussen, C.E., Bülthoff, H.H., Schölkopf, B., Giese, M.A. (eds.) DAGM 2004. LNCS, vol. 3175, pp. 79–86. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  3. 3.
    Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Eklundh, J.-O. (ed.) ECCV 1994. LNCS, vol. 800, pp. 151–158. Springer, Heidelberg (1994)CrossRefGoogle Scholar
  4. 4.
    Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. International Journal of Computer Vision 12(1), 43–77 (1994)CrossRefGoogle Scholar
  5. 5.
    Baker, A., Scharstein, D., Lewis, J., Roth, S., Blak, M., Szeliski, R.: A database and evaluation methodology for optical flow. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1–8. IEEE Press, Los Alamitos (2007)Google Scholar
  6. 6.
    Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence, pp. 674–679 (1981)Google Scholar
  7. 7.
    Bouguet, J.Y.: Pyramid implementation of the Lucas Kanade feature tracker. In: OpenCV Documentation, Intel Corporation Microprocessor Research Labs (2000)Google Scholar
  8. 8.
    Tomasi, C., Kanade, T.: Detection and Tracking of Point Features. Technical Report CMU-CS-91-132, Carnegie Mellon University (1991)Google Scholar
  9. 9.
    Shi, J., Tomasi, C.: Good features to track. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 593–600. IEEE Press, Los Alamitos (1994)Google Scholar
  10. 10.
    Camus, T.: Real-time quantized optical flow. Journal of Real-Time Imaging 3, 71–86 (1995)CrossRefGoogle Scholar
  11. 11.
    Zhang, D., Lu, G.: An edge and color oriented optical flow estimation using block matching. In: Proceedings of the IEEE International Conference on Signal Processing, vol. 2, pp. 1026–1032. IEEE Press, Los Alamitos (2000)Google Scholar
  12. 12.
    Bruhn, A., Weickert, J., Kohlberger, T., Schnörr, C.: A multigrid platform for real-time motion computation with discontinuity-preserving variational methods. International Journal of Computer Vision 70(3), 257–277 (2006)CrossRefGoogle Scholar
  13. 13.
    Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L 1 optical flow. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 214–223. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  14. 14.
    Brox, T., Bregler, C., Malik, J.: Large displacement optical flow. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 41–48. IEEE Press, Los Alamitos (2009)Google Scholar
  15. 15.
    Steinbruecker, F., Pock, T., Cremers, D.: Large displacement optical flow computation without warping. In: Proceesings of the IEEE International Conference on Computer Vision, pp. 1609–1614. IEEE Press, Los Alamitos (2009)Google Scholar
  16. 16.
    Viola, P., Jones, M.: Robust real-time object detection. International Journal of Computer Vision 57(2), 137–154 (2001)CrossRefGoogle Scholar
  17. 17.
    Papageorgiou, C., Poggio, T.: A trainable system for object detection. International of Journal Computer Vision 38(1), 15–33 (2000)CrossRefzbMATHGoogle Scholar
  18. 18.
    Salmen, J., Suttorp, T., Edelbrunner, J., Igel, C.: Evolutionary optimization of wavelet feature sets for real-time pedestrian classification. In: Proceedings of the IEEE Conference on Hybrid Intelligent Systems, pp. 222–227. IEEE Press, Los Alamitos (2007)Google Scholar
  19. 19.
    Grabner, H., Bischof, H.: On-line boosting and vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 260–267. IEEE Press, Los Alamitos (2006)Google Scholar
  20. 20.
    Miettinen, K.: Nonlinear Multiobjective Optimization. Kluwer Academic Publishers, Dordrecht (1999)zbMATHGoogle Scholar
  21. 21.
    Igel, C., Hansen, N., Roth, S.: Covariance matrix adaptation for multi-objective optimization. Evolutionary Computation 15(1), 1–28 (2007)CrossRefGoogle Scholar
  22. 22.
    Beume, N., Naujoks, B., Emmerich, M.: SMS-EMOA: Multiobjective selection based on dominated hypervolume. European Journal of Operational Research 181(3), 1653–1669 (2007)CrossRefzbMATHGoogle Scholar
  23. 23.
    Suttorp, T., Hansen, N., Igel, C.: Efficient covariance matrix update for variable metric evolution strategies. Machine Learning 75(2), 167–197 (2009)CrossRefGoogle Scholar
  24. 24.
    Vaudrey, T., Rabe, C., Klette, R., Milburn, J.: Differences between stereo and motion behaviour on synthetic and real-world stereo sequences. In: Proceedings of the Conference on Image and Vision Computing New Zealand, pp. 1–6. IEEE Press, Los Alamitos (2008)Google Scholar
  25. 25.
    Igel, C., Glasmachers, T., Heidrich-Meisner, V.: Shark. Journal of Machine Learning Research 9, 993–996 (2008)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Jan Salmen
    • 1
  • Lukas Caup
    • 1
  • Christian Igel
    • 2
  1. 1.Institut für NeuroinformatikRuhr-Universität BochumBochumGermany
  2. 2.Department of Computer ScienceUniversity of CopenhagenCopenhagen ØDenmark

Personalised recommendations