Skip to main content
SpringerLink
Log in

Completed Dense Scene Flow in RGB-D Space

  • Conference paper
  • First Online:
Computer Vision - ACCV 2014 Workshops (ACCV 2014)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 9008))

Included in the following conference series:

Abstract

Conventional scene flow containing only translational vectors is not able to model 3D motion with rotation properly. Moreover, the accuracy of 3D motion estimation is restricted by several challenges such as large displacement, noise, and missing data (caused by sensing techniques or occlusion). In terms of solution, there are two kinds of approaches: local approaches and global approaches. However, local approaches can not generate smooth motion field, and global approaches is difficult to handle large displacement motion. In this paper, a completed dense scene flow framework is proposed, which models both rotation and translation for general motion estimation. It combines both a local method and a global method considering their complementary characteristics to handle large displacement motion and enforce smoothness respectively. The proposed framework is applied on the RGB-D image space where the computation efficiency is further improved. According to the quantitative evaluation based on Middlebury dataset, our method outperforms other published methods. The improved performance is further confirmed on the real data acquired by Kinect sensor.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Hadfield, S., Bowden, R.: Scene particles: unregularized particle based scene flow estimation. IEEE Trans. Pattern Anal. Mach. Intell. 36, 564–576 (2014)

    Article  Google Scholar 

  2. Gottfried, J.-M., Fehr, J., Garbe, C.S.: Computing range flow from multi-modal Kinect data. In: Bebis, G., et al. (eds.) ISVC 2011, Part I. LNCS, vol. 6938, pp. 758–767. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  3. Herbst, E., Ren, X., Fox, D.: Rgb-d flow: Dense 3-d motion estimation using color and depth. In: IEEE International Conference on Robotics and Automation (ICRA). IEEE (2013)

    Google Scholar 

  4. Quiroga, J., Devernay, F., Crowley, J.L., et al.: Local/global scene flow estimation. In: ICIP-IEEE International Conference on Image Processing (2013)

    Google Scholar 

  5. Zhang, X., Chen, D., Yuan, Z., Zheng, N.: Dense scene flow based on depth and multi-channel bilateral filter. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012, Part III. LNCS, vol. 7726, pp. 140–151. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  6. Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.: Patchmatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics-TOG 28, 24 (2009)

    Google Scholar 

  7. Barnes, C., Shechtman, E., Goldman, D.B., Finkelstein, A.: The generalized patchmatch correspondence algorithm. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 29–43. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  8. Korman, S., Avidan, S.: Coherency sensitive hashing. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1607–1614. IEEE (2011)

    Google Scholar 

  9. Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans. Pattern Anal. Mach. Intell. 33, 500–513 (2011)

    Article  Google Scholar 

  10. Hornacek, M., Fitzgibbon, A., Carsten, R.: Sphereflow: 6 dof scene flow from rgb-d pairs. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2014)

    Google Scholar 

  11. Junejo, I.N., Dexter, E., Laptev, I., Perez, P.: View-independent action recognition from temporal self-similarities. IEEE Trans. Pattern Anal. Mach. Intell. 33, 172–185 (2011)

    Article  Google Scholar 

  12. Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. Int. J. Comput. Vis. 92, 1–31 (2011)

    Article  Google Scholar 

  13. Vedula, S., Rander, P., Collins, R., Kanade, T.: Three-dimensional scene flow. IEEE Trans. Pattern Anal. Mach. Intell. 27, 475–480 (2005)

    Article  Google Scholar 

  14. Vogel, C., Schindler, K., Roth, S.: Piecewise rigid scene flow. In: 2013 IEEE International Conference on Computer Vision (ICCV). IEEE (2013)

    Google Scholar 

  15. Basha, T., Moses, Y., Kiryati, N.: Multi-view scene flow estimation: a view centered variational approach. Int. J. Comput. Vis. 101, 6–21 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  16. Huguet, F., Devernay, F.: A variational method for scene flow estimation from stereo sequences. In: IEEE 11th International Conference on Computer Vision, ICCV 2007, pp. 1–7. IEEE (2007)

    Google Scholar 

  17. HaCohen, Y., Shechtman, E., Goldman, D.B., Lischinski, D.: Non-rigid dense correspondence with applications for image enhancement. ACM Trans. Graph. (TOG) 30, 70:1–70:9 (2011)

    Article  Google Scholar 

  18. Eshet, Y., Korman, S., Ofek, E., Avidan, S.: Dcsh-matching patches in rgbd images. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 89–96. IEEE (2013)

    Google Scholar 

  19. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)

    Article  Google Scholar 

  20. Lempitsky, V., Rother, C., Roth, S., Blake, A.: Fusion moves for markov random field optimization. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1392–1405 (2010)

    Article  Google Scholar 

  21. Boros, E., Hammer, P.L.: Pseudo-boolean optimization. Discrete Appl. Math. 123, 155–225 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  22. Xu, L., Jia, J., Matsushita, Y.: Motion detail preserving optical flow estimation. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1744–1757 (2012)

    Article  Google Scholar 

  23. Willimon, B., Hickson, S., Walker, I., Birchfield, S.: An energy minimization approach to 3d non-rigid deformable surface estimation using rgbd data. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2711–2717. IEEE (2012)

    Google Scholar 

Download references

Acknowledgement

This work was supported by Microsoft Research, Redmond. We also acknowledge Minqi Li for recording the Kinect RGB-D data sequence in our experiment.

Author information

Authors and Affiliations

  1. Advanced Analytics Institute, University of Technology, Sydney, Australia

    Yucheng Wang, Jian Zhang & Qiang Wu

  2. Microsoft Research, Redmond, WA, USA

    Zicheng Liu, Philip Chou & Zhengyou Zhang

  3. Beijing Lab of Intelligent Information Technology, Beijing Institute of Technology, Beijing, China

    Yucheng Wang & Yunde Jia

Authors
  1. Yucheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zicheng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qiang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Philip Chou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhengyou Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yunde Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yucheng Wang .

Editor information

Editors and Affiliations

  1. Center for Visual Information Technology, International Institute of Information Technology, Hyderabad, India

    C.V. Jawahar

  2. Institue of Computing Technology, Chinese Academy of Sciences, Beijing, China

    Shiguang Shan

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Wang, Y. et al. (2015). Completed Dense Scene Flow in RGB-D Space. In: Jawahar, C., Shan, S. (eds) Computer Vision - ACCV 2014 Workshops. ACCV 2014. Lecture Notes in Computer Science(), vol 9008. Springer, Cham. https://doi.org/10.1007/978-3-319-16628-5_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-16628-5_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-16627-8

  • Online ISBN: 978-3-319-16628-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation