Advertisement

On the Nonlinear Statistics of Optical Flow

  • Henry Adams
  • Johnathan Bush
  • Brittany Carr
  • Lara Kassab
  • Joshua Mirth
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11382)

Abstract

In A naturalistic open source movie for optical flow evaluation, Butler et al. create a database of ground-truth optical flow from the computer-generated video Sintel. We study the high-contrast \(3\times 3\) patches from this video, and provide evidence that this dataset is well-modeled by a torus (a nonlinear 2-dimensional manifold). Our main tools are persistent homology and zigzag persistence, which are popular techniques from the field of computational topology. We show that the optical flow torus model is naturally equipped with the structure of a fiber bundle, which is furthermore related to the statistics of range images.

Keywords

Optical flow Computational topology Persistent homology Fiber bundle Zigzag persistence 

Notes

Acknowledgements

We would like to thank Gunnar Carlsson, Bradley Nelson, Jose Perea, and Guillermo Sapiro for helpful conversations.

References

  1. 1.
    Adams, H., Atanasov, A., Carlsson, G.: Nudged elastic band in topological data analysis. Topological Methods Nonlinear Anal. 45(1), 247–272 (2015)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Adams, H., Carlsson, G.: On the nonlinear statistics of range image patches. SIAM J. Imaging Sci. 2(1), 110–117 (2009)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Adams, H., et al.: Persistence images: a vector representation of persistent homology. J. Mach. Learn. Res. 18(8), 1–35 (2017)MathSciNetGoogle Scholar
  4. 4.
    Armstrong, M.A.: Basic Topology. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-1-4757-1793-8zbMATHGoogle Scholar
  5. 5.
    Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. Int. J. Comput. Vis. 92(1), 1–31 (2011)Google Scholar
  6. 6.
    Bao, W., Li, H., Li, N., Jiang, W.: A liveness detection method for face recognition based on optical flow field. In: 2009 International Conference on Image Analysis and Signal Processing, IASP 2009, pp. 233–236. IEEE (2009)Google Scholar
  7. 7.
    Barron, J.L., Fleet, D.J., Beauchemin, S.S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994)Google Scholar
  8. 8.
    Baryshnikov, Y., Ghrist, R.: Target enumeration via euler characteristic integrals. SIAM J. Appl. Math. 70(3), 825–844 (2009)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Bauer, U.: Ripser: a lean C++ code for the computation of Vietoris-Rips persistence barcodes. Software (2017). https://github.com/Ripser/ripser
  10. 10.
    Beauchemin, S.S., Barron, J.L.: The computation of optical flow. ACM Comput. Surv. (CSUR) 27(3), 433–466 (1995)Google Scholar
  11. 11.
    Bendich, P., Marron, J.S., Miller, E., Pieloch, A., Skwerer, S.: Persistent homology analysis of brain artery trees. Ann. Appl. Stat. 10(1), 198 (2016)MathSciNetGoogle Scholar
  12. 12.
    Bubenik, P.: Statistical topological data analysis using persistence landscapes. J. Mach. Learn. Res. 16(1), 77–102 (2015)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Burghelea, D., Dey, T.K.: Topological persistence for circle-valued maps. Discrete Comput. Geom. 50(1), 69–98 (2013)MathSciNetzbMATHGoogle Scholar
  14. 14.
    Butler, D.J., Wulff, J., Stanley, G.B., Black, M.J.: A naturalistic open source movie for optical flow evaluation. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7577, pp. 611–625. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-33783-3_44Google Scholar
  15. 15.
    Carlsson, G.: Topology and data. Bull. Am. Math. Soc. 46(2), 255–308 (2009)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Carlsson, G., De Silva, V., Morozov, D.: Zigzag persistent homologyand real-valued functions. In: Proceedings of the Twenty-Fifth annual Symposium on Computational Geometry, pp. 247–256. ACM (2009)Google Scholar
  17. 17.
    Carlsson, G., Ishkhanov, T., De Silva, V., Zomorodian, A.: On the local behavior of spaces of natural images. Int. J. Comput. Vis. 76(1), 1–12 (2008)MathSciNetGoogle Scholar
  18. 18.
    Carlsson, G., de Silva, V.: Zigzag persistence. Found. Comput. Math. 10(4), 367–405 (2010)MathSciNetzbMATHGoogle Scholar
  19. 19.
    Chung, M.K., Bubenik, P., Kim, P.T.: Persistence diagrams of cortical surface data. In: Prince, J.L., Pham, D.L., Myers, K.J. (eds.) IPMI 2009. LNCS, vol. 5636, pp. 386–397. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-02498-6_32Google Scholar
  20. 20.
    De Silva, V., Carlsson, G.: Topological estimation using witness complexes. SPBG 4, 157–166 (2004)Google Scholar
  21. 21.
    Edelsbrunner, H., Harer, J.L.: Computational Topology: An Introduction. American Mathematical Society, Providence (2010)zbMATHGoogle Scholar
  22. 22.
    Edelsbrunner, H., Letscher, D., Zomorodian, A.: Topological persistence and simplification. In: 2000 Proceedings of 41st Annual Symposium on Foundations of Computer Science, pp. 454–463. IEEE (2000)Google Scholar
  23. 23.
    Fleet, D., Weiss, Y.: Optical flow estimation. In: Paragios, N., Chen, Y., Faugeras, O. (eds.) Handbook of Mathematical Models in Computer Vision, pp. 237–257. Springer, Boston (2006).  https://doi.org/10.1007/0-387-28831-7_15Google Scholar
  24. 24.
    Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. (IJRR) 32, 1231–1237 (2013)Google Scholar
  25. 25.
    Hatcher, A.: Algebraic Topology. Cambridge University Press, Cambridge (2002)zbMATHGoogle Scholar
  26. 26.
    Horn, B.K., Schunck, B.G.: Determining optical flow. Artif. Intell. 17(1–3), 185–203 (1981)Google Scholar
  27. 27.
    Huang, J., Lee, A.B., Mumford, D.B.: Statistics of range images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 324–332 (2000)Google Scholar
  28. 28.
    Lee, A.B., Pedersen, K.S., Mumford, D.: The nonlinear statistics of high-contrast patches in natural images. Int. J. Comput. Vis. 54(1–3), 83–103 (2003)zbMATHGoogle Scholar
  29. 29.
    Lum, P., et al.: Extracting insights from the shape of complex data using topology. Sci. Rep. 3, 1236 (2013)Google Scholar
  30. 30.
    Mac Aodha, O., Humayun, A., Pollefeys, M., Brostow, G.J.: Learning a confidence measure for optical flow. IEEE Trans. Pattern Anal. Mach. Intell. 35(5), 1107–1120 (2013)Google Scholar
  31. 31.
  32. 32.
    Roosendaal, T.: Sintel. Blender Foundation, Durian Open Movie Project (2010). http://www.sintel.org/
  33. 33.
    Roth, S., Black, M.J.: On the spatial statistics of optical flow. Int. J. Comput. Vis. 74(1), 33–50 (2007)Google Scholar
  34. 34.
    de Silva, V., Ghrist, R.: Coordinate-free coverage in sensor networks with controlled boundaries via homology. Int. J. Robot. Res. 25(12), 1205–1222 (2006)zbMATHGoogle Scholar
  35. 35.
    Topaz, C.M., Ziegelmeier, L., Halverson, T.: Topological data analysis of biological aggregation models. PloS One 10(5), e0126383 (2015)Google Scholar
  36. 36.
    Xia, K., Wei, G.W.: Persistent homology analysis of protein structure, flexibility, and folding. Int. J. Numer. Methods Biomed. Eng. 30(8), 814–844 (2014)MathSciNetGoogle Scholar
  37. 37.
    Zomorodian, A., Carlsson, G.: Computing persistent homology. Discrete Comput. Geom. 33(2), 249–274 (2005)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Henry Adams
    • 1
  • Johnathan Bush
    • 1
  • Brittany Carr
    • 1
  • Lara Kassab
    • 1
  • Joshua Mirth
    • 1
  1. 1.Colorado State UniversityFort CollinsUSA

Personalised recommendations