Advertisement

Spatial and Temporal Interpolation of Multi-view Image Sequences

  • Tobias GurdanEmail author
  • Martin R. Oswald
  • Daniel Gurdan
  • Daniel Cremers
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8753)

Abstract

We propose a simple and effective framework for multi-view image sequence interpolation in space and time. For spatial view point interpolation we present a robust feature-based matching algorithm that allows for wide-baseline camera configurations. To this end, we introduce two novel filtering approaches for outlier elimination and a robust approach for match extrapolations at the image boundaries. For small-baseline and temporal interpolations we rely on an established optical flow based approach. We perform a quantitative and qualitative evaluation of our framework and present applications and results. Our method has a low runtime and results can compete with state-of-the-art methods.

Keywords

Optical Flow Image Boundary Image Interpolation Temporal Interpolation Epipolar Constraint 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Ballan, L., Brostow, G.J., Puwein, J., Pollefeys, M.: Unstructured video-based rendering: interactive exploration of casually captured videos. ACM Trans. Graph. 29(4) (2010). http://dblp.uni-trier.de/db/journals/tog/tog29.html#BallanBPP10
  2. 2.
    Chen, K., Lorenz, D.A.: Image sequence interpolation based on optical flow, segmentation, and optimal control. IEEE Trans. Image Process. 21(3), 1020–1030 (2012). http://dblp.uni-trier.de/db/journals/tip/tip21.html#ChenL12 MathSciNetCrossRefGoogle Scholar
  3. 3.
    Debevec, P.: The campanile movie. In: SIGGRAPH 97 Electronic Theater (1997). http://www.debevec.org/Campanile/ (visited: May 2014)
  4. 4.
    Fehn, C.: Depth-Image-Based Rendering (DIBR), compression and transmission for a new approach on 3D-TV. In: Proceedings of SPIE Stereoscopic Displays and Virtual Reality Systems XI, pp. 93–104 (2004)Google Scholar
  5. 5.
    Fischler, M., Bolles, R.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Fragneto, P., Fusiello, A., Rossi, B., Magri, L., Ruffini, M.: Uncalibrated view synthesis with homography interpolation. In: 3DIMPVT, pp. 270–277. IEEE (2012). http://dblp.uni-trier.de/db/conf/3dim/3dimpvt2012.html#FragnetoFRMR12
  7. 7.
    Germann, M., Hornung, A., Keiser, R., Ziegler, R., Würmlin, S., Gross, M.: Articulated billboards for video-based rendering. Comput. Graph. Forum (Proc. Eurographics) 29(2), 585–594 (2010)CrossRefGoogle Scholar
  8. 8.
    Goesele, M., Ackermann, J., Fuhrmann, S., Haubold, C., Klowsky, R., Steedly, D., Szeliski, R.: Ambient point clouds for view interpolation. ACM Trans. Graph. 29(4), 95:1–95:6 (2010). http://doi.acm.org/10.1145/1778765.1778832 CrossRefGoogle Scholar
  9. 9.
    Hasler, N., Rosenhahn, B., Thormählen, T., Wand, M., Gall, J., Seidel, H.P.: Markerless motion capture with unsynchronized moving cameras. In: CVPR, pp. 224–231 (2009)Google Scholar
  10. 10.
    Inamoto, N., Saito, H.: Free viewpoint video synthesis and presentation from multiple sporting videos. In: ICME, pp. 322–325. IEEE (2005). http://dblp.uni-trier.de/db/conf/icmcs/icme2005.html#InamotoS05
  11. 11.
    Lipski, C.: Virtual video camera: a system for free viewpoint video of arbitrary dynamic scenes. Ph.D. thesis, TU Braunschweig, June 2013Google Scholar
  12. 12.
    Lipski, C., Linz, C., Berger, K., Magnor, M.A.: Virtual video camera: image-based viewpoint navigation through space and time. In: SIGGRAPH Posters. ACM (2009). http://dblp.uni-trier.de/db/conf/siggraph/siggraph2009posters.html#LipskiLBM09
  13. 13.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 20, 91–110 (2003)Google Scholar
  14. 14.
    Mahajan, D., Huang, F.C., Matusik, W., Ramamoorthi, R., Belhumeur, P.N.: Moving gradients: a path-based method for plausible image interpolation. ACM Trans. Graph. 28(3), 42:1–42:11 (2009). doi: 10.1145/1531326.1531348 CrossRefGoogle Scholar
  15. 15.
    Morel, J.M., Yu, G.: Asift: a new framework for fully affine invariant image comparison. SIAM J. Imaging Sci. 2(2), 438–469 (2009). http://dblp.uni-trier.de/db/journals/siamis/siamis2.html#MorelY09 MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Muja, M., Lowe, D.G.: Fast approximate nearest neighbors with automatic algorithm configuration. In: Ranchordas, A., Arajo, H. (eds.) VISAPP (1), pp. 331–340. INSTICC Press (2009). http://dblp.uni-trier.de/db/conf/visapp/visapp2009-1.html#MujaL09
  17. 17.
    Replay Technologies Inc.: freeD\(^{\rm {TM}}\) technology (2013). http://replay-technologies.com/ (visited: May 2014)
  18. 18.
    Scharstein, D., Pal, C.: Learning conditional random fields for stereo. In: CVPR (2007)Google Scholar
  19. 19.
    Seitz, S.M., Dyer, C.R.: Physically-valid view synthesis by image interpolation. In: Proceedings of the IEEE Workshop on Representations of Visual Scenes, pp. 18–25 (1995)Google Scholar
  20. 20.
    Seitz, S.M., Dyer, C.R.: View morphing. In: SIGGRAPH, pp. 21–30 (1996). http://dblp.uni-trier.de/db/conf/siggraph/siggraph1996.html#SeitzD96
  21. 21.
    Snavely, N., Garg, R., Seitz, S.M., Szeliski, R.: Finding paths through the world’s photos. ACM Trans. Graph. (Proceedings of SIGGRAPH 2008) 27(3), 11–21 (2008)Google Scholar
  22. 22.
    Strecha, C., Tuytelaars, T., Gool, L.J.V.: Dense matching of multiple wide-baseline views. In: ICCV, pp. 1194–1201 (2003)Google Scholar
  23. 23.
    Vedula, S., Baker, S., Kanade, T.: Image-based spatio-temporal modeling and view interpolation of dynamic events. ACM Trans. Graph. 24(2), 240–261 (2005)CrossRefGoogle Scholar
  24. 24.
    Vlad, A.: Image morphing techniques. JIDEG 5(1) (2010). http://www.sorging.ro/ro/member/serveFile/format/pdf/slug/image-morphing-techniques
  25. 25.
    Warn, S., Apon, A., Cothren, J.: Accelerating sift on hybrid clusters. In: Proceedings of the ACM SIGSPATIAL Second International Workshop on High Performance and Distributed Geographic Information Systems, HPDGIS ’11, pp. 2–9. ACM, New York (2011). http://doi.acm.org/10.1145/2070770.2070771
  26. 26.
    Werlberger, M., Pock, T., Unger, M., Bischof, H.: Optical flow guided tv-l1 video interpolation and restoration. In: Energy Minimization Methods in Computer Vision and Pattern Recognition (2011)Google Scholar
  27. 27.
    Wolberg, G.: Image morphing: a survey. Vis. Comput. 14, 360–372 (1998). http://ci.nii.ac.jp/naid/80010827845/en/ CrossRefGoogle Scholar
  28. 28.
    Zhang, Z., Deriche, R., Faugeras, O.D., Luong, Q.T.: A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artif. Intell. 78(1–2), 87–119 (1995). http://dblp.uni-trier.de/db/journals/ai/ai78.html#ZhangDFL95 CrossRefGoogle Scholar
  29. 29.
    Zitnick, C.L., Kang, S.B., Uyttendaele, M., Winder, S.A.J., Szeliski, R.: High-quality video view interpolation using a layered representation. ACM Trans. Graph. 23(3), 600–608 (2004). http://dblp.uni-trier.de/db/journals/tog/tog23.html#ZitnickKUWS04 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Tobias Gurdan
    • 1
    • 2
    Email author
  • Martin R. Oswald
    • 1
  • Daniel Gurdan
    • 2
  • Daniel Cremers
    • 1
  1. 1.Department of Computer ScienceTechnische Universität MünchenMunichGermany
  2. 2.Ascending Technologies GmbHKrailingGermany

Personalised recommendations