Abstract
We propose a simple and effective framework for multi-view image sequence interpolation in space and time. For spatial view point interpolation we present a robust feature-based matching algorithm that allows for wide-baseline camera configurations. To this end, we introduce two novel filtering approaches for outlier elimination and a robust approach for match extrapolations at the image boundaries. For small-baseline and temporal interpolations we rely on an established optical flow based approach. We perform a quantitative and qualitative evaluation of our framework and present applications and results. Our method has a low runtime and results can compete with state-of-the-art methods.
This work was supported by the ERC Starting Grant “Convex Vision”.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Ballan, L., Brostow, G.J., Puwein, J., Pollefeys, M.: Unstructured video-based rendering: interactive exploration of casually captured videos. ACM Trans. Graph. 29(4) (2010). http://dblp.uni-trier.de/db/journals/tog/tog29.html#BallanBPP10
Chen, K., Lorenz, D.A.: Image sequence interpolation based on optical flow, segmentation, and optimal control. IEEE Trans. Image Process. 21(3), 1020–1030 (2012). http://dblp.uni-trier.de/db/journals/tip/tip21.html#ChenL12
Debevec, P.: The campanile movie. In: SIGGRAPH 97 Electronic Theater (1997). http://www.debevec.org/Campanile/ (visited: May 2014)
Fehn, C.: Depth-Image-Based Rendering (DIBR), compression and transmission for a new approach on 3D-TV. In: Proceedings of SPIE Stereoscopic Displays and Virtual Reality Systems XI, pp. 93–104 (2004)
Fischler, M., Bolles, R.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)
Fragneto, P., Fusiello, A., Rossi, B., Magri, L., Ruffini, M.: Uncalibrated view synthesis with homography interpolation. In: 3DIMPVT, pp. 270–277. IEEE (2012). http://dblp.uni-trier.de/db/conf/3dim/3dimpvt2012.html#FragnetoFRMR12
Germann, M., Hornung, A., Keiser, R., Ziegler, R., Würmlin, S., Gross, M.: Articulated billboards for video-based rendering. Comput. Graph. Forum (Proc. Eurographics) 29(2), 585–594 (2010)
Goesele, M., Ackermann, J., Fuhrmann, S., Haubold, C., Klowsky, R., Steedly, D., Szeliski, R.: Ambient point clouds for view interpolation. ACM Trans. Graph. 29(4), 95:1–95:6 (2010). http://doi.acm.org/10.1145/1778765.1778832
Hasler, N., Rosenhahn, B., Thormählen, T., Wand, M., Gall, J., Seidel, H.P.: Markerless motion capture with unsynchronized moving cameras. In: CVPR, pp. 224–231 (2009)
Inamoto, N., Saito, H.: Free viewpoint video synthesis and presentation from multiple sporting videos. In: ICME, pp. 322–325. IEEE (2005). http://dblp.uni-trier.de/db/conf/icmcs/icme2005.html#InamotoS05
Lipski, C.: Virtual video camera: a system for free viewpoint video of arbitrary dynamic scenes. Ph.D. thesis, TU Braunschweig, June 2013
Lipski, C., Linz, C., Berger, K., Magnor, M.A.: Virtual video camera: image-based viewpoint navigation through space and time. In: SIGGRAPH Posters. ACM (2009). http://dblp.uni-trier.de/db/conf/siggraph/siggraph2009posters.html#LipskiLBM09
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 20, 91–110 (2003)
Mahajan, D., Huang, F.C., Matusik, W., Ramamoorthi, R., Belhumeur, P.N.: Moving gradients: a path-based method for plausible image interpolation. ACM Trans. Graph. 28(3), 42:1–42:11 (2009). doi:10.1145/1531326.1531348
Morel, J.M., Yu, G.: Asift: a new framework for fully affine invariant image comparison. SIAM J. Imaging Sci. 2(2), 438–469 (2009). http://dblp.uni-trier.de/db/journals/siamis/siamis2.html#MorelY09
Muja, M., Lowe, D.G.: Fast approximate nearest neighbors with automatic algorithm configuration. In: Ranchordas, A., Arajo, H. (eds.) VISAPP (1), pp. 331–340. INSTICC Press (2009). http://dblp.uni-trier.de/db/conf/visapp/visapp2009-1.html#MujaL09
Replay Technologies Inc.: freeD\(^{\rm {TM}}\) technology (2013). http://replay-technologies.com/ (visited: May 2014)
Scharstein, D., Pal, C.: Learning conditional random fields for stereo. In: CVPR (2007)
Seitz, S.M., Dyer, C.R.: Physically-valid view synthesis by image interpolation. In: Proceedings of the IEEE Workshop on Representations of Visual Scenes, pp. 18–25 (1995)
Seitz, S.M., Dyer, C.R.: View morphing. In: SIGGRAPH, pp. 21–30 (1996). http://dblp.uni-trier.de/db/conf/siggraph/siggraph1996.html#SeitzD96
Snavely, N., Garg, R., Seitz, S.M., Szeliski, R.: Finding paths through the world’s photos. ACM Trans. Graph. (Proceedings of SIGGRAPH 2008) 27(3), 11–21 (2008)
Strecha, C., Tuytelaars, T., Gool, L.J.V.: Dense matching of multiple wide-baseline views. In: ICCV, pp. 1194–1201 (2003)
Vedula, S., Baker, S., Kanade, T.: Image-based spatio-temporal modeling and view interpolation of dynamic events. ACM Trans. Graph. 24(2), 240–261 (2005)
Vlad, A.: Image morphing techniques. JIDEG 5(1) (2010). http://www.sorging.ro/ro/member/serveFile/format/pdf/slug/image-morphing-techniques
Warn, S., Apon, A., Cothren, J.: Accelerating sift on hybrid clusters. In: Proceedings of the ACM SIGSPATIAL Second International Workshop on High Performance and Distributed Geographic Information Systems, HPDGIS ’11, pp. 2–9. ACM, New York (2011). http://doi.acm.org/10.1145/2070770.2070771
Werlberger, M., Pock, T., Unger, M., Bischof, H.: Optical flow guided tv-l1 video interpolation and restoration. In: Energy Minimization Methods in Computer Vision and Pattern Recognition (2011)
Wolberg, G.: Image morphing: a survey. Vis. Comput. 14, 360–372 (1998). http://ci.nii.ac.jp/naid/80010827845/en/
Zhang, Z., Deriche, R., Faugeras, O.D., Luong, Q.T.: A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artif. Intell. 78(1–2), 87–119 (1995). http://dblp.uni-trier.de/db/journals/ai/ai78.html#ZhangDFL95
Zitnick, C.L., Kang, S.B., Uyttendaele, M., Winder, S.A.J., Szeliski, R.: High-quality video view interpolation using a layered representation. ACM Trans. Graph. 23(3), 600–608 (2004). http://dblp.uni-trier.de/db/journals/tog/tog23.html#ZitnickKUWS04
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Gurdan, T., Oswald, M.R., Gurdan, D., Cremers, D. (2014). Spatial and Temporal Interpolation of Multi-view Image Sequences. In: Jiang, X., Hornegger, J., Koch, R. (eds) Pattern Recognition. GCPR 2014. Lecture Notes in Computer Science(), vol 8753. Springer, Cham. https://doi.org/10.1007/978-3-319-11752-2_24
Download citation
DOI: https://doi.org/10.1007/978-3-319-11752-2_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11751-5
Online ISBN: 978-3-319-11752-2
eBook Packages: Computer ScienceComputer Science (R0)