We present a scalable multi-view stereo method able to reconstruct accurate 3D models from hundreds of high-resolution input images. Local fusion of disparity maps obtained with semi-global matching enables the reconstruction of large scenes that do not fit into main memory. Since disparity maps may vary widely in quality and resolution, careful modeling of the 3D errors is crucial. We derive a sound stereo error model based on disparity uncertainty, which can vary spatially from tenths to several pixels. We introduce a feature based on total variation that allows pixel-wise classification of disparities into different error classes. For each class, we learn a disparity error distribution from ground-truth data using expectation maximization. We present a novel method for stochastic fusion of data with varying quality by adapting a multi-resolution volumetric fusion process that uses our error classes as a prior and models surface probabilities via an octree of voxels. Conflicts during surface extraction are resolved using visibility constraints and preference for voxels at higher resolutions. Experimental results on several challenging large-scale datasets demonstrate that our method yields improved performance both qualitatively and quantitatively.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Bailer, C., Finckh, M., & Lensch, H. (2012). Scale robust multi view stereo. In ECCV.
Bao, S., Chandraker, M., Lin, Y., Savarese, S. (2013). Dense object reconstruction with semantic priors. In CVPR.
Bodenmüller, T. (2009). Streaming surface reconstruction from real time 3D measurements. Ph.D. thesis, Technical University Munich.
Curless, B., & Levoy, M. (1996). A volumetric method for building complex models from range images. In SIGGRAPH.
Frahm, J.M., Georgel, P., Gallup, D., Johnson, T., Raguram, R., Wu, C., Jen, Y. H., Dunn, E., Clipp, B., Lazebnik, S., & Pollefeys, M. (2010). Building Rome on a cloudless day. In ECCV.
Fuhrmann, S., & Goesele, M. (2011). Fusion of depth maps with multiple scales. In SIGGRAPH Asia.
Fuhrmann, S., & Goesele, M. (2014). Floating scale surface reconstruction. In SIGGRAPH.
Furukawa, R., Itano, T., Morisaka, A., & Kawasaki, H. (2007). Improved space carving method for merging and interpolating multiple range images using information of light sources of active stereo. In ACCV.
Furukawa, Y., & Ponce, J. (2010). Accurate, dense, and robust multiview stereopsis. PAMI, 32, 1362–1376.
Goesele, M., Curless, B., & Seitz, S. (2006). Multi-view stereo revisited. In CVPR.
Goesele, M., Snavely, N., Curless, B., Hoppe, H., & Seitz, S. (2007). Multi-view stereo for community photo collections. In ICCV.
Häne, C., Zach, C., Cohen, A., Angst, R., & Pollefeys, M. (2013). Joint 3D scene reconstruction and class segmentation. In CVPR.
Hernández, C., Vogiatzis, G., & Cipolla, R. (2007). Probabilistic visibility for multi-view stereo. In CVPR.
Hirschmüller, H. (2008). Stereo processing by semi-global matching and mutual information. PAMI, 30, 328–341.
Hirschmüller, H., & Scharstein, D. (2009). Evaluation of stereo matching costs on images with radiometric differences. PAMI, 31, 1582–1599.
Hu, X., Mordohai, P. (2012). Least commitment, viewpoint-based, multi-view stereo. In 3DIMPVT.
Kazhdan, M., Bolitho, M., Hoppe, H. (2006). Poisson surface reconstruction. In Eurographics.
Kazhdan, M., Klein, A., Dalal, K., Hoppe, H. (2007). Unconstrained isosurface extraction on arbitrary octrees. In Eurographics.
Kolev, K., Klodt, M., Brox, T., & Cremers, D. (2009). Continuous global optimization in multiview 3D reconstruction. IJCV, 84, 80–96.
Kuhn, A. (2014). Scalable 3D surface reconstruction by local stochastic fusion of disparity maps. Ph.D. thesis, University of the Bundeswehr.
Kuhn, A., Hirschmüller, H., & Mayer, H. (2013). Multi-resolution range data fusion for multi-view stereo reconstruction. In GCPR.
Kuhn, A., & Mayer, H. (2015). Incremental division of very large point clouds for scalable 3D surface reconstruction. In ICCV Workshop (ICCVW).
Kuhn, A., Mayer, H., Hirschmüller, H., & Scharstein, D. (2014). A TV prior for high-quality local multi-view stereo reconstruction. In 3DV.
Mayer, H., Bartelsen, J., Hirschmüller, H., & Kuhn, A. (2011). Dense 3D reconstruction from wide baseline image sets. In 15th International Workshop on Theoretical Foundations of Computer Vision.
Merrell, P., Akbarzadeh, A., Wang, L., Mordohai, P., Frahm, J. M., Yang, R., Nistér, D., Pollefeys, M. (2007). Real-time visibility-based fusion of depth maps. In CVPR.
Molton, N., & Brady, M. (2000). Practical structure and motion from stereo when motion is unconstrained. IJCV, 39(1), 5–23.
Mücke, P., Klowsky, R., & Goesele, M. (2011). Surface reconstruction from multi-resolution sample points. In VMV.
Newcombe, R., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A., Kohli, P., Shotton, J., Hodges, S., & Fitzgibbon, A. (2011). KinectFusion: Real-time dense surface mapping and tracking. In ISMAR.
Ochs, P., Dosovitskiy, A., Brox, T., & Pock, T. (2013). An iterated L1 algorithm for non-smooth non-convex optimization in computer vision. In CVPR.
Pathak, K., Birk, A., & Schwertfeger, S. (2007). 3D forward sensor modeling and application to occupancy grid based sensor fusion. In IROS.
Rudin, L., Osher, S., & Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica D, 60(1), 259–268.
Sagawa, R., Nishino, K., & Ikeuchi, K. (2005). Adaptively merging large-scale range data with reflectance properties. PAMI, 27(3), 392–405.
Scharstein, D., Hirschmüller, H., Kitajima, Y., Krathwohl, G., Nesic, N., Wang, X., & Westling, P. (2014). High-resolution stereo datasets with subpixel-accurate ground truth. In GCPR.
Scharstein, D., & Pal, C. (2007). Learning conditional random fields in stereo. In CVPR.
Schroers, C., Zimmer, H., Valgaerts, L., Bruhn, A., Demetz, O., & Weickert, J. (2012). Anisotropic range image integration. In DAGM.
Seitz, S., Curless, B., Diebel, J., Scharstein, D.,&Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In CVPR.
Sinha, S., Scharstein, D., & Szeliski, R. (2014). Efficient high-resolution stereo matching using local plane sweeps. In CVPR.
Steinbrücker, F., Kerl, C., Sturm, J., & Cremers, D. (2013). Large-scale multi-resolution surface reconstruction from RGB-D sequences. In ICCV.
Strecha, C., von Hansen, W., Van Gool, L., Fua, P., Thoennessen, U. (2008). On benchmarking camera calibration and multi-view stereo for high resolution imagery. In CVPR.
Thrun, S. (2003). Learning occupancy grid maps with forward sensor models. Autonomous Robots, 15, 111–127.
Vogiatzis, G., & Hernández, C. (2011). Video-based, real-time multi-view stereo. Image and Vision Computing, 29, 434–441.
Vu, H. H., Labatut, P., Pons, J. P., & Keriven, R. (2012). High accuracy and visibility-consistent dense multiview stereo. PAMI, 34, 889–901.
Wei, J., Resch, B., Lensch, H. (2014). Multi-view depth map estimation with cross-view consistency. In BMVC.
Wheeler, M., Sato, Y., Ikeuchi, K. (1998). Consensus surfaces for modeling 3D objects from multiple range images. In ICCV.
Woodford, O., & Vogiatzis, G. (2012). A generative model for online depth fusion. In ECCV.
Wu, C. (2013). Towards linear-time incremental structure from motion. In 3DV.
Wu, C., Agarwal, S., Curless, B., & Seitz, S. (2011). Multicore bundle adjustment. In CVPR.
Xiong, Y., & Matthies, L. (1997). Error analysis of a real-time stereo system. In CVPR.
Zach, C. (2008). Fast and high quality fusion of depth maps. In 3DPVT.
Zach, C., Pock, T., & Bischof, H. (2007). A globally optimal algorithm for robust TV-L1 range image integration. In ICCV.
Communicated by Lourdes Agapito, Hiroshi Kawasaki, Katsushi Ikeuchi and Martial Hebert.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Kuhn, A., Hirschmüller, H., Scharstein, D. et al. A TV Prior for High-Quality Scalable Multi-View Stereo Reconstruction. Int J Comput Vis 124, 2–17 (2017). https://doi.org/10.1007/s11263-016-0946-x
- Multi-View Stereo
- 3D Modeling
- Scalable 3D Surface Reconstruction