Skip to main content

Embedded Voxel Colouring with Adaptive Threshold Selection Using Globally Minimal Surfaces

Abstract

Image-based 3D reconstruction remains a competitive field of research as state-of-the-art algorithms continue to improve. This paper presents a voxel-based algorithm that adapts the earliest space-carving methods and utilises a minimal surface technique to obtain a cleaner result. Embedded Voxel Colouring is built in two stages: (a) progressive voxel carving is used to build a volume of embedded surfaces and (b) the volume is processed to obtain a surface that maximises photo-consistency data in the volume. This algorithm combines the strengths of classical carving techniques with those of minimal surface approaches. We require only a single pass through the voxel volume, this significantly reduces computation time and is the key to the speed of our approach. We also specify three requirements for volumetric reconstruction: monotonic carving order, causality of carving and water-tightness. Experimental results are presented that demonstrate the strengths of this approach.

This is a preview of subscription content, access via your institution.

References

  1. Appleton, B. (2004). Globally minimal contours and surfaces for image segmentation. Ph.D. thesis, University of Queensland.

  2. Appleton, B., & Talbot, H. (2003). Globally optimal surfaces by continuous maximal flows. In C. Sun, H. Talbot, S. Ourselin, & T. Adriaansen (Eds.), Digital image computing: techniques and applications (Vol. 2, pp. 987–996). Sydney, Australia.

    Google Scholar 

  3. Appleton, B., & Talbot, H. (2006). Globally minimal surfaces by continuous maximal flows. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(1), 106–118.

    Article  Google Scholar 

  4. Beardsley, P., Torr, P., & Zisserman, A. (1996). 3D model acquisition from extended image sequences. In European conference on computer vision (pp. 683–695).

    Google Scholar 

  5. Bhotika, R., Fleet, D., & Kutulakos, K. (2002). A probabilistic theory of occupancy and emptiness. In European conference on computer vision (Vol. 3, pp. 112–132). Copenhagen, Denmark.

    Google Scholar 

  6. Broadhurst, A. (2001). A probabilistic framework for space carving. Ph.D. thesis, Dept. of Engineering, University of Cambridge.

  7. Broadhurst, A., Drummond, T., & Cipolla, R. (2001). A probabilistic framework for space carving. In IEEE international conference of computer vision (pp. 388–393). Vancouver, Canada.

    Google Scholar 

  8. Campbell, N., Vogiatzis, G., Hernandez, C., & Cipolla, R. (2008). Using multiple hypotheses to improve depth-maps for multi-view stereo. In Proceedings of European conference on computer vision (pp. 766–779). Marseille, France.

    Google Scholar 

  9. Carceroni, R., & Kutulakos, K. (2001). Multi-view scene capture by surfel sampling: From video streams to non-rigid 3D motion, shape and reflectance. In IEEE international conference on computer vision (pp. 60–67). Vancouver, Canada.

    Google Scholar 

  10. Caselles, V., Kimmel, R., Sapiro, G., & Sbert, C. (1997). Minimal surfaces based object segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(4), 394–398.

    MathSciNet  Article  Google Scholar 

  11. Cohen, L. D., & Kimmel, R. (1997). Global minimum for active contour models: A minimal path approach. International Journal of Computer Vision, 24(1), 57–78.

    Article  Google Scholar 

  12. Colosimo, A., Sarti, A., & Tubaro, S. (2001). Image-based object modeling: A multiresolution level-set approach. In IEEE international conference on image processing (pp. 181–184). Thessaloniki, Greece.

    Google Scholar 

  13. Culbertson, B., Malzbender, T., & Slabaugh, G. (1999). Generalized voxel coloring. In B. Triggs, A. Zisserman, & R. Szeliski (Eds.), Lecture notes in computer science: Vol. 1883. Vision algorithms: Theory and practice (pp. 100–115). Corfu, Greece. Berlin: Springer.

    Chapter  Google Scholar 

  14. Curless, B., & Levoy, M. (1996). A volumetric method for building complex models from range images. In SIGGRAPH, computer graphics proceedings (pp. 303–312). New Orleans, USA.

    Google Scholar 

  15. Dyer, C. R. (2001). Volumetric scene reconstruction from multiple views. In Foundations of image understanding (pp. 469–489). Boston: Kluwer. Chap. 16.

    Chapter  Google Scholar 

  16. Faugeras, O., & Keriven, R. (1998). Variational principles, surface evolution, PDE’s, level set methods and the stereo problem. IEEE Transactions on Image Processing, 7(3), 336–344. Special Issue on Partial Differential Equations and Geometry-Driven Diffusion in Image Processing and Analysis.

    MathSciNet  MATH  Article  Google Scholar 

  17. Freund, J. E. (1992). Mathematical statistics. Englewood Cliffs: Prentice Hall.

    MATH  Google Scholar 

  18. Fua, P. (1997). From multiple stereo views to multiple 3D surfaces. International Journal of Computer Vision, 24(1), 19–35.

    Article  Google Scholar 

  19. Fua, P., & Leclerc, Y. (1995). Object-centered surface reconstruction: Combining multi-image stereo and shading. International Journal of Computer Vision, 16(1), 35–56.

    Article  Google Scholar 

  20. Furukawa, Y., & Ponce, J. (2010). Accurate, dense, and robust multiview stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8), 1362–1376.

    Article  Google Scholar 

  21. Goldluecke, B., & Magnor, M. (2004). Space-time isosurface evolution for temporally coherent 3D reconstruction. In IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 350–355). Washington D.C., USA.

    Google Scholar 

  22. Graham, R. (1972). An efficient algorithm for determining the convex hull of a finite point set. Information Processing Letters, 1(4), 132–133.

    MATH  Article  Google Scholar 

  23. Kimura, M., Saito, H., & Kanade, T. (1999). 3D voxel construction based on epipolar geometry. In IEEE international conference on image processing (Vol. 3, pp. 135–139).

    Google Scholar 

  24. Koch, R. (1995). 3-D surface reconstruction from stereoscopic image sequences. In IEEE international conference on computer vision (pp. 109–114). Cambridge, MA, USA.

    Google Scholar 

  25. Kutulakos, K. N., & Seitz, S. M. (2000). A theory of shape by space carving. International Journal of Computer Vision, 38(3), 199–218.

    MATH  Article  Google Scholar 

  26. Laurentini, A. (1994). The visual hull concept for silhouette-based image understanding. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(2), 150–162.

    Article  Google Scholar 

  27. Leung, C. (2006). Efficient methods for 3D reconstruction from multiple images. Ph.D. thesis, University of Queensland.

  28. Leung, C., Appleton, B., & Sun, C. (2003). Embedded voxel colouring. In C. Sun, H. Talbot, S. Ourselin, & T. Adriaansen (Eds.), Digital image computing: techniques and applications (Vol. 2, pp. 623–632). Sydney, Australia.

    Google Scholar 

  29. Lhuillier, M., & Quan, L. (2005). A quasi-dense approach to surface reconstruction from uncalibrated images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(3), 418–433.

    Article  Google Scholar 

  30. Martin, W. N., & Aggarwal, J. K. (1983). Volumetric descriptions of objects from multiple views. IEEE Transactions on Pattern Analysis and Machine Intelligence, 5(2), 150–158.

    Article  Google Scholar 

  31. MeshLab (2010). http://meshlab.sourceforge.net/.

  32. Moezzi, S., Katkere, A., Kuramura, D. Y., & Jain, R. (1996). Reality modeling and visualization from multiple video sequences. IEEE Computer Graphics and Applications, 16(6), 58–63.

    Article  Google Scholar 

  33. Moezzi, S., Tai, L.-C., & Gerard, P. (1997). Virtual view generation for 3D digital video. IEEE Multimedia, 4(1), 18–26.

    Article  Google Scholar 

  34. Niem, W. (1994). Robust and fast modelling of 3D natural objects from multiple views. In SPIE proceedings image and video processing II (Vol. 2182, pp. 388–397). San Jose.

    Google Scholar 

  35. O’Rouke, J. (1998). Convex hulls in two dimensions. In Computational geometry in C (2nd ed.). Cambridge: Cambridge University Press. Chap. 3.

    Google Scholar 

  36. Pollefeys, M., Koch, R., Vergauwen, M., & Van Gool, L. (1999). Hand-held acquisition of 3D models with a video camera. In International conference on 3D digital imaging and modeling (pp. 14–23). Ottawa, Canada.

    Google Scholar 

  37. Pons, J.-P., Keriven, R., & Faugeras, O. D. (2005). Modelling dynamic scenes by registering multi-view image sequences. In Proceedings of computer vision and pattern recognition (pp. 822–827). San Diego, CA.

    Google Scholar 

  38. Prock, A., & Dyer, C. R. (1998). Towards real-time voxel coloring. In DARPA image understanding workshop (pp. 315–321).

    Google Scholar 

  39. Roy, S., & Cox, I. J. (1998). A maximum-flow formulation of the N-camera correspondence problem. In IEEE international conference on computer vision (pp. 492–499). Bombay, India.

    Google Scholar 

  40. Seitz, S. M., & Dyer, C. R. (1997). Photorealistic scene reconstruction by voxel coloring. In IEEE conference on computer vision and pattern recognition (pp. 1067–1073). Puerto Rico.

    Google Scholar 

  41. Shade, J., Gortler, S., He, L.-W., & Szeliski, R. (1998). Layered depth images. In Proceedings of the 25th annual conference on computer graphics and interactive techniques, SIGGRAPH ’98 (pp. 231–242).

    Chapter  Google Scholar 

  42. Slabaugh, G. G., Culbertson, W. B., Malzbender, T., & Schafer, R. (2001). A survey of volumetric scene reconstruction methods from photographs. In K. Mueller & A. Kaufman (Eds.), Springer computer science. Volume graphics 2001, joint IEEE TCVG and Eurographics workshop (pp. 81–100). Stony Brook, New York, USA.

    Google Scholar 

  43. Slabaugh, G. G., Schafer, R. W., & Hans, M. C. (2002). Multi-resolution space carving using level sets methods. In IEEE international conference on image processing (Vol. 2, pp. 545–548). Rochester, USA.

    Google Scholar 

  44. Slabaugh, G. G., Culbertson, W. B., Malzbender, T., Stevens, M. R., & Schafer, R. W. (2004). Methods for volumetric reconstruction of visual scenes. International Journal of Computer Vision, 57(3), 179–199.

    Article  Google Scholar 

  45. Snow, D., Viola, P., & Zabih, R. (2000). Exact voxel occupancy with graph cuts. In IEEE conference on computer vision and pattern recognition (pp. 345–352).

    Google Scholar 

  46. Stevens, M., Culbertson, B., & Malzbender, T. (2002). A histogram-based color consistency test for voxel coloring. In International conference on pattern recognition (Vol. 4, pp. 118–121). Quebec, Canada.

    Google Scholar 

  47. Sun, C. (2002). Fast stereo matching using rectangular subregioning and 3D maximum-surface techniques. International Journal of Computer Vision, 47(1/2/3), 99–117.

    MATH  Article  Google Scholar 

  48. Vedula, S., Baker, S., Seitz, S., & Kanade, T. (2000). Shape and motion carving in 6D. In IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 592–598). Hilton Head Island, SC, USA.

    Google Scholar 

  49. Vu, H. H., Keriven, R., Labatut, P., & Pons, J.-P. (2009). Towards high-resolution large-scale multi-view stereo. In IEEE conference on computer vision and pattern recognition (pp. 1430–1437). Miami, FL.

    Google Scholar 

  50. Weghorst, H., Hooper, G., & Greenberg, D. P. (1984). Improved computational methods for ray tracing. ACM Transactions on Graphics, 3(1), 52–69.

    Article  Google Scholar 

  51. Yezzi, A., & Soatto, S. (2003). Stereoscopic segmentation. International Journal of Computer Vision, 53(1), 31–43.

    MathSciNet  Article  Google Scholar 

  52. Yezzi, A., Slabaugh, G., Broadhurst, A., Cipolla, R., & Schafer, R. (2002). A surface evolution approach to probabilistic space carving. In International symposium on 3D processing, visualization, and transmission (pp. 612–621). Padova, Italy.

    Google Scholar 

  53. Zeng, G., Paris, S., Quan, L., & Sillion, F. (2007). Accurate and scalable surface representation and reconstruction from images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(1), 141–158.

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Changming Sun.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Leung, C., Appleton, B., Buckley, M. et al. Embedded Voxel Colouring with Adaptive Threshold Selection Using Globally Minimal Surfaces. Int J Comput Vis 99, 215–231 (2012). https://doi.org/10.1007/s11263-012-0525-8

Download citation

Keywords

  • Volumetric 3D reconstruction
  • Embedded voxel colouring
  • Globally minimal surfaces
  • Monotonic carving order
  • Causality of carving
  • Water-tightness