Multi-camera Systems for 3D Video Production

  • Takashi Matsuyama
  • Shohei Nobuhara
  • Takeshi Takai
  • Tony Tung


Drastic advances of digital technologies and the Internet in this decade have made digital (still and video) cameras ubiquitous in everyday life. Moreover, computer vision technologies such as automatic focusing on human faces and image stabilization against hand vibrations have been implemented in modern cameras. This chapter first discusses the design factors of multi-camera systems for 3D video production: camera arrangement, lens and depth-of-focus, shutter speed, lighting, and background. Then three practical studio implementations at Kyoto University are introduced to demonstrate that high fidelity 3D video can be produced with modern off-the-shelf imaging devices. The latter half of the chapter discusses practical geometric and photometric calibration procedures with their quantitative performance evaluation results in the Kyoto University 3D video studios. The imaging devices and their calibration procedures introduced in this chapter can easily be implemented to start research and development of 3D video.


Camera Calibration Bundle Adjustment Reprojection Error Camera Coordinate System Active Camera 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Batlle, J., Mouaddib, E., Salvi, J.: Recent progress in coded structured light as a technique to solve the correspondence problem: a survey. Pattern Recognit. 31(7), 963–982 (1998) CrossRefGoogle Scholar
  2. 2.
    Bayer, B.E.: US Patent 3971065: Color imaging array (1976) Google Scholar
  3. 3.
    Bouguet, J.-Y.: Camera Calibration Toolbox for Matlab.
  4. 4.
    Bradski, G.: The OpenCV Library (2000).
  5. 5.
    Virtualizing Engine. Private communication with Profs. Takeo Kanade and Yaser Sheikh, Robotics Institute, Carnegie Mellon University, PA (2011) Google Scholar
  6. 6.
    Fitzgibbon, A.W., Zisserman, A.: Automatic camera recovery for closed or open image sequences. In: Proc. of European Conference on Computer Vision, pp. 311–326 (1998) Google Scholar
  7. 7.
    Gevers, T., Stokman, H.M.G., van de Weijer, J.: Colour constancy from hyper-spectral data. In: Proc. of British Machine Vision Conference (2000) Google Scholar
  8. 8.
    Goldlüecke, B., Magnor, M.: Joint 3D-reconstruction and background separation in multiple views using graph cuts. In: Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 683–688 (2003) Google Scholar
  9. 9.
    Guillemaut, J.Y., Hilton, A., Starck, J., Kilner, J., Grau, O.: A Bayesian framework for simultaneous matting and 3D reconstruction. In: Proc. of International Conference on 3-D Digital Imaging and Modeling, pp. 167–176 (2007) CrossRefGoogle Scholar
  10. 10.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000) MATHGoogle Scholar
  11. 11.
    Hernandez, C., Schmitt, F., Cipolla, R.: Silhouette coherence for camera calibration under circular motion. IEEE Trans. Pattern Anal. Mach. Intell. 29(2), 343–349 (2007) CrossRefGoogle Scholar
  12. 12.
    ISO/TR 16066: Standard Object Colour Spectra Database for Colour Reproduction Evaluation (SOCS) (2003) Google Scholar
  13. 13.
    Ivanov, Y., Bobick, A., Liu, J.: Fast lighting independent background subtraction. Int. J. Comput. Vis. 37(2), 199–207 (2000) MATHCrossRefGoogle Scholar
  14. 14.
    Kim, S.J., Pollefeys, M.: Robust radiometric calibration and vignetting correction. IEEE Trans. Pattern Anal. Mach. Intell. 30(4), 562–576 (2008) CrossRefGoogle Scholar
  15. 15.
    Levoy, M., Hanrahan, P.: Light field rendering. In: Proc. of ACM SIGGRAPH, pp. 31–42 (1996) Google Scholar
  16. 16.
    Matsuyama, T., Hiura, S., Wada, T., Murase, K., Toshioka, A.: Dynamic memory: architecture for real time integration of visual perception, camera action, and network communication. In: Proc. of IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 728–735 (2000) Google Scholar
  17. 17.
    Matsuyama, T., Ukita, N.: Real-time multitarget tracking by a cooperative distributed vision system. Proc. IEEE 90(7), 1136–1150 (2002) CrossRefGoogle Scholar
  18. 18.
    Miyake, Y., Yokoyama, Y., Tsumura, N., Haneishi, H., Miyata, K., Hayashi, J.: Development of multiband color imaging systems for recordings of art paintings. In: Proc. of SPIE, pp. 218–225 (1998) Google Scholar
  19. 19.
    Morimoto, T., Mihashi, T., Ikeuchi, K.: Color restoration method based on spectral information using normalized cut. Int. J. Autom. Comput. 5, 226–233 (2008) CrossRefGoogle Scholar
  20. 20.
    Nister, D.: An efficient solution to the five-point relative pose problem. IEEE Trans. Pattern Anal. Mach. Intell. 26(6), 756–770 (2004) CrossRefGoogle Scholar
  21. 21.
    Nobuhara, S., Kimura, Y., Matsuyama, T.: Object-oriented color calibration of multi-viewpoint cameras in sparse and convergent arrangement. IPSJ Trans. Comput. Vis. Appl. 2, 132–144 (2010) Google Scholar
  22. 22.
    Nobuhara, S., Tsuda, Y., Ohama, I., Matsuyama, T.: Multi-viewpoint silhouette extraction with 3D context-aware error detection, correction, and shadow suppression. IPSJ Trans. Comput. Vis. Appl. 1, 242–259 (2009) Google Scholar
  23. 23.
    Okutomi, M., Kanade, T.: A multiple-baseline stereo. IEEE Trans. Pattern Anal. Mach. Intell. 15(1), 353–363 (1993) CrossRefGoogle Scholar
  24. 24.
    Pascale, D.: RGB coordinates of the ColorChecker (2006).
  25. 25.
    PMDTechnologies GmbH: CamCube3.0 (2010) Google Scholar
  26. 26.
    Salvi, J., Pagès, J., Batlle, J.: Pattern codification strategies in structured light systems. Pattern Recognit. 37(4), 827–849 (2004) MATHCrossRefGoogle Scholar
  27. 27.
    Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 7–42 (2002) MATHCrossRefGoogle Scholar
  28. 28.
    Starck, J., Maki, A., Nobuhara, S., Hilton, A., Matsuyama, T.: The multiple-camera 3-d production studio. IEEE Trans. Circuits Syst. Video Technol. 19(6), 856–869 (2009) CrossRefGoogle Scholar
  29. 29.
    Toyoura, M., Iiyama, M., Kakusho, K., Minoh, M.: Silhouette extraction with random pattern backgrounds for the volume intersection method. In: Proc. of International Conference on 3-D Digital Imaging and Modeling, pp. 225–232 (2007) CrossRefGoogle Scholar
  30. 30.
    Wada, T., Matsuyama, T.: Appearance sphere: Background model for pan-tilt-zoom camera. In: Proc. of International Conference on Pattern Recognition, pp. A-718–A-722 (1996) Google Scholar
  31. 31.
    Yamaguchi, T., Wilburn, B., Ofek, E.: Video-based modeling of dynamic hair. In: Proc. of PSIVT, pp. 585–596 (2009) Google Scholar
  32. 32.
    Zeng, G., Quan, L.: Silhouette extraction from multiple images of an unknown background. In: Proc. of Asian Conference on Computer Vision, pp. 628–633 (2004) Google Scholar
  33. 33.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000) CrossRefGoogle Scholar
  34. 34.
    Zheng, Y., Yu, J., Kang, S., Lin, S., Kambhamettu, C.: Single-image vignetting correction using radial gradient symmetry. In: Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008) Google Scholar

Copyright information

© Springer-Verlag London 2012

Authors and Affiliations

  • Takashi Matsuyama
    • 1
  • Shohei Nobuhara
    • 1
  • Takeshi Takai
    • 1
  • Tony Tung
    • 1
  1. 1.Graduate School of InformaticsKyoto UniversitySakyoJapan

Personalised recommendations