Visualization of 3D Video

  • Takashi Matsuyama
  • Shohei Nobuhara
  • Takeshi Takai
  • Tony Tung


Visualization is one of the most standard applications of 3D video. Its essential functionality includes interactive free-viewpoint and 3D (pop-up) visualization of the captured scene as is. Following an ordinary 3D video visualization system, this chapter presents a novel free-viewpoint visualization method for 3D video stream of a single human in action. The novelty rests in that the 3D video is visualized from the performer’s viewpoint. Ordinary free-viewpoint visualization methods render the object action viewed from the outside of the scene. We may call it an objective, or third-person, view of the object action. With 3D video data, moreover, we can render a subjective, or first-person, view of the object action, where the object action is visualized as if it were captured from a head-mounted camera. Such subjective visualization is very useful to understand where to look when performing juggling or traditional dances; in MAIKO dances, for example, eye motions are very important to express mental feelings.


Virtual Camera Frontal Face Image Source Pixel Gaze Estimation Method Face Candidate Region 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001) CrossRefGoogle Scholar
  2. 2.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 381–395 (1981) MathSciNetCrossRefGoogle Scholar
  3. 3.
    Furukawa, Y., Ponce, J.: Accurate, dense, and robust multi-view stereopsis. In: Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007) Google Scholar
  4. 4.
    Hansen, D.W., Ji, Q.: In the eye of the beholder: a survey of models for eyes and gaze. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 478–500 (2010) CrossRefGoogle Scholar
  5. 5.
    Just, M.A., Carpenter, P.A.: Eye fixations and cognitive processes. Cogn. Psychol. 8(4), 441–480 (1976) CrossRefGoogle Scholar
  6. 6.
    Kawaguchi, T., Rizon, M., Hidaka, D.: Detection of eyes from human faces by hough transform and separability filter. Electron. Commun. Jpn. 88(5), 29–39 (2005) CrossRefGoogle Scholar
  7. 7.
    Kuroda, M., Nobuhara, S., Matsuyama, T.: 3d face geometry and gaze estimation from multi-view images using symmetry prior. In: Proc. of MIRU (2011) (in Japanese) Google Scholar
  8. 8.
    Nobuhara, S., Kimura, Y., Matsuyama, T.: Object-oriented color calibration of multi-viewpoint cameras in sparse and convergent arrangement. IPSJ Trans. Comput. Vis. Appl. 2, 132–144 (2010) Google Scholar
  9. 9.
    Nobuhara, S., Tsuda, Y., Ohama, I., Matsuyama, T.: Multi-viewpoint silhouette extraction with 3D context-aware error detection, correction, and shadow suppression. IPSJ Trans. Comput. Vis. Appl. 1, 242–259 (2009) Google Scholar
  10. 10.
    Sugimoto, A., Matsuyama, T.: Active wearable vision sensor: detecting person’s blink points and estimating human motion trajectory. In: Proceedings of the IEEE/ASME International Conference on Advanced Intelligent Mechatronics, 2003, AIM2003, vol. 1, pp. 539–545 (2003) CrossRefGoogle Scholar
  11. 11.
    Tobii Technology: X120 eye tracker Google Scholar
  12. 12.
    Tung, T., Nobuhara, S., Matsuyama, T.: Simultaneous super-resolution and 3D video using graph-cuts. In: Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008) Google Scholar
  13. 13.
    Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57, 137–154 (2004) CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2012

Authors and Affiliations

  • Takashi Matsuyama
    • 1
  • Shohei Nobuhara
    • 1
  • Takeshi Takai
    • 1
  • Tony Tung
    • 1
  1. 1.Graduate School of InformaticsKyoto UniversitySakyoJapan

Personalised recommendations