Abstract
Free-viewpoint video in sports TV production presents a challenging problem involving the conflicting requirements of broadcast picture quality with video-rate generation of novel views, together with practical problems in developing robust systems for cost effective deployment at live events.To date most multiple view video systems have been developed for studio applications with a fixed capture volume, controlled illumination and backgrounds. Live outdoor events such as sports present a number of additional challenges for both acquisition and processing. Multiple view capture systems in sports such as football must cover the action taking place over an entire pitch with video acquisition at sufficient resolution for analysis and production of desired virtual camera views.In this chapter we identify the requirements for broadcast production of free-viewpoint video in sports and review the state-of-the-art. We present the iview free-viewpoint video system which enables production of novel camera views of sports action for use in match commentary, for example the referees viewpoint. Automatic online calibration, segmentation and reconstruction is performed to allow rendering of novel viewpoints from the moving match cameras.Results are reported of production trials in football (soccer) and rugby which demonstrate free-viewpoint video with a visual quality comparable to the broadcast footage.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., Thrun, S.: Performance capture from sparse multi-view video. In: Proc. ACM SIGGRAPH, 27(3) (2008)
Boykov, Y., Veksle, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)
Broadhurst, A., Drummond, T.W., Cipolla, R.: A probabilistic framework for space carving. In: International Conf. on Computer Vision, pp. 388–393 (2001)
Carranza, J., Theobalt, C., Magnor, M., Seidel, H.-P.: Free-viewpoint video of human actors. In: Proc. ACM SIGGRAPH, pp. 565–577 (2003)
Chen, S.E., Williams, L.: View interpolation for image synthesis. In: Proc. ACM SIGGRAPH (1993)
Connor, K., Reid, I.: A multiple view layered representation for dynamic novel view synthesis. In: British Machine Vision Conference (2003)
Debevec, P., Yu, Y., Borshukov, G.: Efficient view-dependent image-based rendering with projective texture-mapping. In: 9th Eurographics Rendering Workshop, pp. 105–116 (1998)
Deutscher, J., Davidson, A., Reid, I.: Automatic partitioning of high-dimensional search spaces associated with articulated body motion capture. In: Conference on Computer Vision and Pattern Recognition, pp. 669–676 (2001)
Eisemann, M., Decker, B., Magnor, M.A., Bekaert, P., Aguiar, E., Ahmed, N., Theobalt, C., Sellent, A.: Floating textures. Comput. Graph. Forum 27(2), 409–418 (2006)
Franco, F.S., Boyer, E.: Exact polyhedral visual hulls. In: British Machine Vision Conference, pp. 329–338 (2003)
Franco, F.S., Boyer, E.: Fusion of multi-view silhouette cues using a space occupacy grid. In: International Conf. on Computer Vision, pp. 1747–1753 (2005)
Franco, F.S., Menier, C., Boyer, E., Raffin, B.: A distributed approach for real-time 3D modeling. In: CVPR Workshop on Real-Time 3D Sensors and their Applications (2004)
Goldluecke, B., Magnor, M.: Space–time isosurface evolution for temporally coherent 3D reconstruction. In: Conference on Computer Vision and Pattern Recognition, pp. S–E, Washington, DC, USA, July (2004). IEEE Computer Society.
Grau, O., Easterbrook, J.: Effects of camera aperture correction on keying of broadcast video. In: Proc. of the 5rd European Conference on Visual Media Production (CVMP) (2008)
Grau, O., Hilton, A., Kilner, J., Miller, G., Sargeant, T., Starck, J.: A free-viewpoint video system for visualisation of sports scenes. SMPTE Motion Imag. J. 116(5–6), 213–219 (2007)
Grau, O., Prior-Jones, M., Thomas, G.: 3D modelling and rendering of studio and sport scenes for TV applications. In: Proc. of WIAMIS (2005)
Grau, O., Thomas, G.A., Hilton, A., Kilner, J., Starck, J.: A robust free-viewpoint video system for sport scenes. In: Proceeding of 3DTV conference 2007, Kos, Greece (2007)
Guillemaut, J.-Y., Kilner, J., Hilton, A.: Robust graph-cut scene segmentation and reconstruction for free-viewpoint video of complex dynamic scenes. In: IEEE Int.Conf. on Computer Vision, ICCV (2009)
Guillemaut, J.-Y., Kilner, J., Starck, J., Hilton, A.: Dynamic feathering: minimising blending artefacts in view dependent rendering. In: IET European Conference on Visual Media Production, pp. 1–8 (2007)
Guillemaut, J.Y., Hilton, A., Starck, J., Kilner, J.J., Grau, O.: A Baysian framework for simultaneous reconstruction and matting. In: IEEE Int.Conf. on 3D Imaging and Modeling (2007)
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)
Hillman, P., Hannah, J., Renshaw, D.: Foreground/background segmentation of motion picture images and image sequences. IEE Trans. Vis. Image Signal Process. 152(4), 387–397 (2005)
Inamoto, N., Saito, H.: Virtual viewpoint replay for a soccer match by view interpolation from multiple cameras. IEEE Trans. Multimedia 9(6), 1155–1166 (2007)
Irani, M., Hassner, T., Anandan, P.: What does the scene look like from a scene point? In: European Conference on Computer Vision (2002)
Kanade, T., Rander, P.: Virtualized reality: Constructing virtual worlds from real scenes. IEEE MultiMedia 4(2), 34–47 (1997)
Kanade, T., Rander, P.W., Narayanan, P.J.: Virtualized reality: constructing virtual worlds from real scenes. IEEE MultiMedia 4(1), 34–47 (1997)
Kilner, J.J., Starck, J., Hilton, A., Guillemaut, J.Y., Grau, O.: Dual mode deformable models for free-viewpoint video of outdoor sports events. In: IEEE Int. Conf. on 3D Imaging and Modeling (2007)
Kimura, K., Saito, H.: Player viewpoint video synthesis using multiple cameras. In: IEE European Conference on Visual Media Production, pp. 112–121 (2005)
Laurentini, A.: The visual hull concept for silhouette based image understanding. IEEE Trans. Pattern Anal. Mach. Intell. 16(2), 150–162 (1994)
Matusik, W., Buehler, C., Raskar, R., Gortler, S.J., McMillan, L.: Image-based visual hulls. In: Proceedings of ACM SIGGRAPH, pp. 369–374 (2000)
Miller, G., Hilton, A., Starck, J.: Interactive free-viewpoint video. In: IEE European Conf. on Visual Media Production, pp. 50–59 (2005)
Miller, G., Starck, J.R., Hilton, A.: Projective surface refinement for free-viewpoint video. In: IET European Conference on Visual Media Production, pp. 153–162 (2006)
Moeslund, T., Hilton, A., Kruger, V.: A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 104(2–3), 90–127 (2006)
Moezzi, S., Katkere, A., Kuramura, D.Y., Jain, R.: Reality modeling and visualization from multiple video sequences. IEEE Comput. Graph. Appl. 16(6), 58–63 (1996)
Moezzi, S., Tai, L.C., Gerard, P.: Virtual view generation for 3D digital video. IEEE MultiMedia 4(1), 18–25 (1997)
Scarstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1), 7–42 (2002)
Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: Conference on Computer Vision and Pattern Recognition, pp. 519–528 (2006)
Seitz, S.M., Dyer, C.R.: View morphing: synthesizing 3D metamorphosis using image transforms. In: Proc. ACM SIGGRAPH, pp. 21–30 (1996)
Seitz, S.M., Dyer, C.R.: Photorealistic scene reconstruction by voxel coloring. Int. J. Comput. Vis. 35(2), 1–23 (1997)
Shum, H.-Y., Kang, S.B., Chan, S.-C.: Survey of image-based representations and compression techniques. IEEE Trans. Circuits Syst. Video Technol. 13(11) (2003)
Slabaugh, G., Culbertson, W.B., Malzbender, T., Stevens, M., Schafer, R.: A survey of methods for volumetric scene reconstruction from photographs. In: Volume Graphics (2001)
Starck, J., Hilton, A.: Virtual view synthesis of people from multiple view video. Graph. Models 67(6), 600–620 (2005)
Starck, J., Hilton, A.: Surface capture for performance-based animation. IEEE Comput. Graph. Appl. 27(3), 21–31 (2007)
Stich, T., Linz, C., Wallraven, C., Cunningham, D., Magnor, M.: Perception-motivated interpolation of image sequences. In: Proc. ACM APGV, pp. 97–106 (2008)
Thomas, G.A.: Real-time camera pose estimation for augmenting sports scenes. In: European Conference on Visual Media Production, pp. 10–19 (2006)
Vedula, S., Baker, S., Rander, P., Collins, R., Kanade, T.: Three-dimensional scene flow. IEEE Trans. Pattern Anal. Mach. Intell. 27(3), 475–480 (2005)
Vlasic, D., Baran, I., Matusik, W., Popović, J.: Articulated mesh animation from multi-view silhouettes. In: Proc. ACM SIGGRAPH, pp. 1–9 (2008)
Zitnick, C.L., Kang, S.B., Uyttendaele, M., Winder, S., Szeliski, R.: High-quality video view interpolation using a layered representation. In: Proc. ACM SIGGRAPH, pp. 600–608 (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Hilton, A., Guillemaut, JY., Kilner, J., Grau, O., Thomas, G. (2010). Free-Viewpoint Video for TV Sport Production. In: Ronfard, R., Taubin, G. (eds) Image and Geometry Processing for 3-D Cinematography. Geometry and Computing, vol 5. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12392-4_4
Download citation
DOI: https://doi.org/10.1007/978-3-642-12392-4_4
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12391-7
Online ISBN: 978-3-642-12392-4
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)