Annotation of Video by Alignment to Reference Imagery
Video as an entertainment or information source in consumer, military, and broadcast television applications is widespread. Typically however, the video is simply presented to the viewer, with only minimal manipulation. Examples include chroma-keying (often used in news and weather broadcasts) where specific color components are detected and used to control the video source. In the past few years, the advent of digital video and increases in computational power has meant that more complex manipulation can be performed. In this paper we present some highlights of our work in annotating video by aligning features extracted from the video to a reference set of features.
Video insertion and annotation require manipulation of the video stream to composite synthetic imagery and information with real video imagery. The manipulation may involve only the 2D image space or the 3D scene space. The key problems to be solved are: (i) indexing and matching to determine the location of insertion, (ii) stable and jitter-free tracking to compute the time variation of the camera, and (iii) seamlessly blended insertion for an authentic viewing experience. We highlight our approach to these problems by showing three example scenarios: (i) 2D synthetic pattern insertion in live video, (ii) annotation of aerial imagery through geo-registration with stored reference imagery and annotations, and (iii) 3D object insertion in a video for a 3D scene.
KeywordsReference Image Video Frame Video Stream Current Image Correlation Surface
Unable to display preview. Download preview PDF.
- 1.J. R. Bergen, P. Anandan, K. J. Hanna, and R. Hingorani. Hierarchical model-based motion estimation. In European Conference on Computer Vision, Santa-Margherita Ligure, Italy, 1992.Google Scholar
- 2.Burt et. al. Object tracking with a moving camera. In IEEE Workshop on Visual Motion, Irvine CA, 1989.Google Scholar
- 3.R. Kumar et al. Registration of video to geo-referenced imagery. In Proc. International Conference on Pattern Recognition, 1998.Google Scholar
- 4.K. Hanna and P. Burt. US Patent 5,566,251-October 15, 1996.Google Scholar
- 5.R. I. Hartley. Estimation of relative camera positions for uncalibrated cameras. In Proc. 2nd European Conference on Computer Vision, pages 579–587, 1992.Google Scholar
- 6.Michal Irani. Applications of image mosaics. In International Conference on Computer Vision, Cambridge, MA, November 1995.Google Scholar
- 7.R. Kumar, P. Anandan, and K. Hanna. Direct recovery of shape from multiple views: a parallax based approach. In Proc 12th ICPR, 1994.Google Scholar
- 8.Princeton Video Image. http://www.pvimage.com.
- 9.RADIUS PI Reports and Technical papers. Proc. darpa image understanding workshop, 1996. pp. 255–525.Google Scholar
- 10.R. Rosser and M. Leach. US Patent 5,264,933-November 23, 1993.Google Scholar
- 11.Harpreet Sawhney. 3D geometry from planar parallax. In Proc. CVPR 94, June 1994.Google Scholar
- 12.Harpreet S. Sawhney, Steve Hsu, and R. Kumar. Robust video mosaicing through topology inference and local to global alignment. In ECCV, pages 103–119, 1998.Google Scholar
- 13.A. Shashua and N. Navab. Relative affine structure, theory and application to 3d reconstruction from 2d views. In IEEE Conference on Computer Vision and Pattern Recognition, June 1994.Google Scholar
- 14.R. Szeliski and H. Shum. Creating full view panoramic image mosaics and environment maps. In Proc. ofSIGGRAPH, pages 251–258, 1997.Google Scholar