Real-Time Image Mosaicing Using Non-rigid Registration

  • Rafael Henrique Castanheira de Souza
  • Masatoshi Okutomi
  • Akihiko Torii
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7087)


Mosaicing is a classical application of image registration where images from the same scene are stitched together to generate a larger seamless image. This paper presents a real-time incremental mosaicing method that generates 2D mosaics by stitching video key-frames as soon as they are detected. The contributions are three-fold: (1) we propose a “fast” key-frame selection procedure based solely on the distribution of the distance of matched feature descriptors. This procedure automatically selects key-frames that are used to expand the mosaics while achieving real-time performance; (2) we register key-frame images by using a non-rigid deformation model in order to “smoothly” stitch images when scene transformations can not be expressed by homography: (3) we add a new constraint on the non-rigid deformation model that penalizes over-deformation in order to create “visually natural” mosaics. The performance of the proposed method was validated by experiments in non-controlled conditions and by comparison with the state-of-the-art method.


mosaic non-rigid registration feature based real-time 


  1. 1.
    Bay, H., Tuytelaars, T., Van Gool, L.: SURF: Speeded Up Robust Features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  2. 2.
    Brown, M., Lowe, D.: Automatic panoramic image stitching using invariant features. International Journal of Computer Vision 74, 59–73 (2007)CrossRefGoogle Scholar
  3. 3.
    Can, A., Stewart, C.V., Roysam, B., Tanenbaum, H.L.: A feature-based technique for joint, linear estimation of high-order image-to-mosaic transformations: application to mosaicing the curved human retina. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 585–591 (2000)Google Scholar
  4. 4.
    Chaiyasarn, K., Kim, T.-K., Viola, F., Cipolla, R., Soga, K.: Image mosaicing via quadric surface estimation with priors for tunnel inspection. In: 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 537–540 (2009)Google Scholar
  5. 5.
    Chui, H., Rangarajan, A.: A new point matching algorithm for non-rigid registration. Computer Vision and Image Understanding 89(2-3), 114–141 (2003)CrossRefzbMATHGoogle Scholar
  6. 6.
    Crispell, D., Mundy, J., Taubin, G.: Parallax-free registration of aerial video. In: Proc. British Machine Vision Conf. (2008)Google Scholar
  7. 7.
    Deng, Y., Zhang, T.: Generating panorama photos. In: Proc. of SPIE Internet Multimedia Management Systems IV (2003)Google Scholar
  8. 8.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Hsu, S., Sawhney, H.S., Kumar, R.: Automated mosaics via topology inference. IEEE Computer Graphics and Applications 22(2), 44–54 (2002)CrossRefGoogle Scholar
  10. 10.
    Peleg, S., Rousso, B., Rav-Acha, A., Zomet, A.: Mosaicing on adaptive manifolds. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(10), 1144–1154 (2000)CrossRefGoogle Scholar
  11. 11.
    Pilet, J., Lepetit, V., Fua, P.: Real-time non-rigid surface detection. In: Proc. IEEE Conf. Computer Vision Pattern Recognition, pp. 822–828 (2005)Google Scholar
  12. 12.
    Sawhney, H.S., Hsu, S., Kumar, R.: Robust Video Mosaicing Through Topology Inference and Local to Global Alignment. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 103–119. Springer, Heidelberg (1998)Google Scholar
  13. 13.
    Szeliski, R.: Image alignment and stitching: a tutorial. Found. Trends. Comput. Graph. Vis. 2, 1–104 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Zhu, J., Lyu, M.R., Huang, T.S.: A fast 2d shape recovery approach by fusing features and appearance. IEEE Transactions on Pattern Analysis and Machine Intelligence 31, 1210–1224 (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Rafael Henrique Castanheira de Souza
    • 1
  • Masatoshi Okutomi
    • 1
  • Akihiko Torii
    • 1
  1. 1.Tokyo Institute of TechnologyJapan

Personalised recommendations