Advertisement

GreenWarps: A Two-Stage Warping Model for Stitching Images Using Diffeomorphic Meshes and Green Coordinates

  • Geethu Miriam JacobEmail author
  • Sukhendu Das
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11132)

Abstract

Image Stitching is a hard task to solve in the presence of large parallax in the images. Specifically, for a sequence of frames from unconstrained videos which are considerably shaky, recent works fail to align such a sequence of images accurately. The proposed method “GreenWarps” aims to accurately align frames/images with large parallax. The method consists of two novel stages, namely, Prewarping and Diffeomorphic Mesh warping. The first stage warps unaligned image to the reference image using Green Coordinates. The second stage of the model refines the alignment by using a demon-based diffeomorphic warping method for mesh deformation termed “DiffeoMeshes”. The warping is performed using Green Coordinates in both the stages without the assumption of any motion model. The combination of the two stages provide accurate alignment of the images. Experiments were performed on two standard image stitching datasets and one dataset consisting of images created from unconstrained videos. The results show superior performance of our method compared to the state-of-the-art methods.

Keywords

Green coordinates Diffeomorphic registration Content preserving warps Image stitching 

Supplementary material

478824_1_En_67_MOESM1_ESM.pdf (1.1 mb)
Supplementary material 1 (pdf 1146 KB)

References

  1. 1.
    Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74(1), 59–73 (2007)CrossRefGoogle Scholar
  2. 2.
    Szeliski, R., Shum, H.Y.: Creating full view panoramic image mosaics and environment maps. In: SIGGRAPH (1997)Google Scholar
  3. 3.
    Liu, F., Gleicher, M., Jin, H., Agarwala, A.: Content-preserving warps for 3D video stabilization. ACM Trans. Graph. (TOG) 28(3), 44 (2009)Google Scholar
  4. 4.
    Zaragoza, J., Chin, T.J., Brown, M.S., Suter, D.: As-projective-as-possible image stitching with moving DLT. In: CVPR (2013)Google Scholar
  5. 5.
    Lin, C.C., Pankanti, S.U., Natesan Ramamurthy, K., Aravkin, A.Y.: Adaptive as-natural-as-possible image stitching. In: CVPR (2015)Google Scholar
  6. 6.
    Yan, W., Hou, C., Lei, J., Fang, Y., Gu, Z., Ling, N.: Stereoscopic image stitching based on a hybrid warping model. IEEE Trans. Circuits Syst. Video Technol. 27(9), 1934–1946 (2017)CrossRefGoogle Scholar
  7. 7.
    Gao, J., Kim, S.J., Brown, M.S.: Constructing image panoramas using dual-homography warping. In: CVPR (2011)Google Scholar
  8. 8.
    Chang, C.H., Sato, Y., Chuang, Y.Y.: Shape-preserving half-projective warps for image stitching. In: CVPR (2014)Google Scholar
  9. 9.
    Lin, K., Jiang, N., Liu, S., Cheong, L.F., Lu, M.D.J.: Direct photometric alignment by mesh deformation. In: CVPR (2017)Google Scholar
  10. 10.
    Zhang, F., Liu, F.: Parallax-tolerant image stitching. In: CVPR (2014)Google Scholar
  11. 11.
    Chen, Y.-S., Chuang, Y.-Y.: Natural image stitching with the global similarity prior. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 186–201. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46454-1_12CrossRefGoogle Scholar
  12. 12.
    Lin, K., Jiang, N., Cheong, L.-F., Do, M., Lu, J.: SEAGULL: seam-guided local alignment for parallax-tolerant image stitching. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 370–385. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46487-9_23CrossRefGoogle Scholar
  13. 13.
    Lipman, Y., Levin, D., Cohen-Or, D.: Green coordinates. ACM Trans. Graph. (TOG) 27(3), 78 (2008)CrossRefGoogle Scholar
  14. 14.
    Vercauteren, T., Pennec, X., Perchant, A., Ayache, N.: Diffeomorphic demons: efficient non-parametric image registration. NeuroImage 45(1), S61–S72 (2009)CrossRefGoogle Scholar
  15. 15.
    Lipman, Y., Levin, D.: Derivation and analysis of green coordinates. Comput. Methods Funct. Theory 10(1), 167–188 (2010)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Li, J., Wang, Z., Lai, S., Zhai, Y., Zhang, M.: Parallax-tolerant image stitching based on robust elastic warping. IEEE Trans. Multimedia 20, 1672–1687 (2017)CrossRefGoogle Scholar
  17. 17.
    Thirion, J.P.: Image matching as a diffusion process: an analogy with Maxwell’s demons. Med. Image Anal. 2(3), 243–260 (1998)CrossRefGoogle Scholar
  18. 18.
    Santos-Ribeiro, A., Nutt, D.J., McGonigle, J.: Inertial demons: a momentum-based diffeomorphic registration framework. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9902, pp. 37–45. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46726-9_5CrossRefGoogle Scholar
  19. 19.
    Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1), 89–97 (2004)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Burt, P.J., Adelson, E.H.: A multiresolution spline with application to image mosaics. ACM Trans. Graph. (TOG) 2(4), 217–236 (1983)CrossRefGoogle Scholar
  21. 21.
    Li, N., Xu, Y., Wang, C.: Quasi-homography warps in image stitching. arXiv preprint arXiv:1701.08006 (2017)

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Visualization and Perception Lab, Department of Computer Science and EngineeringIndian Institute of Technology, MadrasChennaiIndia

Personalised recommendations