Advertisement

Quality evaluation-based iterative seam estimation for image stitching

  • Tianli Liao
  • Jing Chen
  • Yifang XuEmail author
Original Paper
  • 32 Downloads

Abstract

Seam-driven approaches have been proven effective for handling imperfect image series in image stitching. Generally, seam-driven approach utilizes seam-cutting to find a best stitching seam from one or finite alignment hypotheses based on a predefined seam quality metric. However, the quality metrics in these methods are defined in order to evaluate the average performance of the pixels along the seam. In many cases, the seam with the minimal cost is not necessarily optimal in terms of human perception. In this paper, we propose a novel iterative seam estimation method where the iteration procedure is guided by our quality evaluation for the pixels along the seam. First, we estimate a stitching seam via the conventional seam-cutting and introduce a hybrid quality evaluation to evaluate the pixels along the seam; the evaluated costs are then used to recalculate the difference map of the overlapping region and re-estimate a stitching seam. This evaluation–re-estimation procedure iterates until the current seam changes negligibly comparing with the previous seams. Experiments show that comparing with the conventional seam-cutting and other seam-driven methods, our method can produce results with less artifacts and overall is more visually pleasing.

Keywords

Image stitching Seam estimation Quality evaluation Human perception 

Notes

Supplementary material

11760_2019_1466_MOESM1_ESM.pdf (22.8 mb)
Supplementary material 1 (pdf 23354 KB)

References

  1. 1.
    Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., Cohen, M.: Interactive digital photomontage. ACM Trans. Graph. 23(3), 294–302 (2004)CrossRefGoogle Scholar
  2. 2.
    Anbarjafari, G.: An objective no-reference measure of illumination assessment. Meas. Sci. Rev. 15(6), 319–322 (2015)CrossRefGoogle Scholar
  3. 3.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  4. 4.
    Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74(1), 59–73 (2007)CrossRefGoogle Scholar
  5. 5.
    Chen, Y.S., Chuang, Y.Y.: Natural image stitching with the global similarity prior. In: Proceedings of 14th European Conference on Computer Vision, pp. 186–201 (2016)Google Scholar
  6. 6.
    Davis, J.: Mosaics of scenes with moving objects. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 354–360 (1998)Google Scholar
  7. 7.
    Eden, A., Uyttendaele, M., Szeliski, R.: Seamless image stitching of scenes with large motions and exposure differences. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 2498–2505 (2006)Google Scholar
  8. 8.
    Fang, X., Zhu, J., Luo, B.: Image mosaic with relaxed motion. Signal Image Video Process. 6(4), 647–667 (2012)CrossRefGoogle Scholar
  9. 9.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Gao, J., Kim, S.J., Brown, M.S.: Constructing image panoramas using dual-homography warping. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 49–56 (2011)Google Scholar
  11. 11.
    Gao, J., Li, Y., Chin, T.J., Brown, M.S.: Seam-driven image stitching. Eurographics, pp. 45–48 (2013)Google Scholar
  12. 12.
    Hejazifar, H., Khotanlou, H.: Fast and robust seam estimation to seamless image stitching. Signal Image Video Process. 12(5), 885–893 (2018)CrossRefGoogle Scholar
  13. 13.
    Herrmann, C., Wang, C., Strong Bowen, R., Keyder, E., Zabih, R.: Object-centered image stitching. In: Proceeings of 15th European Conference on Computer Vision, pp. 821–835 (2018)Google Scholar
  14. 14.
    Jia, J., Tang, C.K.: Image stitching using structure deformation. IEEE Trans. Pattern Anal. Mach. Intell. 30(4), 617–631 (2008)CrossRefGoogle Scholar
  15. 15.
    Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: image and video synthesis using graph cuts. ACM Trans. Graph. 22(3), 277–286 (2003)CrossRefGoogle Scholar
  16. 16.
    Li, N., Liao, T., Wang, C.: Perception-based seam cutting for image stitching. Signal Image Video Process. 12(5), 967–974 (2018)CrossRefGoogle Scholar
  17. 17.
    Lin, K., Jiang, N., Cheong, L.F., Do, M., Lu, J.: Seagull: seam-guided local alignment for parallax-tolerant image stitching. In: Proceedings of 14th European Conference on Computer Vision, pp. 370–385 (2016)Google Scholar
  18. 18.
    Lin, W.Y., Liu, S., Matsushita, Y., Ng, T.T., Cheong, L.F.: Smoothly varying affine stitching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 345–352 (2011)Google Scholar
  19. 19.
    Liu, F., Gleicher, M., Jin, H., Agarwala, A.: Content-preserving warps for 3d video stabilization. ACM Trans. Graph. 28(3), 44 (2009)Google Scholar
  20. 20.
    Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. 22(3), 313–318 (2003)CrossRefGoogle Scholar
  21. 21.
    Rzhanov, Y.: Photo-mosaicing of images of pipe inner surface. Signal Image Video Process. 7(5), 865–871 (2013)CrossRefGoogle Scholar
  22. 22.
    Szeliski, R.: Image alignment and stitching: a tutorial. Found. Trends Comput. Graph. Vis. 2(1), 1–104 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Wang, Z., Bovik, A.C.: A universal image quality index. IEEE Signal Process. Lett. 9(3), 81–84 (2002)CrossRefGoogle Scholar
  24. 24.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  25. 25.
    Xu, Z.: Consistent image alignment for video mosaicing. Signal Image Video Process. 7(1), 129–135 (2013)CrossRefGoogle Scholar
  26. 26.
    Zaragoza, J., Chin, T.J., Tran, Q.H., Brown, M.S., Suter, D.: As-projective-as-possible image stitching with moving DLT. IEEE Trans. Pattern Anal. Mach. Intell. 7(36), 1285–1298 (2014)Google Scholar
  27. 27.
    Zeng, L., Zhang, W., Zhang, S., Wang, D.: Video image mosaic implement based on planar-mirror-based catadioptric system. Signal Image Video Process. 8(6), 1007–1014 (2014)CrossRefGoogle Scholar
  28. 28.
    Zhang, F., Liu, F.: Parallax-tolerant image stitching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3262–3269 (2014)Google Scholar
  29. 29.
    Zhang, G., He, Y., Chen, W., Jia, J., Bao, H.: Multi-viewpoint panorama construction with wide-baseline images. IEEE Trans. Image Process. 25(7), 3099–3111 (2016)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  1. 1.Center for CombinatoricsNankai UniversityTianjinChina

Personalised recommendations