Advertisement

Multiple View 3D Reconstruction with Rolling Shutter Cameras

  • Gaspard DuchampEmail author
  • Omar Ait-Aider
  • Eric Royer
  • Jean-Marc Lavest
Part of the Communications in Computer and Information Science book series (CCIS, volume 598)

Abstract

Nowadays Rolling shutter CMOS cameras are embedded on a lot of devices. This type of cameras does not have its retina exposed simultaneously but line by line. The resulting distortions affect structure from motion methods developed for global shutter, like CCD cameras. The bundle adjustment method presented in this paper deals with rolling shutter cameras. We use a projection model which considers pose and velocity and needs 6 more parameters for one view in comparison to the global shutter model. We propose a simplified model which only considers distortions due to rotational speed. We compare it to the global shutter model and the full rolling shutter one. The model does not need any condition on the inter-frame motion so it can be applied to fully independent views, even with global shutter images equivalent to a null velocity. We also propose a way to handle epipolar geometry for rolling shutter. It is shown that constraint using essential matrix becomes non linear, and we show how to use it to recover poses and speeds from matched points. Results with both synthetic and real images shows that the simplified model can be considered as a good compromise between a correct geometrical modelling of rolling shutter effects and the reduction of the number of extra parameters.

Keywords

Optical Flow Bundle Adjustment Projection Equation Epipolar Constraint Essential Matrix 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Liang, C.K., Chang, L.W., Chen, H.H.: Analysis and compensation of rolling shutter effect. IEEE Trans. Image Process. 17, 1323–1330 (2008)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Baker, S., Bennett, E., Kang, S.B., Szeliski, R.: Removing rolling shutter wobble. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2392–2399. IEEE (2010)Google Scholar
  3. 3.
    Bradley, D., Atcheson, B., Ihrke, I., Heidrich, W.: Synchronization and rolling shutter compensation for consumer video camera arrays. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2009, CVPR Workshopps 2009, pp. 1–8. IEEE (2009)Google Scholar
  4. 4.
    Ait-Aider, O., Andreff, N., Lavest, J.-M., Martinet, P.: Simultaneous object pose and velocity computation using a single view from a rolling shutter camera. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 56–68. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  5. 5.
    Ait-Aider, O., Bartoli, A., Andreff, N.: Kinematics from lines in a single rolling shutter image. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007, pp. 1–6. IEEE (2007)Google Scholar
  6. 6.
    Meingast, M., Geyer, C., Sastry, S.: Geometric models of rolling-shutter cameras. arXiv preprint cs/0503076 (2005)
  7. 7.
    Magerand, L., Bartoli, A., Ait-Aider, O., Pizarro, D.: Global optimization of object pose and motion from a single rolling shutter image with automatic 2D-3D matching. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012, Part I. LNCS, vol. 7572, pp. 456–469. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  8. 8.
    Meilland, M., Drummond, T., Comport, A.I.: A unified rolling shutter and motion blur model for 3d visual registration. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 2016–2023. IEEE (2013)Google Scholar
  9. 9.
    Ait-Aider, O., Berry, F.: Structure and kinematics triangulation with a rolling shutter stereo rig. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1835–1840. IEEE (2009)Google Scholar
  10. 10.
    Hedborg, J., Forssen, P.E., Felsberg, M., Ringaby, E.: Rolling shutter bundle adjustment. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1434–1441. IEEE (2012)Google Scholar
  11. 11.
    Saurer, O., Koser, K., Bouguet, J.Y., Pollefeys, M.: Rolling shutter stereo. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 465–472. IEEE (2013)Google Scholar
  12. 12.
    Anderson, S., Barfoot, T.D.: Towards relative continuous-time slam. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), pp. 1033–1040. IEEE (2013)Google Scholar
  13. 13.
    Lovegrove, S., Patron-Perez, A., Sibley, G.: Spline fusion: a continuous-time representation for visual-inertial fusion with application to rolling shutter cameras. In: British Machine Vision Conference (2013)Google Scholar
  14. 14.
    Li, M., Kim, B.H., Mourikis, A.I.: Real-time motion tracking on a cellphone using inertial sensing and a rolling-shutter camera. In: 2013 IEEE International Conference on Robotics and Automation (ICRA), pp. 4712–4719. IEEE (2013)Google Scholar
  15. 15.
    Li, M., Mourikis, A.I.: Vision-aided inertial navigation with rolling-shutter cameras. Int. J. Robot. Res. 33, 1490–1507 (2014)CrossRefGoogle Scholar
  16. 16.
    Hedborg, J., Ringaby, E., Forssén, P.E., Felsberg, M.: Structure and motion estimation from rolling shutter video. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 17–23. IEEE (2011)Google Scholar
  17. 17.
    Shoemake, K.: Animating rotation with quaternion curves. In: ACM SIGGRAPH computer graphics, vol. 19, pp. 245–254. ACM (1985)Google Scholar
  18. 18.
    Ringaby, E., Forssén, P.E.: Efficient video rectification and stabilisation for cell-phones. Int. J. Comput. Vis. 96, 335–352 (2012)CrossRefGoogle Scholar
  19. 19.
    Mouragnon, E., Lhuillier, M., Dhome, M., Dekeyser, F., Sayd, P.: Generic and real-time structure from motion using local bundle adjustment. Image Vis. Comput. 27, 1178–1193 (2009)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Gaspard Duchamp
    • 1
    Email author
  • Omar Ait-Aider
    • 1
  • Eric Royer
    • 1
  • Jean-Marc Lavest
    • 1
  1. 1.Institut PascalAubiereFrance

Personalised recommendations