Weak-Perspective and Scaled-Orthographic Structure from Motion with Missing Data

  • Levente HajderEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 983)


Perspective n-Point (PnP) problem is in focus of 3D computer vision community since the late 80’s. Standard solutions deal with the pinhole camera model, the problem is challenging due to the perspectivity. The well-known PnP algorithms assume that the intrinsic camera parameters are known, therefore, only extrinsic ones are needed to estimate. It is carried out by a rough estimation, usually given in closed forms, then the accurate camera parameters are obtained via numerical optimization. In this paper, we show that both the weak-perspective and scaled orthographic camera models can be optimally calibrated including the intrinsic camera parameters. Moreover, the latter one is done without iteration if the \(L_2\) norm is used. It is also shown that the calibration can be inserted into a structure from motion algorithm. We also show that the scaled orthographic version can be powered by GPUs, yielding real-time performance.


Weak-perspective camera model Perpective n-point problem Structure from motion 


  1. 1.
    Arun, K.S., Huang, T.S., Blostein, S.D.: Least-squares fitting of two 3-D point sets. IEEE Trans. PAMI 9(5), 698–700 (1987)CrossRefGoogle Scholar
  2. 2.
    Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment—a modern synthesis. In: Triggs, B., Zisserman, A., Szeliski, R. (eds.) IWVA 1999. LNCS, vol. 1883, pp. 298–372. Springer, Heidelberg (2000). Scholar
  3. 3.
    Björck, Å.: Numerical Methods for Least Squares Problems. Siam, Philadelphia (1996)CrossRefGoogle Scholar
  4. 4.
    Buchanan, A.M., Fitzgibbon, A.W.: Damped Newton algorithms for matrix factorization with missing data. In: Proceedings of the 2005 IEEE CVPR, pp. 316–322 (2005)Google Scholar
  5. 5.
    Bue, A.D., Xavier, J., Agapito, L., Paladini, M.: Bilinear modeling via augmented lagrange multipliers (balm). IEEE Trans. PAMI 34(8), 1496–1508 (2012)CrossRefGoogle Scholar
  6. 6.
    Choudhary, S., Gupta, S., Narayanan, P.J.: Practical time bundle adjustment for 3D reconstruction on the GPU. In: Kutulakos, K.N. (ed.) ECCV 2010. LNCS, vol. 6554, pp. 423–435. Springer, Heidelberg (2012). Scholar
  7. 7.
    DeMenthon, D.F., Davis, L.S.: Model-based object pose in 25 lines of code. IJCV 15, 123–141 (1995)CrossRefGoogle Scholar
  8. 8.
    Hajder, L.: W-PnP method: optimal solution for the weak-perspective n-Point problem and its application to structure from motion. In: Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP), pp. 265–276 (2017)Google Scholar
  9. 9.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)zbMATHGoogle Scholar
  10. 10.
    Hartley, R., Kahl, F.: Optimal algorithms in multiview geometry. In: Yagi, Y., Kang, S.B., Kweon, I.S., Zha, H. (eds.) ACCV 2007. LNCS, vol. 4843, pp. 13–34. Springer, Heidelberg (2007). Scholar
  11. 11.
    Hartley, R., Schaffalitzky, F.: Powerfactorization: 3D reconstruction with missing or uncertain data (2003)Google Scholar
  12. 12.
    Hesch, J.A., Roumeliotis, S.I.: A direct least-squares (DLS) method for PnP. In: International Conference on Computer Vision, pp. 383–390. IEEE (2011)Google Scholar
  13. 13.
    Horaud, R., Dornaika, F., Lamiroy, B., Christy, S.: Object pose: the link between weak perspective, paraperspective and full perspective. Int. J. Comput. Vis. 22(2), 173–189 (1997)CrossRefGoogle Scholar
  14. 14.
    Horn, B., Hilden, H., Negahdaripourt, S.: Closed-form solution of absolute orientation using orthonormal matrices. J. Opt. Soc. Am. 5(7), 1127–1135 (1988)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Jenkins, M.A., Traub, J.F.: A three-stage variables-shift iteration for polynomial zeros and its relation to generalized Rayleigh iteration. Numer. Math. 14, 252–263 (1970)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Kahl, F., Hartley, R.I.: Multiple-view geometry under the linfinity-norm. IEEE Trans. Pattern Anal. Mach. Intell. 30(9), 1603–1617 (2008)CrossRefGoogle Scholar
  17. 17.
    Kanatani, K., Sugaya, Y., Ackermann, H.: Uncalibrated factorization using a variable symmetric affine camera. IEICE - Trans. Inf. Syst. E90-D(5), 851–858 (2007)CrossRefGoogle Scholar
  18. 18.
    Kazó, C., Hajder, L.: Rapid weak-perspective structure from motion with missing data. In: ICCV Workshops, pp. 491–498 (2011)Google Scholar
  19. 19.
    Ke, Q., Kanade, T.: Quasiconvex optimization for robust geometric reconstruction. In: Proceedings of the Tenth IEEE International Conference on Computer Vision, ICCV 2005, pp. 986–993 (2005)Google Scholar
  20. 20.
    Hajder, L., Pernek, Á., Kazó, C.: Weak-perspective structure from motion by fast alternation. Vis. Comput. 27(5), 387–399 (2011)CrossRefGoogle Scholar
  21. 21.
    Lepetit, V., Moreno-Noguer, F., Fua, P.: EPnP: an accurate O(n) solution to the PnP problem. Int. J. Comput. Vis. 81(2), 155–166 (2009)CrossRefGoogle Scholar
  22. 22.
    Marques, M., Costeira, J.: Estimating 3D shape from degenerate sequences with missing data. CVIU 113(2), 261–272 (2009)Google Scholar
  23. 23.
    Okatani, T., Deguchi, K.: On the Wiberg algorithm for matrix factorization in the presence of missing components. IJCV 72(3), 329–337 (2006)CrossRefGoogle Scholar
  24. 24.
    Pernek, A., Hajder, L., Kazó, C.: Metric reconstruction with missing data under weak-perspective. In: BMVC, pp. 109–116 (2008)Google Scholar
  25. 25.
    Poelman, C.J., Kanade, T.: A paraperspective factorization method for shape and motion recovery. IEEE Trans. PAMI 19(3), 312–322 (1997)CrossRefGoogle Scholar
  26. 26.
    Ruhe, A.: Numerical computation of principal components when several observations are missing. Technical report, Umea Univesity, Sweden (1974)Google Scholar
  27. 27.
    Schweighofer, G., Pinz, A.: Globally optimal O(n) solution to the PnP problem for general camera models. In: BMVC (2008)Google Scholar
  28. 28.
    Shum, H.Y., Ikeuchi, K., Reddy, R.: Principal component analysis with missing data and its application to polyhedral object modeling. IEEE Trans. Pattern Anal. Mach. Intell. 17(9), 854–867 (1995)CrossRefGoogle Scholar
  29. 29.
    Stühmer, J., Gumhold, S., Cremers, D.: Real-time dense geometry from a handheld camera. In: Goesele, M., Roth, S., Kuijper, A., Schiele, B., Schindler, K. (eds.) DAGM 2010. LNCS, vol. 6376, pp. 11–20. Springer, Heidelberg (2010). Scholar
  30. 30.
    Sturm, P., Triggs, B.: A factorization based algorithm for multi-image projective structure and motion. In: Buxton, B., Cipolla, R. (eds.) ECCV 1996. LNCS, vol. 1065, pp. 709–720. Springer, Heidelberg (1996). Scholar
  31. 31.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization approach. Int. J. Comput. Vis. 9, 137–154 (1992)CrossRefGoogle Scholar
  32. 32.
    Tomasi, C., Shi, J.: Good features to track. In: IEEE Conference Computer Vision and Pattern Recognition, pp. 593–600 (1994)Google Scholar
  33. 33.
    Tzevanidis, K., Zabulis, X., Sarmis, T., Koutlemanis, P., Kyriazis, N., Argyros, A.: From multiple views to textured 3D meshes: a GPU-powered approach. In: Kutulakos, K.N. (ed.) ECCV 2010. LNCS, vol. 6554, pp. 384–397. Springer, Heidelberg (2012). Scholar
  34. 34.
    Wang, G., Wu, Q.M.J., Sun, G.: Quasi-perspective projection with applications to 3D factorization from uncalibrated image sequences. In: CVPR (2008)Google Scholar
  35. 35.
    Weinshall, D., Tomasi, C.: Linear and incremental acquisition of invariant shape models from image sequences. IEEE Trans. PAMI 17(5), 512–517 (1995)CrossRefGoogle Scholar
  36. 36.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. PAMI 22(11), 1330–1334 (2000)CrossRefGoogle Scholar
  37. 37.
    Zheng, Y., Kuang, Y., Sugimoto, S., Åström, K., Okutomi, M.: Revisiting the PnP problem: a fast, general and optimal solution. In: ICCV, pp. 2344–2351 (2013)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Algorithms and Their ApplicationsEötvös Loránd UniversityBudapestHungary
  2. 2.Machine Perception LaboratoryMTA SZTAKIBudapestHungary

Personalised recommendations