Skip to main content
SpringerLink
Log in

Exact Simultaneous Recovery of Locations and Structure from Known Orientations and Corrupted Point Correspondences

  • Published:
Discrete & Computational Geometry Aims and scope Submit manuscript

Abstract

Let \(t_1,\ldots ,t_{n_l} \in {\mathbb {R}}^d\) and \(p_1,\ldots ,p_{n_s} \in {\mathbb {R}}^d\) and consider the bipartite location recovery problem: given a subset of pairwise direction observations \(\{(t_i - p_j) / \Vert t_i - p_j\Vert _2\}_{i,j \in [n_\ell ] \times [n_\text {s}]}\), where a constant fraction of these observations are arbitrarily corrupted, find \(\{t_i\}_{i \in [n_\ell ]}\) and \(\{p_j\}_{j \in [n_\text {s}]}\) up to a global translation and scale. This task arises in the Structure from Motion problem from computer vision, which consists of recovering the three-dimensional structure of a scene from photographs at unknown vantage points. We study the recently introduced ShapeFit algorithm as a method for solving this bipartite location recovery problem. In this case, ShapeFit consists of a simple convex program over \(d(n_l + n_s)\) real variables. We prove that this program recovers a set of \(n_l+n_s\) i.i.d. Gaussian locations exactly and with high probability if the observations are given by a bipartite Erdős–Rényi graph, d is large enough, and provided that at most a constant fraction of observations involving any particular location are adversarially corrupted. This recovery theorem is based on a set of deterministic conditions that we prove are sufficient for exact recovery. Finally, we propose a modified pipeline for the Structure for Motion problem, based on this bipartite location recovery problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. That is, E can be partitioned into the disjoint union of \(E_g\) and \(E_b\).

  2. G is an undirected graph with edge set E over the vertices \(V_\ell \cup V_\text {s}\). For each \(i \in V_\ell \) and \(j \in V_\text {s}\), the edge (i, j) is present in E with probability p, independent from other edges. No edges in \(V_\ell \times V_\ell \) or \(V_\text {s}\times V_\text {s}\) are present in E.

  3. To see this, apply Lemma 2.2 part (i) with \(t_1,t_2, t_3, t_4, v_1, v_2, v_3, v_4, \alpha , \tilde{\delta }_{12}, \tilde{\delta }_{23}, \tilde{\delta }_{34}, \tilde{\delta }_{41}\) set to \(t^{(0)}_{a}, t^{(0)}_{b}, t^{(0)}_{c}, t^{(0)}_{d}, t_{a}, t_{b}, t_{c}, t_{d}, 1, \delta _{ab}, \delta _{bc}, \delta _{cd}, \delta _{da}\), respectively. By applying Condition 2, one obtains \(\eta (ab,cd) \ge \beta \Vert t^{(0)}_{ab}\Vert _2 |\delta _{ab} - \delta _{cd}|\). By Condition 2 and the fact that \(\delta _{ab}>\delta _{cd}\), one concludes \(\eta (ab,cd) > 0\).

  4. Lemma 2.2, part (i) should be applied with the values of \(t_1,t_2, t_3, t_4, v_1, v_2, v_3, v_4, \alpha , \tilde{\delta }_{12}, \tilde{\delta }_{23}, \tilde{\delta }_{34}, \tilde{\delta }_{41}\) set to \(t^{(0)}_{i}, t^{(0)}_{j}, t^{(0)}_{k}, t^{(0)}_{\ell }, t_{i}, t_{j}, t_{k}, t_{\ell }, 1, \delta _{ij}, \delta _{jk}, \delta _{k\ell }, \delta _{\ell i}\), respectively.

  5. Lemma 2.2, part (i) should be applied with the values of \(t_1,t_2, t_3, t_4, v_1, v_2, v_3, v_4, \alpha , \tilde{\delta }_{12}, \tilde{\delta }_{23}, \tilde{\delta }_{34}, \tilde{\delta }_{41}\) set to \(t^{(0)}_{i}, t^{(0)}_{j}, t^{(0)}_{k}, t^{(0)}_{\ell }, t_{i}, t_{j}, t_{k}, t_{\ell }, 1, \delta _{ij}, \delta _{jk}, \delta _{k\ell }, \delta _{\ell i}\), respectively.

References

  1. Angluin, D., Valiant, L.G.: Fast probabilistic algorithms for Hamiltonian circuits and matchings. In: Proceedings of the Ninth Annual ACM Symposium on Theory of Computing (STOC’77), pp. 30–41. ACM, New York (1977)

  2. Arie-Nachimson, M., Kovalsky, S.Z., Kemelmacher-Shlizerman, I., Singer, A., Basri, R.: Global motion estimation from point matches. In: Second International Conference on 3D Imaging. Modeling, Processing, Visualization and Transmission (3DIMPVT’12), pp. 81–88. IEEE, Los Alamitos (2012)

  3. Candès, E.J., Romberg, J.K., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  4. Crandall, D., Owens, A., Snavely, N., Huttenlocher, D.: Discrete-continuous optimization for large-scale structure from motion. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11), pp. 3001–3008. IEEE, Los Alamitos (2011)

  5. Cui, Z., Jiang, N., Tang, C., Tan, P.: Linear global translation estimation with feature tracks. In: Xie, X. et al. (eds.) Proceedings of the 26th British Machine Vision Conference (BMVC), N 46. BMVA Press (2015)

  6. Cui, Z., Tan, P.: Global structure-from-motion by similarity averaging. In: IEEE International Conference on Computer Vision (ICCV’15), pp. 864–872. IEEE, Los Alamitos (2015)

  7. Diestel, R.: Graph Theory. Graduate Texts in Mathematics, vol. 173, 3rd edn. Springer, Berlin (2005)

    Google Scholar 

  8. Enqvist, O., Kahl, F., Olsson, C.: Non-sequential structure from motion. In: IEEE International Conference on Computer Vision Workshops (ICCV Workshops’11), pp. 264–271. IEEE, Los Alamitos (2011)

  9. Goldstein, T., Hand, P., Lee, C., Voroninski, V., Soatto, S.: ShapeFit and ShapeKick for Robust, scalable structure from motion. In: Leibe, B., et al. (eds.) Computer Vision—ECCV 2016, pp. 289–304. Springer, Berlin (2016)

    Google Scholar 

  10. Govindu, V.M.: Combining two-view constraints for motion estimation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’01), vol. 2, pp. 218–225. IEEE, Los Alamitos (2001)

  11. Govindu, V.M.: Lie-algebraic averaging for globally consistent motion estimation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’04), vol. 1, pp. 684–691. IEEE, Los Alamitos (2004)

  12. Hand, P., Lee, C., Voroninski, V.: ShapeFit: exact location recovery from corrupted pairwise directions. Comm. Pure Appl. Math. 71(1), 3–50 (2018). https://doi.org/10.1002/cpa.21727

    Article  MATH  Google Scholar 

  13. Hartley, R., Trumpf, J., Dai, Y., Li, H.: Rotation averaging. Int. J. Comput. Vis. 103(3), 267–305 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  14. Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004)

    Book  MATH  Google Scholar 

  15. Jiang, N., Cui, Z., Tan, P.: A global linear method for camera pose registration. In: IEEE International Conference on Computer Vision (ICCV’13), pp. 481–488. IEEE, Los Alamitos (2013)

  16. Kahl, F.: Multiple view geometry and the \(L^\infty \)-norm. In: Tenth IEEE International Conference on Computer Vision (ICCV’05), vol. 2, pp. 1002–1009. IEEE, Los Alamitos (2005)

  17. Kahl, F., Hartley, R.: Multiple-view geometry under the \(L_\infty \)-norm. IEEE Trans. Pattern Anal. Mach. Intell. 30(9), 1603–1617 (2008)

    Article  Google Scholar 

  18. Löfberg, J.: YALMIP : a toolbox for modeling and optimization in MATLAB. In: IEEE International Symposium on Computer Aided Control Systems Design (CACSD’04). IEEE, Los Alamitos (2004) http://users.isy.liu.se/johanl/yalmip

  19. Ma, Y., Soatto, S., Košecká, J., Sastry, S.S.: An Invitation to 3-D Vision: From Images to Geometric Models. Interdisciplinary Applied Mathematics, vol. 26. Springer, New York (2003)

    MATH  Google Scholar 

  20. Martinec, D., Pajdla, T.: Robust rotation and translation estimation in multiview reconstruction. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR’07), 8 pp. IEEE, Los Alamitos (2007)

  21. Moulon, P., Monasse, P., Marlet, R.: Global fusion of relative motions for robust, accurate and scalable structure from motion. In: IEEE International Conference on Computer Vision (ICCV’13), pp. 3248–3255. IEEE, Los Alamitos (2013)

  22. Özyesl, O., Singer, A., Basri, R.: Stable camera motion estimation using convex programming. SIAM J. Imaging Sci. 8(2), 1220–1262 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  23. Sim, K., Hartley, R.: Recovering camera motion using \(L^\infty \) minimization. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 1, pp. 1230–1237. IEEE, Los Alamitos (2006)

  24. Sim, K., Hartley, R.: Removing outliers using the Linfty norm. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 1, pp. 485–494. IEEE, Los Alamitos (2006)

  25. Sinha, S.N., Steedly, D., Szeliski, R.: A multi-stage linear approach to structure from motion. In: Kutulakos, K.N. (ed.) Trends and Topics in Computer Vision (ECCV’10), Part II, pp. 267–281. Lecture Notes in Computer Science, vol. 6554. Springer, Berlin (2010)

  26. Snavely, N., Seitz, S.M., Szeliski, R.: Skeletal graphs for efficient structure from motion. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR’08), vol. 1, 8 pp. IEEE, Los Alamitos (2008)

  27. Svärm, L., Oskarsson, M.: Structure from motion estimation with positional cues. In: Kämäräinen, J.-K., Koskela, M. (eds.) Image Analysis. Scandinavian Conference on Image Analysis (SCIA’13), pp. 522–532. Springer, Berlin (2013)

  28. Sweeney, C., Sattler, T., Höllerer, T., Turk, M., Pollefeys, M.: Optimizing the viewing graph for structure-from-motion. In: IEEE International Conference on Computer Vision (ICCV’15), pp. 801–809. IEEE, Los Alamitos (2015)

  29. Toh, K.C., Todd, M.J., Tütüncü, R.H.: SDPT3—a MATLAB software package for semidefinite programming, Version 1.3. Optim. Methods Softw. 11, 545–581 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  30. Tron, R., Vidal, R.: Distributed image-based 3-D localization of camera sensor networks. In: Proceedings of the 48th IEEE Conference on Decision and Control (CDC/CCC’09), pp. 901–908. IEEE, Los Alamitos (2009)

  31. Tron, R., Zhou, X., Daniilidis, K.: A survey on rotation optimization in structure from motion. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’16), pp. 1032–1040. IEEE, Los Alamitos (2016)

  32. Tütüncü, R.H., Toh, K.C., Todd, M.J.: Solving semidefinite–quadratic–linear programs using SDPT3. Math. Program. Ser. B 95(2), 189–217 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  33. Vershynin, R.: Introduction to the non-asymptotic analysis of random matrices. In: Eldar, Y.C., Kutyniok, G. (eds.) Compressed Sensing: Theory and Applications, pp. 210–268. Cambridge University Press, Cambridge (2012)

    Chapter  Google Scholar 

  34. Wang, L., Singer, A.: Exact and stable recovery of rotations for robust synchronization. Inf. Inference 2(2), 145–193 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  35. Wilson, K., Snavely, N.: Robust global translations with 1DSfM. In: Fleet, D., et al. (eds.) Computer Vision—ECCV 2014, Part III, pp. 61–75. Springer, Berlin (2014)

    Google Scholar 

  36. Zach, C., Klopschitz, M., Pollefeys, M.: Disambiguating visual relations using loop constraints. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’10), vol. 2, pp. 1426–1433. IEEE, Los Alamitos (2010)

Download references

Acknowledgements

We are grateful to Stefano Soatto for suggesting to VV the problem formulation addressed in this paper. VV is partially supported by the Office of Naval Research. CL is partially supported by the National Science Foundation Grant DMS-1362326. PH is partially supported by the National Science Foundation Grant DMS-1418971.

Author information

Authors and Affiliations

  1. Department of Mathematics, Massachusetts Institute of Technology, Cambridge, MA, 02139-4307, USA

    Choongbum Lee & Vladislav Voroninski

  2. Department of Computational and Applied Mathematics, Rice University, Houston, TX, 77005-1827, USA

    Paul Hand

Authors
  1. Paul Hand
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Choongbum Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vladislav Voroninski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vladislav Voroninski.

Additional information

Editor in Charge: Kenneth Clarkson

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hand, P., Lee, C. & Voroninski, V. Exact Simultaneous Recovery of Locations and Structure from Known Orientations and Corrupted Point Correspondences. Discrete Comput Geom 59, 413–450 (2018). https://doi.org/10.1007/s00454-017-9892-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00454-017-9892-9

Keywords

Mathematics Subject Classification

Search

Navigation