Advertisement

Springer Nature is making Coronavirus research free. View research | View latest news | Sign up for updates

Exact Simultaneous Recovery of Locations and Structure from Known Orientations and Corrupted Point Correspondences

Abstract

Let \(t_1,\ldots ,t_{n_l} \in {\mathbb {R}}^d\) and \(p_1,\ldots ,p_{n_s} \in {\mathbb {R}}^d\) and consider the bipartite location recovery problem: given a subset of pairwise direction observations \(\{(t_i - p_j) / \Vert t_i - p_j\Vert _2\}_{i,j \in [n_\ell ] \times [n_\text {s}]}\), where a constant fraction of these observations are arbitrarily corrupted, find \(\{t_i\}_{i \in [n_\ell ]}\) and \(\{p_j\}_{j \in [n_\text {s}]}\) up to a global translation and scale. This task arises in the Structure from Motion problem from computer vision, which consists of recovering the three-dimensional structure of a scene from photographs at unknown vantage points. We study the recently introduced ShapeFit algorithm as a method for solving this bipartite location recovery problem. In this case, ShapeFit consists of a simple convex program over \(d(n_l + n_s)\) real variables. We prove that this program recovers a set of \(n_l+n_s\) i.i.d. Gaussian locations exactly and with high probability if the observations are given by a bipartite Erdős–Rényi graph, d is large enough, and provided that at most a constant fraction of observations involving any particular location are adversarially corrupted. This recovery theorem is based on a set of deterministic conditions that we prove are sufficient for exact recovery. Finally, we propose a modified pipeline for the Structure for Motion problem, based on this bipartite location recovery problem.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3

Notes

  1. 1.

    That is, E can be partitioned into the disjoint union of \(E_g\) and \(E_b\).

  2. 2.

    G is an undirected graph with edge set E over the vertices \(V_\ell \cup V_\text {s}\). For each \(i \in V_\ell \) and \(j \in V_\text {s}\), the edge (ij) is present in E with probability p, independent from other edges. No edges in \(V_\ell \times V_\ell \) or \(V_\text {s}\times V_\text {s}\) are present in E.

  3. 3.

    To see this, apply Lemma 2.2 part (i) with \(t_1,t_2, t_3, t_4, v_1, v_2, v_3, v_4, \alpha , \tilde{\delta }_{12}, \tilde{\delta }_{23}, \tilde{\delta }_{34}, \tilde{\delta }_{41}\) set to \(t^{(0)}_{a}, t^{(0)}_{b}, t^{(0)}_{c}, t^{(0)}_{d}, t_{a}, t_{b}, t_{c}, t_{d}, 1, \delta _{ab}, \delta _{bc}, \delta _{cd}, \delta _{da}\), respectively. By applying Condition 2, one obtains \(\eta (ab,cd) \ge \beta \Vert t^{(0)}_{ab}\Vert _2 |\delta _{ab} - \delta _{cd}|\). By Condition 2 and the fact that \(\delta _{ab}>\delta _{cd}\), one concludes \(\eta (ab,cd) > 0\).

  4. 4.

    Lemma 2.2, part (i) should be applied with the values of \(t_1,t_2, t_3, t_4, v_1, v_2, v_3, v_4, \alpha , \tilde{\delta }_{12}, \tilde{\delta }_{23}, \tilde{\delta }_{34}, \tilde{\delta }_{41}\) set to \(t^{(0)}_{i}, t^{(0)}_{j}, t^{(0)}_{k}, t^{(0)}_{\ell }, t_{i}, t_{j}, t_{k}, t_{\ell }, 1, \delta _{ij}, \delta _{jk}, \delta _{k\ell }, \delta _{\ell i}\), respectively.

  5. 5.

    Lemma 2.2, part (i) should be applied with the values of \(t_1,t_2, t_3, t_4, v_1, v_2, v_3, v_4, \alpha , \tilde{\delta }_{12}, \tilde{\delta }_{23}, \tilde{\delta }_{34}, \tilde{\delta }_{41}\) set to \(t^{(0)}_{i}, t^{(0)}_{j}, t^{(0)}_{k}, t^{(0)}_{\ell }, t_{i}, t_{j}, t_{k}, t_{\ell }, 1, \delta _{ij}, \delta _{jk}, \delta _{k\ell }, \delta _{\ell i}\), respectively.

References

  1. 1.

    Angluin, D., Valiant, L.G.: Fast probabilistic algorithms for Hamiltonian circuits and matchings. In: Proceedings of the Ninth Annual ACM Symposium on Theory of Computing (STOC’77), pp. 30–41. ACM, New York (1977)

  2. 2.

    Arie-Nachimson, M., Kovalsky, S.Z., Kemelmacher-Shlizerman, I., Singer, A., Basri, R.: Global motion estimation from point matches. In: Second International Conference on 3D Imaging. Modeling, Processing, Visualization and Transmission (3DIMPVT’12), pp. 81–88. IEEE, Los Alamitos (2012)

  3. 3.

    Candès, E.J., Romberg, J.K., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)

  4. 4.

    Crandall, D., Owens, A., Snavely, N., Huttenlocher, D.: Discrete-continuous optimization for large-scale structure from motion. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11), pp. 3001–3008. IEEE, Los Alamitos (2011)

  5. 5.

    Cui, Z., Jiang, N., Tang, C., Tan, P.: Linear global translation estimation with feature tracks. In: Xie, X. et al. (eds.) Proceedings of the 26th British Machine Vision Conference (BMVC), N 46. BMVA Press (2015)

  6. 6.

    Cui, Z., Tan, P.: Global structure-from-motion by similarity averaging. In: IEEE International Conference on Computer Vision (ICCV’15), pp. 864–872. IEEE, Los Alamitos (2015)

  7. 7.

    Diestel, R.: Graph Theory. Graduate Texts in Mathematics, vol. 173, 3rd edn. Springer, Berlin (2005)

  8. 8.

    Enqvist, O., Kahl, F., Olsson, C.: Non-sequential structure from motion. In: IEEE International Conference on Computer Vision Workshops (ICCV Workshops’11), pp. 264–271. IEEE, Los Alamitos (2011)

  9. 9.

    Goldstein, T., Hand, P., Lee, C., Voroninski, V., Soatto, S.: ShapeFit and ShapeKick for Robust, scalable structure from motion. In: Leibe, B., et al. (eds.) Computer Vision—ECCV 2016, pp. 289–304. Springer, Berlin (2016)

  10. 10.

    Govindu, V.M.: Combining two-view constraints for motion estimation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’01), vol. 2, pp. 218–225. IEEE, Los Alamitos (2001)

  11. 11.

    Govindu, V.M.: Lie-algebraic averaging for globally consistent motion estimation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’04), vol. 1, pp. 684–691. IEEE, Los Alamitos (2004)

  12. 12.

    Hand, P., Lee, C., Voroninski, V.: ShapeFit: exact location recovery from corrupted pairwise directions. Comm. Pure Appl. Math. 71(1), 3–50 (2018). https://doi.org/10.1002/cpa.21727

  13. 13.

    Hartley, R., Trumpf, J., Dai, Y., Li, H.: Rotation averaging. Int. J. Comput. Vis. 103(3), 267–305 (2013)

  14. 14.

    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004)

  15. 15.

    Jiang, N., Cui, Z., Tan, P.: A global linear method for camera pose registration. In: IEEE International Conference on Computer Vision (ICCV’13), pp. 481–488. IEEE, Los Alamitos (2013)

  16. 16.

    Kahl, F.: Multiple view geometry and the \(L^\infty \)-norm. In: Tenth IEEE International Conference on Computer Vision (ICCV’05), vol. 2, pp. 1002–1009. IEEE, Los Alamitos (2005)

  17. 17.

    Kahl, F., Hartley, R.: Multiple-view geometry under the \(L_\infty \)-norm. IEEE Trans. Pattern Anal. Mach. Intell. 30(9), 1603–1617 (2008)

  18. 18.

    Löfberg, J.: YALMIP : a toolbox for modeling and optimization in MATLAB. In: IEEE International Symposium on Computer Aided Control Systems Design (CACSD’04). IEEE, Los Alamitos (2004) http://users.isy.liu.se/johanl/yalmip

  19. 19.

    Ma, Y., Soatto, S., Košecká, J., Sastry, S.S.: An Invitation to 3-D Vision: From Images to Geometric Models. Interdisciplinary Applied Mathematics, vol. 26. Springer, New York (2003)

  20. 20.

    Martinec, D., Pajdla, T.: Robust rotation and translation estimation in multiview reconstruction. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR’07), 8 pp. IEEE, Los Alamitos (2007)

  21. 21.

    Moulon, P., Monasse, P., Marlet, R.: Global fusion of relative motions for robust, accurate and scalable structure from motion. In: IEEE International Conference on Computer Vision (ICCV’13), pp. 3248–3255. IEEE, Los Alamitos (2013)

  22. 22.

    Özyesl, O., Singer, A., Basri, R.: Stable camera motion estimation using convex programming. SIAM J. Imaging Sci. 8(2), 1220–1262 (2015)

  23. 23.

    Sim, K., Hartley, R.: Recovering camera motion using \(L^\infty \) minimization. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 1, pp. 1230–1237. IEEE, Los Alamitos (2006)

  24. 24.

    Sim, K., Hartley, R.: Removing outliers using the Linfty norm. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 1, pp. 485–494. IEEE, Los Alamitos (2006)

  25. 25.

    Sinha, S.N., Steedly, D., Szeliski, R.: A multi-stage linear approach to structure from motion. In: Kutulakos, K.N. (ed.) Trends and Topics in Computer Vision (ECCV’10), Part II, pp. 267–281. Lecture Notes in Computer Science, vol. 6554. Springer, Berlin (2010)

  26. 26.

    Snavely, N., Seitz, S.M., Szeliski, R.: Skeletal graphs for efficient structure from motion. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR’08), vol. 1, 8 pp. IEEE, Los Alamitos (2008)

  27. 27.

    Svärm, L., Oskarsson, M.: Structure from motion estimation with positional cues. In: Kämäräinen, J.-K., Koskela, M. (eds.) Image Analysis. Scandinavian Conference on Image Analysis (SCIA’13), pp. 522–532. Springer, Berlin (2013)

  28. 28.

    Sweeney, C., Sattler, T., Höllerer, T., Turk, M., Pollefeys, M.: Optimizing the viewing graph for structure-from-motion. In: IEEE International Conference on Computer Vision (ICCV’15), pp. 801–809. IEEE, Los Alamitos (2015)

  29. 29.

    Toh, K.C., Todd, M.J., Tütüncü, R.H.: SDPT3—a MATLAB software package for semidefinite programming, Version 1.3. Optim. Methods Softw. 11, 545–581 (1998)

  30. 30.

    Tron, R., Vidal, R.: Distributed image-based 3-D localization of camera sensor networks. In: Proceedings of the 48th IEEE Conference on Decision and Control (CDC/CCC’09), pp. 901–908. IEEE, Los Alamitos (2009)

  31. 31.

    Tron, R., Zhou, X., Daniilidis, K.: A survey on rotation optimization in structure from motion. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’16), pp. 1032–1040. IEEE, Los Alamitos (2016)

  32. 32.

    Tütüncü, R.H., Toh, K.C., Todd, M.J.: Solving semidefinite–quadratic–linear programs using SDPT3. Math. Program. Ser. B 95(2), 189–217 (2003)

  33. 33.

    Vershynin, R.: Introduction to the non-asymptotic analysis of random matrices. In: Eldar, Y.C., Kutyniok, G. (eds.) Compressed Sensing: Theory and Applications, pp. 210–268. Cambridge University Press, Cambridge (2012)

  34. 34.

    Wang, L., Singer, A.: Exact and stable recovery of rotations for robust synchronization. Inf. Inference 2(2), 145–193 (2013)

  35. 35.

    Wilson, K., Snavely, N.: Robust global translations with 1DSfM. In: Fleet, D., et al. (eds.) Computer Vision—ECCV 2014, Part III, pp. 61–75. Springer, Berlin (2014)

  36. 36.

    Zach, C., Klopschitz, M., Pollefeys, M.: Disambiguating visual relations using loop constraints. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’10), vol. 2, pp. 1426–1433. IEEE, Los Alamitos (2010)

Download references

Acknowledgements

We are grateful to Stefano Soatto for suggesting to VV the problem formulation addressed in this paper. VV is partially supported by the Office of Naval Research. CL is partially supported by the National Science Foundation Grant DMS-1362326. PH is partially supported by the National Science Foundation Grant DMS-1418971.

Author information

Correspondence to Vladislav Voroninski.

Additional information

Editor in Charge: Kenneth Clarkson

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hand, P., Lee, C. & Voroninski, V. Exact Simultaneous Recovery of Locations and Structure from Known Orientations and Corrupted Point Correspondences. Discrete Comput Geom 59, 413–450 (2018). https://doi.org/10.1007/s00454-017-9892-9

Download citation

Keywords

  • Structure from Motion
  • Corruption robust recovery
  • Convex programming

Mathematics Subject Classification

  • 94B99
  • 60C05
  • 68R05
  • 90C25