Skip to main content
SpringerLink
Log in

Robust Fitting in Computer Vision: Easy or Hard?

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Robust model fitting plays a vital role in computer vision, and research into algorithms for robust fitting continues to be active. Arguably the most popular paradigm for robust fitting in computer vision is consensus maximisation, which strives to find the model parameters that maximise the number of inliers. Despite the significant developments in algorithms for consensus maximisation, there has been a lack of fundamental analysis of the problem in the computer vision literature. In particular, whether consensus maximisation is “tractable” remains a question that has not been rigorously dealt with, thus making it difficult to assess and compare the performance of proposed algorithms, relative to what is theoretically achievable. To shed light on these issues, we present several computational hardness results for consensus maximisation. Our results underline the fundamental intractability of the problem, and resolve several ambiguities existing in the literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. It may be the case that MAXCON is in class APX, i.e., that it could be approximated in polynomial time to some factor. However, we are not aware of any such algorithms.

  2. Since RANSAC does not provide any approximation guarantees, it is not an “approximation scheme” by standard definition (Vazirani 2001).

  3. A dicussion between Tat-Jun Chin and Komei Fukuda in Feb 2018 suggested that a direct reduction from MaxFS to MAXCON is itself infeasible, due to the intractability of bounding a hyperplane arrangement (Fukuda et al. 1997, Theorem 5.1).

  4. If one was ever interested in only, say, robust affine registration of 2D point sets, then one could say that robust 2D affine registration is tractable, since the technique of Enqvist et al. (2012, 2015) can be used to construct an \(\mathcal {O}(N^7)\) algorithm, which is polynomial in the number of input correspondences N. However, robust fitting in general, where N and d can both vary, is not tractable by the reasons already alluded to above.

  5. https://en.wikipedia.org/wiki/Parameterized_complexity.

  6. This can be ensured by infinitesimal data perturbations (Matoušek 1995).

  7. https://en.wikipedia.org/wiki/Computational_complexity_theory#Measuring_the_size_of_an_instance.

  8. https://github.com/ZhipengCai/Demo---MAXCON-hardness.

  9. We do not report the result of Algorithm 1 or 2 since they are too slow even for small o and d (both algorithms did finish in 2 hours for one data instance when \(d > 7\) and \(o > 7\)).

  10. https://en.wikipedia.org/wiki/Single-precision_floating-point_format.

  11. https://en.wikipedia.org/wiki/2-satisfiability.

  12. https://en.wikipedia.org/wiki/L-reduction.

References

  • Amaldi, E., & Kann, V. (1995). The complexity and approximability of finding maximum feasible subsystems of linear relations. Theoretical Computer Science, 147, 181–210.

    Article  MathSciNet  Google Scholar 

  • Aronov, B., & Har-Peled, S. (2008). On approximating the depth and related problems. SIAM Journal on Computing, 38(3), 899–921.

    Article  MathSciNet  Google Scholar 

  • Bazin, J. C., Li, H., Kweon, I. S., Demonceaux, C., Vasseur, P., & Ikeuchi, K. (2013). A branch-and-bound approach to correspondence and grouping problems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(7), 1565–1576.

    Article  Google Scholar 

  • Ben-David, S., Eiron, N., & Simon, H. (2002). The computational complexity of densest region detection. Journal of Computer and System Sciences, 64(1), 22–47.

    Article  MathSciNet  Google Scholar 

  • Bernholt, T. (2005). Robust estimators are hard to compute. Technical report 52, Technische Universität.

  • Cai, Z., Chin, T. J., Le, H., Suter, D. (2018). Deterministic consensus maximization with biconvex programming. In European conference on computer vision (ECCV).

    Chapter  Google Scholar 

  • Campbell, D., Petersson, L., Kneip, L., Li, H. (2017). Globally-optimal inlier set maximisation for simultaneous camera pose and feature correspondence. In IEEE international conference on computer vision (ICCV).

  • Cheney, E. W. (1966). Introduction to approximation theory. New York: McGraw-Hill.

    MATH  Google Scholar 

  • Chin, T. J., Cai, Z., Neumann, F. (2018). Robust fitting in computer vision: easy or hard? In European conference on computer vision (ECCV).

  • Chin, T. J., Kee, Y. H., Eriksson, A., Neumann, F. (2016). Guaranteed outlier removal with mixed integer linear programs. In IEEE computer society conference on computer vision and pattern recognition (CVPR).

  • Chin, T. J., Purkait, P., Eriksson, A., Suter, D. (2015). Efficient globally optimal consensus maximisation with tree search. In IEEE computer society conference on computer vision and pattern recognition (CVPR).

  • Choi, S., Kim, T., Yu, W. (2009). Performance evaluation of RANSAC family. In British machine vision conference (BMVC).

  • Cygan, M., Fomin, F. V., Kowalik, Ł., Lokshtanov, D., Marx, D., Pilipczuk, M., et al. (2015). Parameterized algorithms (Vol. 3). Berlin: Springer.

    Book  Google Scholar 

  • Downey, R. G., & Fellows, M. R. (1999). Parametrized complexity. New York: Springer.

    Book  Google Scholar 

  • Enqvist, O., Ask, E., Kahl, F., & Åström, K. (2012). Robust fitting for multiple view geometry. In European conference on computer vision (ECCV).

  • Enqvist, O., Ask, E., Kahl, F., & Åström, K. (2015). Tractable algorithms for robust model estimation. International Journal of Computer Vision, 112(1), 115–129.

    Article  MathSciNet  Google Scholar 

  • Erickson, J., Har-Peled, S., & Mount, D. M. (2006). On the least median square problem. Discrete & Computational Geometry, 36(4), 593–607.

    Article  MathSciNet  Google Scholar 

  • Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.

    Article  MathSciNet  Google Scholar 

  • Fukuda, K., Liebling, T. M., & Margot, F. (1997). Analysis of backtrack algorithms for listing all vertices and all faces of a convex polyhedron. Computational Geometry, 8, 1–12.

    Article  MathSciNet  Google Scholar 

  • Garey, M. R., & Johnson, D. S. (1990). Computers and intractability: a guide to the theory of NP-completeness. New York: W H Freeman & Co.

    MATH  Google Scholar 

  • Geiger, A., Lenz, P., Urtasun, R. (2012). Are we ready for autonomous driving? the kitti vision benchmark suite. In Conference on computer vision and pattern recognition (CVPR).

  • Giannopoulos, P., Knauer, C., Rote, G.: The parameterized complexity of some geometric problems in unbounded dimension. In International workshop on parameterized and exact computation (IWPEC) (2009).

    Chapter  Google Scholar 

  • Hart, P. E., Nilsson, N. J., & Raphael, B. (1968). A formal basis for the heuristic determination of minimum cost paths. IEEE Transactions Systems Science and Cybernetics, 4(2), 100–107.

    Article  Google Scholar 

  • Hartley, R., & Zisserman, A. (2003). Multiple view geometry in computer vision. Cambridge: Cambridge University Press.

    MATH  Google Scholar 

  • Johnson, D. S. (1974). Approximation algorithms for combinatorial problems. Journal of Computer and System Sciences, 9, 256–278.

    Article  MathSciNet  Google Scholar 

  • Johnson, D. S., & Preparata, F. P. (1978). The densest hemisphere problem. Theoretical Computer Science, 6, 93–107.

    Article  MathSciNet  Google Scholar 

  • Le, H., Chin, T. J., Suter, D. (2017). An exact penalty method for locally convergent maximum consensus. In IEEE computer society conference on computer vision and pattern recognition (CVPR).

  • Li, H. (2009). Consensus set maximization with guaranteed global optimality for robust geometry estimation. In: IEEE international conference on computer vision (ICCV).

  • Lowe, D. G. (1999) Object recognition from local scale-invariant features. In The proceedings of the seventh IEEE international conference on computer vision, 1999 (Vol. 2, pp. 1150–1157). IEEE

  • Matoušek, J. (1995). On geometric optimization with few violated constraints. Discrete and Computational Geometry, 14(4), 365–384.

    Article  MathSciNet  Google Scholar 

  • Meer, P. (2004). Robust techniques for computer vision. In G. Medioni & S. B. Kang (Eds.), Emerging topics in computer vision. New York: Prentice Hall.

    Google Scholar 

  • Parra Bustos, A., Chin, T. J., Suter, D.: Fast rotation search with stereographic projections for 3d registration. In IEEE computer society conference on computer vision and pattern recognition (CVPR) (2014)

  • Parra Bustos, A., Chin, T. J. (2015). Guaranteed outlier removal for rotation search. In IEEE international conference on computer vision (ICCV).

  • Purkait, P., Zach, C., Eriksson, A. (2017) Maximum consensus parameter estimation by reweighted L1 methods. In Energy minimization methods in computer vision and pattern recognition (EMMCVPR).

  • Raguram, R., Chum, O., Pollefeys, M., Matas, J., & Frahm, J. M. (2013). USAC: A universal framework for random sample consensus. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 2022–2038.

    Article  Google Scholar 

  • Svärm, L., Enqvist, O., Oskarsson, M., Kahl, F. (2014). Accurate localization and pose estimation for large 3d models. In IEEE computer society conference on computer vision and pattern recognition (CVPR)

  • Tran, Q. H., Chin, T. J., Chojnacki, W., & Suter, D. (2014). Sampling minimal subsets with large spans for robust estimation. International Journal of Computer Vision (IJCV), 106(1), 93–112.

    Article  MathSciNet  Google Scholar 

  • Vazirani, V. (2001). Approximation algorithms. Berlin: Springer.

    MATH  Google Scholar 

  • Vedaldi, A., Fulkerson, B. (2010). Vlfeat: An open and portable library of computer vision algorithms. In Proceedings of the 18th ACM international conference on Multimedia (pp. 1469–1472). ACM.

  • Yang, J., Li, H., Jia, Y.: Optimal essential matrix estimation via inlier-set maximization. In European conference on computer vision (ECCV)

  • Zheng, Y., Sugimoto, S., Okutomi, M.: Deterministically maximizing feasible subsystems for robust model fitting with unit norm constraints. In IEEE computer society conference on computer vision and pattern recognition (CVPR) (2011)

Download references

Acknowledgements

This work was supported by ARC Grant DP160103490.

Author information

Authors and Affiliations

  1. The University of Adelaide, Adelaide, Australia

    Tat-Jun Chin, Zhipeng Cai & Frank Neumann

Authors
  1. Tat-Jun Chin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhipeng Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Frank Neumann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tat-Jun Chin.

Additional information

Communicated by Yair Weiss.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chin, TJ., Cai, Z. & Neumann, F. Robust Fitting in Computer Vision: Easy or Hard?. Int J Comput Vis 128, 575–587 (2020). https://doi.org/10.1007/s11263-019-01207-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-019-01207-y

Keywords

Search

Navigation