Skip to main content
Log in

A fast algorithm for globally solving Tikhonov regularized total least squares problem

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

The total least squares problem with the general Tikhonov regularization can be reformulated as a one-dimensional parametric minimization problem (PM), where each parameterized function evaluation corresponds to solving an n-dimensional trust region subproblem. Under a mild assumption, the parametric function is differentiable and then an efficient bisection method has been proposed for solving (PM) in literature. In the first part of this paper, we show that the bisection algorithm can be greatly improved by reducing the initially estimated interval covering the optimal parameter. It is observed that the bisection method cannot guarantee to find the globally optimal solution since the nonconvex (PM) could have a local non-global minimizer. The main contribution of this paper is to propose an efficient branch-and-bound algorithm for globally solving (PM), based on a new underestimation of the parametric function over any given interval using only the information of the parametric function evaluations at the two endpoints. We can show that the new algorithm (BTD Algorithm) returns a global \(\epsilon \)-approximation solution in a computational effort of at most \(O(n^3/\sqrt{\epsilon })\) under the same assumption as in the bisection method. The numerical results demonstrate that our new global optimization algorithm performs even much faster than the improved version of the bisection heuristic algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Beck, A., Ben-Tal, A.: On the solution of the Tikhonov regularization of the total least squares problem. SIAM J. Optim. 17(3), 98–118 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  2. Beck, A., Ben-Tal, A., Teboulle, M.: Finding a global optimal solution for a quadratically constrained fractional quadratic problem with applications to the regularized total least squares. SIAM J. Matrix Anal. Appl. 28(2), 425–445 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  3. Beck, A., Teboulle, M.: A convex optimization approach for minimizing the ratio of indefinite quadratic functions over an ellipsoid. Math. Program. 118(1), 13–35 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  4. Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust-Region Methods. MPS/SIAM Series on Optimization. SIAM, Philadelphia (2000)

    Book  MATH  Google Scholar 

  5. Falk, J.E., Soland, R.M.: An algorithm for separable nonconvex programming problems. Manag. Sci. 15, 550–569 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  6. Fortin, C., Wolkowicz, H.: The trust region subproblem and semidefinite programming. Optim. Methods Softw. 19, 41–67 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  7. Gander, W., Golub, G.H., von Matt, U.: A constrained eigenvalue problem. Linear Algebra Appl. 114(115), 815–839 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  8. Gay, D.M.: Computing optimal locally constrained steps. SIAM J. Sci. Stat. Comput. 2(2), 186–197 (1981)

    Article  MathSciNet  MATH  Google Scholar 

  9. Golub, G.H., Van Loan, C.F.: An analysis of the total least-squares problem. SIAM J. Numer. Anal. 17(6), 883–893 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  10. Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. The Johns Hopkins University Press, Baltimore (1996)

    MATH  Google Scholar 

  11. Hansen, P.C.: Regularization tools: a Matlab package for analysis and solution of discrete ill-posed problems. Numer. Algorithm 6, 1–35 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  12. Hansen, P.C., O’Leary, D.P.: The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J. Sci. Comput. 14, 1487–1503 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  13. Jain, A.K.: Fundamentals of Digital Image Processing. Prentice-Hall, Englewood Cliffs (1989)

    MATH  Google Scholar 

  14. Joerg, L., Heinrich, V.: Large-scale Tikhonov regularization of total least squares. J. Comput. Appl. Math. 238, 95–108 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  15. Lbaraki, T., Schaible, S.: Fractional programming. Eur. J. Oper. Res. 12(4), 325–338 (2004)

    MathSciNet  Google Scholar 

  16. Moré, J.J., Sorensen, D.C.: Computing a trust region step. SIAM J. Sci. Stat. Comput. 4(3), 553–572 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  17. Moré, J.J.: Generalizations of the trust region problem. Optim. Methods Softw. 2, 189–209 (1993)

    Article  Google Scholar 

  18. Pong, T.K., Wolkowicz, H.: Generalizations of the trust region subproblem. Comput. Optim. Appl. 58(2), 273–322 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  19. Rendel, F., Wolkowicz, H.: A semidefinite framework for trust region subproblems with applications to large scale minimization. Math. Program. 77(2), 273–299 (1997)

    MathSciNet  MATH  Google Scholar 

  20. Schaible, S., Shi, J.M.: Fractional programming: the sum-of-ratios case. Optim. Methods Softw. 18(2), 219–229 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  21. Sorensen, D.C.: Minimization of a large-scale quadratic function subject to a spherical constraint. SIAM J. Optim. 7(1), 141–161 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  22. Tikhonov, A.N., Arsenin, V.Y.: Solution of Ill-Posed Problems. V.H. Winston, Washington (1977)

    MATH  Google Scholar 

  23. Van Huffel, S., Lemmerling, P.: Total Least Squares and Errors-in-Variables Modeling. Kluwer, Dordrecht (2002)

    Book  MATH  Google Scholar 

  24. Van Huffel, S., Vandewalle, J.: The Total Least Squares Problem: Computational Aspects and Analysis. Frontiers in Applied Mathematics, vol. 9. SIAM, Philadelphia (1991)

    Book  MATH  Google Scholar 

  25. Xia, Y., Wang, S., Sheu, R.L.: S-lemma with equality and its applications. Math. Program. Ser. A 156(1–2), 513–547 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  26. Yang, M., Xia, Y., Wang, J., Peng, J.: Efficiently solving total least squares with Tikhonov identical regularization. Comput. Optim. Appl. 70(2), 571–592 (2018)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors are grateful to the two anonymous referees for their valuable comments and suggestions.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Meijia Yang.

Additional information

This research was supported by NSFC under Grants 11571029, 11471325 and 11771056, and by fundamental research funds for the Central Universities under Grant YWF-18-BJ-Y-16.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xia, Y., Wang, L. & Yang, M. A fast algorithm for globally solving Tikhonov regularized total least squares problem. J Glob Optim 73, 311–330 (2019). https://doi.org/10.1007/s10898-018-0719-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-018-0719-x

Keywords

Mathematics Subject Classification

Navigation