Journal of Global Optimization

, Volume 73, Issue 2, pp 311–330 | Cite as

A fast algorithm for globally solving Tikhonov regularized total least squares problem

  • Yong Xia
  • Longfei Wang
  • Meijia YangEmail author


The total least squares problem with the general Tikhonov regularization can be reformulated as a one-dimensional parametric minimization problem (PM), where each parameterized function evaluation corresponds to solving an n-dimensional trust region subproblem. Under a mild assumption, the parametric function is differentiable and then an efficient bisection method has been proposed for solving (PM) in literature. In the first part of this paper, we show that the bisection algorithm can be greatly improved by reducing the initially estimated interval covering the optimal parameter. It is observed that the bisection method cannot guarantee to find the globally optimal solution since the nonconvex (PM) could have a local non-global minimizer. The main contribution of this paper is to propose an efficient branch-and-bound algorithm for globally solving (PM), based on a new underestimation of the parametric function over any given interval using only the information of the parametric function evaluations at the two endpoints. We can show that the new algorithm (BTD Algorithm) returns a global \(\epsilon \)-approximation solution in a computational effort of at most \(O(n^3/\sqrt{\epsilon })\) under the same assumption as in the bisection method. The numerical results demonstrate that our new global optimization algorithm performs even much faster than the improved version of the bisection heuristic algorithm.


Total least squares Tikhonov regularization Trust region subproblem Fractional program Lower bound Branch and bound 

Mathematics Subject Classification

65F20 90C26 90C32 90C20 



The authors are grateful to the two anonymous referees for their valuable comments and suggestions.


  1. 1.
    Beck, A., Ben-Tal, A.: On the solution of the Tikhonov regularization of the total least squares problem. SIAM J. Optim. 17(3), 98–118 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Beck, A., Ben-Tal, A., Teboulle, M.: Finding a global optimal solution for a quadratically constrained fractional quadratic problem with applications to the regularized total least squares. SIAM J. Matrix Anal. Appl. 28(2), 425–445 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Beck, A., Teboulle, M.: A convex optimization approach for minimizing the ratio of indefinite quadratic functions over an ellipsoid. Math. Program. 118(1), 13–35 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Conn, A.R., Gould, N.I.M., Toint, P.L.: Trust-Region Methods. MPS/SIAM Series on Optimization. SIAM, Philadelphia (2000)CrossRefzbMATHGoogle Scholar
  5. 5.
    Falk, J.E., Soland, R.M.: An algorithm for separable nonconvex programming problems. Manag. Sci. 15, 550–569 (1969)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Fortin, C., Wolkowicz, H.: The trust region subproblem and semidefinite programming. Optim. Methods Softw. 19, 41–67 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Gander, W., Golub, G.H., von Matt, U.: A constrained eigenvalue problem. Linear Algebra Appl. 114(115), 815–839 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Gay, D.M.: Computing optimal locally constrained steps. SIAM J. Sci. Stat. Comput. 2(2), 186–197 (1981)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Golub, G.H., Van Loan, C.F.: An analysis of the total least-squares problem. SIAM J. Numer. Anal. 17(6), 883–893 (1980)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. The Johns Hopkins University Press, Baltimore (1996)zbMATHGoogle Scholar
  11. 11.
    Hansen, P.C.: Regularization tools: a Matlab package for analysis and solution of discrete ill-posed problems. Numer. Algorithm 6, 1–35 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Hansen, P.C., O’Leary, D.P.: The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J. Sci. Comput. 14, 1487–1503 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Jain, A.K.: Fundamentals of Digital Image Processing. Prentice-Hall, Englewood Cliffs (1989)zbMATHGoogle Scholar
  14. 14.
    Joerg, L., Heinrich, V.: Large-scale Tikhonov regularization of total least squares. J. Comput. Appl. Math. 238, 95–108 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Lbaraki, T., Schaible, S.: Fractional programming. Eur. J. Oper. Res. 12(4), 325–338 (2004)MathSciNetGoogle Scholar
  16. 16.
    Moré, J.J., Sorensen, D.C.: Computing a trust region step. SIAM J. Sci. Stat. Comput. 4(3), 553–572 (1983)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Moré, J.J.: Generalizations of the trust region problem. Optim. Methods Softw. 2, 189–209 (1993)CrossRefGoogle Scholar
  18. 18.
    Pong, T.K., Wolkowicz, H.: Generalizations of the trust region subproblem. Comput. Optim. Appl. 58(2), 273–322 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Rendel, F., Wolkowicz, H.: A semidefinite framework for trust region subproblems with applications to large scale minimization. Math. Program. 77(2), 273–299 (1997)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Schaible, S., Shi, J.M.: Fractional programming: the sum-of-ratios case. Optim. Methods Softw. 18(2), 219–229 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Sorensen, D.C.: Minimization of a large-scale quadratic function subject to a spherical constraint. SIAM J. Optim. 7(1), 141–161 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Tikhonov, A.N., Arsenin, V.Y.: Solution of Ill-Posed Problems. V.H. Winston, Washington (1977)zbMATHGoogle Scholar
  23. 23.
    Van Huffel, S., Lemmerling, P.: Total Least Squares and Errors-in-Variables Modeling. Kluwer, Dordrecht (2002)CrossRefzbMATHGoogle Scholar
  24. 24.
    Van Huffel, S., Vandewalle, J.: The Total Least Squares Problem: Computational Aspects and Analysis. Frontiers in Applied Mathematics, vol. 9. SIAM, Philadelphia (1991)CrossRefzbMATHGoogle Scholar
  25. 25.
    Xia, Y., Wang, S., Sheu, R.L.: S-lemma with equality and its applications. Math. Program. Ser. A 156(1–2), 513–547 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Yang, M., Xia, Y., Wang, J., Peng, J.: Efficiently solving total least squares with Tikhonov identical regularization. Comput. Optim. Appl. 70(2), 571–592 (2018)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.State Key Laboratory of Software Development Environment, LMIB of the Ministry of Education, School of Mathematics and System SciencesBeihang UniversityBeijingPeople’s Republic of China

Personalised recommendations