Computational Optimization and Applications

, Volume 70, Issue 2, pp 571–592 | Cite as

Efficiently solving total least squares with Tikhonov identical regularization

  • Meijia Yang
  • Yong Xia
  • Jiulin Wang
  • Jiming Peng


The Tikhonov identical regularized total least squares (TI) is to deal with the ill-conditioned system of linear equations where the data are contaminated by noise. A standard approach for (TI) is to reformulate it as a problem of finding a zero point of some decreasing concave non-smooth univariate function such that the classical bisection search and Dinkelbach’s method can be applied. In this paper, by exploring the hidden convexity of (TI), we reformulate it as a new problem of finding a zero point of a strictly decreasing, smooth and concave univariate function. This allows us to apply the classical Newton’s method to the reformulated problem, which converges globally to the unique root with an asymptotic quadratic convergence rate. Moreover, in every iteration of Newton’s method, no optimization subproblem such as the extended trust-region subproblem is needed to evaluate the new univariate function value as it has an explicit expression. Promising numerical results based on the new algorithm are reported.


Fractional programming Quadratic programming Total least square Tikhonov regularization Bisection method Newton’s method Trust-region subproblem S-lemma 

Mathematics Subject Classification

90C26 90C20 90C32 


  1. 1.
    Beck, A., Ben-Tal, A.: On the solution of the Tikhonov regularization of the total least squares problem. SIAM J. Optim. 17(1), 98–118 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Beck, A., Teboulle, M.: A convex optimization approach for minimizing the ratio of indefinite quadratic functions over an ellipsoid. Math. Program. Ser. A. 118(1), 13–15 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Beck, A., Ben-Tal, A., Teboulle, M.: Finding a global optimal solution for a quadratically constrained fractional quadratic problem with applications to the regularized total least squares. SIAM J. Matrix Anal. Appl. 28(2), 425–445 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Charnes, A., Cooper, W.W.: Programming with linear fractional functionals. Nav. Res. Logist. Q. 9, 181–186 (1962)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Dinkelbach, W.: On nonlinear fractional programming. Manag. Sci. 13(7), 492–498 (1967)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Gol’stein, E.G.: Dual problems of convex and fractionally-convex programming in functional spaces. Sov. Math. Dokl. 8, 212–216 (1967)Google Scholar
  7. 7.
    Golub, G.H., Van Loan, C.F.: An analysis of the total least-squares problem. SIAM J. Numer. Anal. 17(6), 883–893 (1980)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. The Johns Hopkins University Press, Baltimore (1996)zbMATHGoogle Scholar
  9. 9.
    Hansen, P.C.: Regularization tools: a Matlab package for analysis and solution of discrete ill-posed problems. Numer. Algorithm 6(1), 1–35 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Hansen, P.C., O’Leary, D.P.: The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J. Sci. Comput. 14(6), 1487–1503 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Hebden, M.D.: An algorithm for minimization using exact second derivatives. Int. J. Distrib. Sens. Netw. 2014(3), 1–8 (1973)Google Scholar
  12. 12.
    Jagannathan, R.: On some properities of programming problems in parametric form pertaining to fractional programming. Manag. Sci. 12, 609–615 (1966)CrossRefzbMATHGoogle Scholar
  13. 13.
    Jain, A.K.: Fundamentals of Digital Image Processing. Prentice-Hall, Englewood Cliffs (1989)zbMATHGoogle Scholar
  14. 14.
    Lampe, J., Voss, H.: Large-scale Tikhonov regularization of total least squares. J. Comput. Appl. Math. 238, 95–108 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Moosaei, H., Ketabchi, S., Pardalos, P.M.: Tikhonov regularization for infeasible absolute value equations. Optimization 65(8), 1531–1537 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Moré, J.J.: Generalization of the trust region problem. Optim. Methods Softw. 2(3), 189–209 (1993)CrossRefGoogle Scholar
  17. 17.
    Renaut, R.A., Guo, H.: Efficient algorithms for solution of regularized total least squares. SIAM J. Matrix Anal. Appl. 26(2), 457–476 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Schaible, S.: Fractional programming II: on Dinkelbach’s algorithm. Manag. Sci. 22(8), 868–873 (1976)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Sima, D., Van Huffel, S., Golub, G.H.: Regularized total least squares based on quadratic eigenvalue problem solvers. BIT 44, 793–812 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Tikhonov, A.N., Arsenin, V.Y.: Solution of Ill-Posed Problems. V.H. Winston, Washington (1977)zbMATHGoogle Scholar
  21. 21.
    Van Huffel, S., Lemmerling, P.: Total Least Squares and Errors-in-Variables Modeling. Kluwer Academic Publishers, Dordrecht (2002)CrossRefzbMATHGoogle Scholar
  22. 22.
    Van Huffel, S., Vandewalle, J.: The Total Least Squares Problem: Computational Aspects and Analysis. Frontiers in Applied Mathematics. SIAM, Philadelphia (1991)CrossRefzbMATHGoogle Scholar
  23. 23.
    Xia, Y., Wang, S., Sheu, R.L.: S-Lemma with equality and its applications. Math. Program. Ser. A. 156(1), 513–547 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  24. 24.
    Zhang, A., Hayashi, S.: Celis–Dennis–Tapia based approach to quadratic fractional programming problems with two quadratic constraints. Numer. Algebra Control Optim. 1(1), 83–98 (2011)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.LMIB of the Ministry of Education, School of Mathematics and System SciencesBeihang UniversityBeijingPeople’s Republic of China
  2. 2.Department of Industrial EngineeringUniversity of HoustonHoustonUSA

Personalised recommendations