Skip to main content
SpringerLink
Log in

Efficiently solving total least squares with Tikhonov identical regularization

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

The Tikhonov identical regularized total least squares (TI) is to deal with the ill-conditioned system of linear equations where the data are contaminated by noise. A standard approach for (TI) is to reformulate it as a problem of finding a zero point of some decreasing concave non-smooth univariate function such that the classical bisection search and Dinkelbach’s method can be applied. In this paper, by exploring the hidden convexity of (TI), we reformulate it as a new problem of finding a zero point of a strictly decreasing, smooth and concave univariate function. This allows us to apply the classical Newton’s method to the reformulated problem, which converges globally to the unique root with an asymptotic quadratic convergence rate. Moreover, in every iteration of Newton’s method, no optimization subproblem such as the extended trust-region subproblem is needed to evaluate the new univariate function value as it has an explicit expression. Promising numerical results based on the new algorithm are reported.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Beck, A., Ben-Tal, A.: On the solution of the Tikhonov regularization of the total least squares problem. SIAM J. Optim. 17(1), 98–118 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  2. Beck, A., Teboulle, M.: A convex optimization approach for minimizing the ratio of indefinite quadratic functions over an ellipsoid. Math. Program. Ser. A. 118(1), 13–15 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  3. Beck, A., Ben-Tal, A., Teboulle, M.: Finding a global optimal solution for a quadratically constrained fractional quadratic problem with applications to the regularized total least squares. SIAM J. Matrix Anal. Appl. 28(2), 425–445 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  4. Charnes, A., Cooper, W.W.: Programming with linear fractional functionals. Nav. Res. Logist. Q. 9, 181–186 (1962)

    Article  MathSciNet  MATH  Google Scholar 

  5. Dinkelbach, W.: On nonlinear fractional programming. Manag. Sci. 13(7), 492–498 (1967)

    Article  MathSciNet  MATH  Google Scholar 

  6. Gol’stein, E.G.: Dual problems of convex and fractionally-convex programming in functional spaces. Sov. Math. Dokl. 8, 212–216 (1967)

    Google Scholar 

  7. Golub, G.H., Van Loan, C.F.: An analysis of the total least-squares problem. SIAM J. Numer. Anal. 17(6), 883–893 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  8. Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. The Johns Hopkins University Press, Baltimore (1996)

    MATH  Google Scholar 

  9. Hansen, P.C.: Regularization tools: a Matlab package for analysis and solution of discrete ill-posed problems. Numer. Algorithm 6(1), 1–35 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  10. Hansen, P.C., O’Leary, D.P.: The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J. Sci. Comput. 14(6), 1487–1503 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  11. Hebden, M.D.: An algorithm for minimization using exact second derivatives. Int. J. Distrib. Sens. Netw. 2014(3), 1–8 (1973)

    Google Scholar 

  12. Jagannathan, R.: On some properities of programming problems in parametric form pertaining to fractional programming. Manag. Sci. 12, 609–615 (1966)

    Article  MATH  Google Scholar 

  13. Jain, A.K.: Fundamentals of Digital Image Processing. Prentice-Hall, Englewood Cliffs (1989)

    MATH  Google Scholar 

  14. Lampe, J., Voss, H.: Large-scale Tikhonov regularization of total least squares. J. Comput. Appl. Math. 238, 95–108 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  15. Moosaei, H., Ketabchi, S., Pardalos, P.M.: Tikhonov regularization for infeasible absolute value equations. Optimization 65(8), 1531–1537 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  16. Moré, J.J.: Generalization of the trust region problem. Optim. Methods Softw. 2(3), 189–209 (1993)

    Article  Google Scholar 

  17. Renaut, R.A., Guo, H.: Efficient algorithms for solution of regularized total least squares. SIAM J. Matrix Anal. Appl. 26(2), 457–476 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  18. Schaible, S.: Fractional programming II: on Dinkelbach’s algorithm. Manag. Sci. 22(8), 868–873 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  19. Sima, D., Van Huffel, S., Golub, G.H.: Regularized total least squares based on quadratic eigenvalue problem solvers. BIT 44, 793–812 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  20. Tikhonov, A.N., Arsenin, V.Y.: Solution of Ill-Posed Problems. V.H. Winston, Washington (1977)

    MATH  Google Scholar 

  21. Van Huffel, S., Lemmerling, P.: Total Least Squares and Errors-in-Variables Modeling. Kluwer Academic Publishers, Dordrecht (2002)

    Book  MATH  Google Scholar 

  22. Van Huffel, S., Vandewalle, J.: The Total Least Squares Problem: Computational Aspects and Analysis. Frontiers in Applied Mathematics. SIAM, Philadelphia (1991)

    Book  MATH  Google Scholar 

  23. Xia, Y., Wang, S., Sheu, R.L.: S-Lemma with equality and its applications. Math. Program. Ser. A. 156(1), 513–547 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  24. Zhang, A., Hayashi, S.: Celis–Dennis–Tapia based approach to quadratic fractional programming problems with two quadratic constraints. Numer. Algebra Control Optim. 1(1), 83–98 (2011)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. LMIB of the Ministry of Education, School of Mathematics and System Sciences, Beihang University, Beijing, 100191, People’s Republic of China

    Meijia Yang, Yong Xia & Jiulin Wang

  2. Department of Industrial Engineering, University of Houston, Houston, USA

    Jiming Peng

Authors
  1. Meijia Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiulin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiming Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong Xia.

Additional information

This research was supported by NSFC under Grants 11571029, 11471325 and 11771056, by NSF under Grants CMMI-1537712 and CMMI-1359548.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, M., Xia, Y., Wang, J. et al. Efficiently solving total least squares with Tikhonov identical regularization. Comput Optim Appl 70, 571–592 (2018). https://doi.org/10.1007/s10589-018-0004-4

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-018-0004-4

Keywords

Mathematics Subject Classification

Search

Navigation