An Efficient Gradient Method with Approximately Optimal Stepsize Based on Tensor Model for Unconstrained Optimization
- 31 Downloads
A new type of stepsize, which was recently introduced by Liu et al. (Optimization 67(3):427–440, 2018), is called approximately optimal stepsize and is very efficient for gradient method. Interestingly, all gradient methods can be regarded as gradient methods with approximately optimal stepsizes. In this paper, we present an efficient gradient method with approximately optimal stepsize based on tensor model for unconstrained optimization. In the proposed method, if the objective function is not close to a minimizer and a quadratic function on a line segment between the current and latest iterates, then a tensor model is exploited to generate approximately optimal stepsize for gradient method. Otherwise, quadratic approximation models are constructed to generate approximately optimal stepsizes for gradient method. The global convergence of the proposed method is established under weak conditions. Numerical results indicate that the proposed method is very promising.
KeywordsGradient method Approximately optimal stepsize Quadratic model Tensor model Global convergence
Mathematics Subject Classification90C06 65K
We would like to thank Professors Hager and Zhang, H. C. for their C code of CG_DESCENT, and thank Professor Dai, Y. H. for his help in the numerical experiments. This research is supported by National Science Foundation of China (No.11461021), Shangxi Science Foundation (No. 2017JM1014), Guangxi Science Foundation (Nos. 2018GXNSFBA281180, 2017GXNSFBA198031), Project of Guangxi Education Department Grant (2017KY0648), Scientific Research Project of Hezhou University (Nos. 2014YBZK06, 2016HZXYSX03).
- 1.Cauchy, A.: Méthode générale pour la résolution des systéms déquations simultanées. Comp. Rend. Sci. Paris 25, 46–89 (1847)Google Scholar
- 20.Liu, Z.X., Liu, H.W.: An efficient Barzilai–Borwein conjugate gradient method for unconstrained optimization. J. Optim. Theory Appl. https://doi.org/10.1007/s10957-018-1393-3 (2018)
- 21.Yuan, G.L., Wei, Z.X., Zhao, Q.M.: A modified Polak–Ribière–Polyak conjugate gradient algorithm for large-scale optimization problems. IIE Trans. 46, 397–413 (2014)Google Scholar
- 23.Figueiredo, M.A.T., Nowak, R.D., Wright, S.J.: Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. 1(4), 586–597 (2007)Google Scholar
- 32.Yuan, Y.X., Sun, W.Y.: Theory and Methods of Optimization. Science Press of China, Beijing (1999)Google Scholar