Abstract
Minimizing a differentiable function restricted to the set of matrices with orthonormal columns finds applications in several fields of science and engineering. In this paper, we propose to solve this problem through a nonmonotone variation of the inexact restoration method consisting of two phases: restoration phase, aimed to improve feasibility, and minimization phase, aimed to decrease the function value in the tangent set of the constraints. For this, we give a suitable characterization of the tangent set of the orthogonality constraints, allowing us to (i) deal with the minimization phase efficiently and (ii) employ the Cayley transform to bring a point in the tangent set back to feasibility, leading to a SVD-free restoration phase. Based on previous global convergence results for the nonmonotone inexact restoration algorithm, it follows that any limit point of the sequence generated by the new algorithm is stationary. Moreover, numerical experiments on three different classes of the problem indicate that the proposed method is reliable and competitive with other recently developed methods.
Similar content being viewed by others
References
Abrudan, T., Eriksson, J., Koivunen, V.: Steepest descent algorithms for optimization under unitary matrix constraint. IEEE Trans. Signal Process. 56(3), 1134–1147 (2008)
Absil, P.-A., Baker, C.G., Gallivan, K.A.: Trust-region methods on Riemannian manifolds. Found. Comput. Math. 7(3), 303–330 (2007)
Absil, P.A., Mahony, R., Sepulchre, R.: Optimization algorithms on matrix manifolds. Princeton University Press, Princeton (2008)
Absil, P.A., Malick, J.: Projection-like retractions on matrix manifolds. SIAM J. Optim. 22(1), 135–158 (2012)
Arouxét, B., Echebest, N.E., Pilotta, E.A.: Inexact Restoration method for nonlinear optimization without derivatives. J. Comput. Appl. Math. 290(15), 26–43 (2015)
Barzilai, J., Borwein, J.M.: Two point step size gradient methods. IMA J. Numer. Anal. 8, 141–148 (1988)
Birgin, E.G., Bueno, L.F., Martínez, J.M.: Assessing the reability of general-purpose inexact restoration methods. J. Comput. Appl. Math. 282, 1–16 (2015)
Birgin, E.G., Martínez, J.M., Raydan, Marcos: Nonmonotone spectral projected gradient methods on convex sets. SIAM J. Optim. 10(4), 1196–1211 (2000)
Birgin, E.G., Martínez, J.M.: Local convergence of an inexact-restoration method and numerical experiments. J. Optim. Theory Appl. 127(2), 229–247 (2005)
Bueno, L.F., Martínez, J.M.: On the complexity of an inexact restoration method for constrained optimization. SIAM J. Optim. 30(1), 80–101 (2020)
Boumal, N., Mishra, B., Absil, P.-A., Sepulchre, R.: MAnopt, a Matlab Toolbox for optimization on manifolds. J. Mach. Learn. Res. 15, 1455–1459 (2014)
Bueno, L.F., Friedlander, A., Martínez, J.M., Sobral, F.N.C.: Inexact restoration method for derivative-free optimization with smooth constraints. SIAM J. Optim. 23(2), 1189–1213 (2013)
Bueno, L.F., Haeser, G., Martínez, J.M.: A flexible inexact-restoration method for constrained optimization. J. Optim. Theory Appl. 165, 188–208 (2015)
Cancès, E., Chakir, R., Maday, Y.: Numerical analysis of nonlinear eigenvalue problems. J. Sci. Comput. 45, 90–117 (2010)
Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)
Edelman, A., Arias, T.A., Smith, S.T.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20(2), 303–353 (1998)
Francisco, J.B., Gonçalves, D.S., Bazán, F.S.V., Paredes, L.L.T.: Nonmonotone Inexact Restoration Method for nonlinear programming, Computational Optimization and Applications to appear. https://doi.org/10.1007/s10589-019-00129-2(2019)
Francisco, J.B., Viloche Bazán, F.S.: Nonmonotone algorithm for minimization on closed sets with application to minimization on Stiefel manifolds. J. Comp. and Appl. Math. 236(10), 2717–2727 (2012)
Francisco, J.B., Bazán, F.S.V., Weber Mendonça, M.: Non-monotone algorithm for minimization on arbitrary domains with applications to large-scale orthogonal Procrustes problem, Appl. Num. Math. 112, 51–64 (2017)
Francisco, J.B., Martínez, J.M., Martínez, L., Pisnitchenko, F.: Inexact restoration method for minimization problems arising in electronic structure calculations. Comput. Optim. Appl. 50, 555–590 (2011)
Fischer, A., Friedlander, A.: A new line search inexact restoration approach for nonlinear programming. Comput. Optim. Appl. 46, 333–346 (2010)
Golub, G.H., Van Loan, C.F.: Matrix Computations, 4th ed. The Johns Hopkins University Press (2013)
Helgaker, T., JøRgensen, J., Olsen, J.: Electronic - Structure theory. Wiley, New York (2000)
Janin, R.: Directional derivative of the marginal function in non linear programming. In: Sensitivity, Stability and Parametric Analysis, Math. Program. Stud., pp. 110–126. Springer, Berlin (1984)
Jiang, B., Dai, Y.H.: A framework of constraint preserving update schemes for optimization on Stiefel manifold. Math. Program., Ser. A 153(2), 535–575 (2015)
Journée, M., Nesterov, Y., Richtárik, P., Sepulchre, R.: Generalized power method for sparse principal component analysis. J. Mach. Learn. Res. 11, 517–553 (2010)
Manton, J.H.: Optimization algorithms exploiting unitary constraints. IEEE Trans. Signal Process. 50(3), 635–650 (2002)
Martínez, J.M., Pilotta, E.A.: Inexact restoration algorithm for constrained optimization. J. Optim. Theory App. 104(1), 135–163 (2000)
Martínez, J.M., Svaiter, B.F.: A practical optimality condition without constraint qualifications for nonlinear programming. J. Optimiz. Theory App. 118 (1), 117–133 (2003)
Nishimori, Y., Akaho, S.: Learning algorithms utilizing quasi-geodesic flows on the Stiefel manifold. Neurocomputing 67, 106–135 (2005)
Shariff, M.: A constrained conjugate gradient method and the solution of linear equations. Comp. Mathem. Appl. 30(11), 25–37 (1995)
Karas, E. W., Pilotta, E., Ribeiro, A.: Numerical comparison of merit function with filter criterion in inexact restoration algorithms using hard-spheres problems. Comput. Optim. Appl. 44, 427–441 (2009)
Kaya, C.Y.: Inexact restoration for Runge–Kutta discretization of optimal control problems. SIAM J. Numer. Anal. 48(4), 1492–1517 (2010)
Kohn, W.: Nobel lecture: electronic structure of matter–wave functions and density functionals. Rev. Modern Phys. 71(5), 1253–1266 (1999)
Krejić, N., Martínez, J.M.: Inexact restoration approach for minimization with inexact evaluation of the objective function. Mathematics of Computation 85, 1775–1791 (2016)
Wen, Z., Yin, W.: A feasible method for optimization with orthogonality constraints. Math. Program. Ser. A, 142, 397–434 (2013)
Zhang, H., Hager, W.: A nonmonotone line search technique and its application to unconstrained optimization. SIAM J. Optimiz. 14(4), 1043–1056 (2004)
Zhao, Z., Bai, Z.-J., Jin, X.-Q.: A Riemannian Newton algorithm for nonlinear eigenvalue problems. SIAM J. Matrix Anal. Appl. 36(2), 752–774 (2015)
Zhu, X.: A feasible filter method for the nearest low-rank correlation matrix problem. Numer. Algorithm. 69, 763–784 (2015)
Acknowledgments
D.S.G thanks CAPES/Print Processes 88881.310538/2018-01 and 88887.465828/2019-00 which allow him to present part of this work at ICCOPT 2019 and supported his period as visiting professor at École Polytechnique where part of this work was carried out. We also thank anonymous referees whose comments helped to improve the quality of this work.
Funding
J.B.F., D.S.G, and F.S.V.B are grateful to CNPq by the financial support (Grants n. 421386/2016-9 and 308523/2017-2). L.L.T.P would like to thank CAPES for the scholarship during her Ph.D. at Universidade Federal de Santa Catarina. This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix
A Tables of numerical results
This appendix gathers the tables with numerical results for the experiments discussed in Sections 6.3, 6.4, and 6.5. The symbol “–” means that the solver could not achieve \(\|\tilde {P}_{S_{k}} (\nabla f(Y_{k})) \| < \varepsilon = 10^{-6}\), using less than IT\(_{\max \limits }=2,000\) iterations and FE\(_{\max \limits }=4,000\) function evaluations.
Only for test problems of Table 7, the maximum number of iterations was increased to IT\(_{\max \limits }=5,000\) and function evaluations to IT\(_{\max \limits }=10,000\). Moreover, for this last case, we set M = 0 and \(\delta _{CG} = 10^{-4} \| P_{S_{0}} (Y_{0}) \|\) in IRNOM.
Rights and permissions
About this article
Cite this article
Francisco, J.B., Gonçalves, D.S., Bazán, F.S.V. et al. Nonmonotone inexact restoration approach for minimization with orthogonality constraints. Numer Algor 86, 1651–1684 (2021). https://doi.org/10.1007/s11075-020-00948-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-020-00948-z