LetF(x,y) be a function of the vector variablesx∈R n andy∈R m. One possible scheme for minimizingF(x,y) is to successively alternate minimizations in one vector variable while holding the other fixed. Local convergence analysis is done for this vector (grouped variable) version of coordinate descent, and assuming certain regularity conditions, it is shown that such an approach is locally convergent to a minimizer and that the rate of convergence in each vector variable is linear. Examples where the algorithm is useful in clustering and mixture density decomposition are given, and global convergence properties are briefly discussed.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Zangwill, W.,Nonlinear Programming: A Unified Approach, Prentice-Hall, Englewood Cliffs, New Jersey, 1969.
Stewart, G. W.,Introduction to Matrix Computations, Academic Press, Orlando, Florida, 1973.
Bezdek, J. C.,Pattern Recognition with Fuzzy Objective Function Algorithms, Plenum Press, New York, New York, 1981.
Hathaway, R. J.,Another Interpretation of the EM Algorithm, Statistics and Probability Letters, Vol. 4, pp. 53–56, 1986.
Redner, R. A., andWalker, H. F.,Mixture Densities, Maximum Likelihood, and the EM Algorithm, SIAM Review, Vol. 26, pp. 195–239, 1984.
Dennis, J. E., Jr., andSteihaug, T.,On the Successive Projections Approach to Least-Squares Problems, Technical Report No. 83-18, Department of Mathematical Sciences, Rice University, Houston, Texas, 1983.
This research was supported in part by NSF Grant No. IST-84-07860. The authors are indebted to Professor R. A. Tapia for his help in improving this paper.
Communicated by R. A. Tapia
About this article
Cite this article
Bezdek, J.C., Hathaway, R.J., Howard, R.E. et al. Local convergence analysis of a grouped variable version of coordinate descent. J Optim Theory Appl 54, 471–477 (1987). https://doi.org/10.1007/BF00940196
- Coordinate descent
- local linear convergence