Asymptotic Characterization of Log-Likelihood Maximization Based Algorithms and Applications

Abstract

The asymptotic distribution of estimates that are based on a sub-optimal search for the maximum of the log-likelihood function is considered. In particular, estimation schemes that are based on a two-stage approach, in which an initial estimate is used as the starting point of a subsequent local maximization, are analyzed. We show that asymptotically the local estimates follow a Gaussian mixture distribution, where the mixture components correspond to the modes of the likelihood function. The analysis is relevant for cases where the log-likelihood function is known to have local maxima in addition to the global maximum, and there is no available method that is guaranteed to provide an estimate within the attraction region of the global maximum. Two applications of the analytic results are offered. The first application is an algorithm for finding the maximum likelihood estimator. The algorithm is best suited for scenarios in which the likelihood equations do not have a closed form solution, the iterative search is computationally cumbersome and highly dependent on the data length, and there is a risk of convergence to a local maximum. The second application is a scheme for aggregation of local estimates, e.g. generated by a network of sensors, at a fusion center. This scheme provides the means to intelligently combine estimates from remote sensors, where bandwidth constraints do not allow access to the complete set of data. The result on the asymptotic distribution is validated and the performance of the proposed algorithms is evaluated by computer simulations.