Adaptive Rejection Metropolis Simulated Annealing for Detecting Global Maximum Regions
- 136 Downloads
A finite mixture model has been used to fit the data from heterogeneous populations to many applications. An Expectation Maximization (EM) algorithm is the most popular method to estimate parameters in a finite mixture model. A Bayesian approach is another method for fitting a mixture model. However, the EM algorithm often converges to the local maximum regions, and it is sensitive to the choice of starting points. In the Bayesian approach, the Markov Chain Monte Carlo (MCMC) sometimes converges to the local mode and is difficult to move to another mode. Hence, in this paper we propose a new method to improve the limitation of EM algorithm so that the EM can estimate the parameters at the global maximum region and to develop a more effective Bayesian approach so that the MCMC chain moves from one mode to another more easily in the mixture model. Our approach is developed by using both simulated annealing (SA) and adaptive rejection metropolis sampling (ARMS). Although SA is a well-known approach for detecting distinct modes, the limitation of SA is the difficulty in choosing sequences of proper proposal distributions for a target distribution. Since ARMS uses a piecewise linear envelope function for a proposal distribution, we incorporate ARMS into an SA approach so that we can start a more proper proposal distribution and detect separate modes. As a result, we can detect the maximum region and estimate parameters for this global region. We refer to this approach as ARMS annealing. By putting together ARMS annealing with the EM algorithm and with the Bayesian approach, respectively, we have proposed two approaches: an EM-ARMS annealing algorithm and a Bayesian-ARMS annealing approach. We compare our two approaches with traditional EM algorithm alone and Bayesian approach alone using simulation, showing that our two approaches are comparable to each other but perform better than EM algorithm alone and Bayesian approach alone. Our two approaches detect the global maximum region well and estimate the parameters in this region. We demonstrate the advantage of our approaches using an example of the mixture of two Poisson regression models. This mixture model is used to analyze a survey data on the number of charitable donations.
KeywordsAdaptive rejection metropolis sampling Expectation Maximization algorithm Finite mixture models Simulated annealing
AMS 2000 Subject Classification62A99
Unable to display preview. Download preview PDF.
- Böhning D, Dietz E, Schlattmann P, Mendonca L, Kirchner U (1995) The zero-inflated Poisson model and the decayed, missing and filled teeth index in dental epidemiology. J R Stat Soc Ser A 162:194–209Google Scholar
- Park T, Park S (2004) An economic study on charitable giving of individuals in Korea: some new findings from 2002 survey data. Presented at the 6th ISTR conference, Toronto, Canada, 11–14 July 2004Google Scholar
- Smith VH, Kchoe MR, Creamer ME (1999) The private provision of public goods: altruism and voluntary giving. J Public Econ 77:213–242Google Scholar
- von Neumann J (1951) Various techniques used in connection with random digits: Monte Carlo methods. Natl Bur Stan AMS 12:36–38Google Scholar