An Infinite Latent Generalized Linear Model

Abstract

We propose an Infinite Latent Generalized Linear Model (ILGLM), a Dirichlet process mixture of generalized linear model in latent space for classification problem. In ILGLM, we assume latent variable z _n is generated from a low-dimensional DPM model in latent space, and the corresponding observed feature x _n and class label y _n are generated from some latent probability model and local linear classification model independently conditioned on z _n . Then in ILGLM, we will jointly learn the latent variable model and multiple local generalized linear model under the framework of Dirichlet process mixture. On one hand, ILGLM can model the multiple local linearity of data distribution adaptively according to data complexity; on the other hand, it avoid the curse of dimensionality problem. ILGLM can be extended to semi-supervised setting, training the model using both labeled and unlabeled data. Because ILGLM is a general model framework, it can incorporate any kind of latent variable models and linear classification models. Then we implement ILGLM based on Factor Analysis and MultiNomial Logit model, which results in the Infinite Latent MultiNomial Logit (ILMNL) model as an example of ILGLM. We also develop an approximate posterior inference algorithm for ILMNL using Gibbs sampling. Experiments on several real-world datasets demonstrate the advantages of ILMNL in dealing with high-dimensional data classification problems compared with competitive models.