Abstract
In this paper we consider the problem of multivariate density estimation assuming that the density allows some form of dimensionality reduction. Estimation of high-dimensional densities and dimensionality reduction models are important topics in nonparametric and semi-parametric econometrics.We start with the Independent Component Analysis (ICA) model, which can be considered as a form of dimensionality reduction of a multivariate density. We then consider multiple index model, describing the situations where high-dimensional data has a low-dimensional non-Gaussian component while in all other directions the data are Gaussian, and the independent factor analysis (IFA) model, which generalizes the ordinary factor analysis, principal component analysis, and ICA. For each of these models, we review recent results, obtained in our joint work with Tsybakov, Amato, and Antoniadis, on the accuracy of the corresponding density estimators, which combine model selection with estimation. One of the main applications of multivariate density estimators is in classification, where they can be used to construct plug-in classifiers by estimating the densities of each labeled class. We give a bound to the excess risk of nonparametric plug-in classifiers in terms of the MISE of the density estimators of each class. Combining this bound with the above results on the accuracy of density estimation, we show that the rate of the excess Bayes risk of the corresponding plug-in classifiers does not depend on the dimensionality of the data.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Amato, U., Antoniadis, A., Samarov, A., & Tsybakov, A. (2010). Noisy independent factor analysis model for density estimation and classification. Electronic Journal of Statistics, 4, 707–736.
Artiles, L. M. (2001). Adaptive Minimax Estimation in Classes of Smooth Functions (Ph.D. thesis). University of Utrecht.
Belitser, E., & Levit, B. (2001). Asymptotically local minimax estimation of infinitely smooth density with censored data. Annals of the Institute of Statistical Mathematics, 53, 289–306.
Blanchard, B., Kawanabe, G. M., Sugiyama, M., Spokoiny, V., & Müller, K. R. (2006). In search of non-gaussian components of a high-dimensional distribution. Journal of Machine Learning Research, 7, 247–282.
Cook, R. D., & Li, B. (2002). Dimension reduction for conditional mean in regression. Annals of Statistics, 32, 455–474.
Hristache, M., Juditsky, A., Polzehl J., & Spokoiny, V. (2001). Structure adaptive approach for dimension reduction. Annals of Statistics, 29, 1537–1566.
Huber, P. (1985). Projection pursuit. Annals of Statistics, 13, 435–475.
Hyvarinen, A., Karhunen, J., & Oja, E. (2001). Independent component analysis. New York: Wiley.
Juditsky, A., Rigollet, P., & Tsybakov, A. B. (2008). Learning by mirror averaging. Annals of Statistics, 36, 2183–2206.
Juditsky, A. B., Nazin, A. V., Tsybakov, A. B., & Vayatis, N. (2005). Recursive aggregation of estimators by the mirror descent algorithm with averaging. Problems of Information Transmission, 41, 368–384.
Li, K.-C. (1991). Sliced inverse regression for dimension reduction. Journal of the American Statistical Association, 86, 316–342.
McLachlan, G. J., & Peel, D. (2000). Finite mixture models. New York: Wiley.
Roweis, S., & Saul, L. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290, 2323–2326.
Samarov, A., & Tsybakov, A. B. (2004). Nonparametric independent component analysis. Bernoulli, 10, 565–582.
Samarov, A., & Tsybakov, A. B. (2007). Aggregation of density estimators and dimension reduction. In V. Nair (Ed.), Advances in statistical modeling and inference, essays in honor of K. Doksum. Series in Biostatistics (Vol. 3, pp. 233–251). London: World Scientific.
Tenenbaum, J. B., de Silva, V., & Langford, J. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290, 2319–2323.
Titterington, D., Smith, A., & Makov, U. (1985). Statistical analysis of finite mixture distributions. New York: Wiley.
Acknowledgements
Partial support provided by the Singapore-MIT Alliance in Computation and Systems Biology.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Samarov, A. (2015). Dimensionality Reduction Models in Density Estimation and Classification. In: Beran, J., Feng, Y., Hebbel, H. (eds) Empirical Economic and Financial Research. Advanced Studies in Theoretical and Applied Econometrics, vol 48. Springer, Cham. https://doi.org/10.1007/978-3-319-03122-4_30
Download citation
DOI: https://doi.org/10.1007/978-3-319-03122-4_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-03121-7
Online ISBN: 978-3-319-03122-4
eBook Packages: Business and EconomicsEconomics and Finance (R0)