Dimensionality Reduction of Proportional Data Through Data Separation Using Dirichlet Distribution

  • Walid MasoudimansourEmail author
  • Nizar Bouguila
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9164)


In this paper, a novel method is proposed for dimensionality reduction of proportional data. Non-negative, unit-sum data, namely, proportional data emerges in many applications such as document classification, image classification using visual bag of words, etc. The introduced method is supervised and can be used for classification of data into binary classes. In the proposed method, the intra-class correlation is maximized while minimizing the interclass correlation, using a linear transform. Design of this transform is formulated as an optimization problem with proper cost function. The projected data is matched to two Dirichlet distributions with careful parameter selection which allows to separate the classes in the Dirichlet parameter space. Finally, simulations are performed to demonstrate the effectiveness of the algorithm.


Dimensionality reduction Supervised learning Data classification Dirichlet distribution 


  1. 1.
    Donoho, D.L., et al.: High-dimensional data analysis: The curses and blessings of dimensionality. In: AMS Math Challenges Lecture, pp. 1–32 (2000)Google Scholar
  2. 2.
    Lu, X., Yuan, Y.: Hybrid structure for robust dimensionality reduction. J. Neurocomput. 124, 131–138 (2014)CrossRefGoogle Scholar
  3. 3.
    Wang, S.J., Yan, S., Yang, J., Zhou, C.G., Fu, X.: A general exponential framework for dimensionality reduction. IEEE Trans. Image Proces. 23(2), 920–930 (2014)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Jolliffe, I.: Principal Component Analysis. Springer Series in Statistics. Springer, Heidelberg (2002)Google Scholar
  5. 5.
    Abdi, H., Williams, L.J.: Principal component analysis. Wiley Interdisc. Rev.: Comput. Statis. 2(4), 433–459 (2010)CrossRefGoogle Scholar
  6. 6.
    Mulaik, S.A.: The foundations of factor analysis, vol. 88. McGraw-Hill, New York (1972)Google Scholar
  7. 7.
    Kokiopoulou, E., Saad, Y.: Orthogonal neighborhood preserving projections: a projection-based dimensionality reduction technique. IEEE Trans. Pattern Anal. Mach. Intel. 29(12), 2143–2156 (2007)CrossRefGoogle Scholar
  8. 8.
    Mika, S., Schölkopf, B., Smola, A.J., Müller, K.R., Scholz, M., Rätsch, G.: Kernel PCA and de-noising in feature spaces. Proc. Neural Inform. Proces. Syst. 11, 536–542 (1998)Google Scholar
  9. 9.
    Verbeek, J.: Learning nonlinear image manifolds by global alignment of local linear models. IEEE Trans. Pattern Anal. Mach. Intel. 28(8), 1236–1250 (2006)CrossRefGoogle Scholar
  10. 10.
    Welling, M., Rosen-Zvi, M., Hinton, G.E.: Exponential family harmoniums with an application to information retrieval. Proc. Neural Inform. Proces. Syst. 17, 1481–1488 (2004)Google Scholar
  11. 11.
    Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, June 2004Google Scholar
  12. 12.
    Koren, Y., Carmel, L.: Robust linear dimensionality reduction. IEEE Trans. Vis. Comput. Graph. 10(4), 459–470 (2004)CrossRefGoogle Scholar
  13. 13.
    Bouguila, N., Ziou, D.: A dirichlet process mixture of generalized dirichlet distributions for proportional data modeling. IEEE Trans. Neural Netw. 21(1), 107–122 (2010)CrossRefGoogle Scholar
  14. 14.
    Wang, H.Y., Yang, Q., Qin, H., Zha, H.: Dirichlet component analysis: Feature extraction for compositional data. In: Proceedings of the 25th International Conference on Machine Learning (ICML), pp. 1128–1135. ACM (2008)Google Scholar
  15. 15.
    Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Athena Scientific, September 1999Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Department of Electrical and Computer EngineeringConcordia UniversityMontrealCanada
  2. 2.Concordia Institute for Information Systems EngineeringConcordia UniversityMontrealCanada

Personalised recommendations