A Multilabel Extension of LDA Based on the Gram-Schmidt Orthogonalization Procedure

  • Juan Bekios-Calfa
  • Brian Keith
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10657)


Multilabel classification is a generalization of the traditional unidimensional classification problem, the goal of multilabel classification is to learn a function that maps instances into a set of relevant labels. This article proposes an extension to linear discriminant analysis in the context of multilabel classification. The new method is based on Gram-Schmidt orthogonalization procedure. The theoretical basis and underlying assumptions of the new model are described and the method is experimentally evaluated on the Emotions data set for multilabel classification. The analysis of the empirical results support that this new method is competitive and in some instances superior to the baseline.


Linear discriminant analysis Gram-Schmidt orthogonalization Multilabel classification 


  1. 1.
    Björck, Å.: Numerics of Gram-Schmidt orthogonalization. Linear Algebra Appl. 197, 297–316 (1994)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Cai, D., He, X., Han, J.: SRDA: an efficient algorithm for large-scale discriminant analysis. IEEE Trans. Knowl. Data Eng. 20(1), 1–12 (2008)CrossRefGoogle Scholar
  3. 3.
    Cevikalp, H., Neamtu, M., Wilkes, M., Barkana, A.: Discriminative common vectors for face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 27(1), 4–13 (2005)CrossRefGoogle Scholar
  4. 4.
    Cohen, H.: A Course in Computational Algebraic Number Theory, vol. 138. Springer Science & Business Media, Heidelberg (1993). MATHGoogle Scholar
  5. 5.
    Farebrother, R.: Algorithm as 79: Gram-Schmidt regression. J. Roy. Stat. Soc.: Ser. C (Appl. Stat.) 23(3), 470–476 (1974)Google Scholar
  6. 6.
    Gibaja, E., Ventura, S.: Multi-label learning: a review of the state of the art and ongoing research. Wiley Interdisc. Rev.: Data Min. Knowl. Discov. 4(6), 411–444 (2014)Google Scholar
  7. 7.
    Izenman, A.J.: Linear discriminant analysis. In: Modern Multivariate Statistical Techniques. STS, pp. 237–280. Springer, Heidelberg (2013).
  8. 8.
    Ji, S., Ye, J.: Linear dimensionality reduction for multi-label classification. In: IJCAI, vol. 9, pp. 1077–1082 (2009)Google Scholar
  9. 9.
    Oikonomou, M., Tefas, A.: Direct multi-label linear discriminant analysis. In: Iliadis, L., Papadopoulos, H., Jayne, C. (eds.) EANN 2013. CCIS, vol. 383, pp. 414–423. Springer, Heidelberg (2013). CrossRefGoogle Scholar
  10. 10.
    Park, C.H., Lee, M.: On applying linear discriminant analysis for multi-labeled problems. Pattern Recogn. Lett. 29(7), 878–887 (2008)CrossRefGoogle Scholar
  11. 11.
    Rodgers, J.L., Nicewander, W.A., Toothaker, L.: Linearly independent, orthogonal, and uncorrelated variables. Am. Stat. 38(2), 133–134 (1984)Google Scholar
  12. 12.
    Trohidis, K., Tsoumakas, G., Kalliris, G., Vlahavas, I.P.: Multi-label classification of music into emotions. In: Bello, J.P., Chew, E., Turnbull, D. (eds.) ISMIR, pp. 325–330 (2008).
  13. 13.
    Tsoumakas, G., Katakis, I.: Multi-label classification: an overview. Int. J. Data Warehouse. Min. 3(3), 1 (2007)CrossRefGoogle Scholar
  14. 14.
    Wan, H., Wang, H., Guo, G., Wei, X.: Separability-oriented subclass discriminant analysis. IEEE Trans. Pattern Anal. Mach. Intell. (2017)Google Scholar
  15. 15.
    Wang, H., Ding, C., Huang, H.: Multi-label linear discriminant analysis. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6316, pp. 126–139. Springer, Heidelberg (2010). CrossRefGoogle Scholar
  16. 16.
    Wieczorkowska, A., Synak, P., Raś, Z.W.: Multi-label classification of emotions in music. In: Kłopotek, M.A., Wierzchoń, S.T., Trojanowski, K. (eds.) Intelligent Information Processing and Web Mining. AINSC, vol. 35, pp. 307–315. Springer, Heidelberg (2006). CrossRefGoogle Scholar
  17. 17.
    Zhang, M.L., Zhou, Z.H.: A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26(8), 1819–1837 (2014)CrossRefGoogle Scholar
  18. 18.
    Zheng, W., Zou, C., Zhao, L.: Real-time face recognition using Gram-Schmidt orthogonalization for LDA. In: Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, vol. 2, pp. 403–406. IEEE (2004)Google Scholar
  19. 19.
    Zhu, M., Martinez, A.M.: Selecting principal components in a two-stage LDA algorithm. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 132–137. IEEE (2006)Google Scholar
  20. 20.
    Zhu, M., Martinez, A.M.: Subclass discriminant analysis. IEEE Trans. Pattern Anal. Mach. Intell. 28(8), 1274–1286 (2006)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Departamento de Ingeniería de Sistemas y ComputaciónUniversidad Católica del NorteAntofagastaChile

Personalised recommendations