Skip to main content
SpringerLink
Log in

Discriminant Analysis

  • Chapter
  • First Online:
Neural Networks and Statistical Learning

  • 9781 Accesses

Abstract

Discriminant analysis plays an important role in statistical pattern recognition. LDA, originally derived by Fisher, is one of the most popular discriminant analysis techniques. Under the assumption that the class distributions are identically distributed Gaussians, LDA is Bayes optimal [44]. Like PCA, LDA is widely applied to image retrieval, face recognition, information retrieval, and pattern recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Baudat, G., & Anouar, F. (2000). Generalized discriminant analysis using a kernel approach. Neural Computation, 12, 2385–2404.

    Google Scholar 

  2. Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. J. (1997). Eigenfaces vs. fisherface: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 197, 711–720.

    Google Scholar 

  3. Cai, D., He, X., & Han, J. (2008). SRDA: An efficient algorithm for large-scale discriminant analysis. IEEE Transactions on Knowledge and Data Engineering, 20(1), 1–12.

    Google Scholar 

  4. Cevikalp, H., Neamtu, M., Wilkes, M., & Barkana, A. (2005). Discriminative common vectors for face recognition. IEEE Transactions on Patttern Anaysis and Machine Inteligence, 27(1), 4–13.

    Google Scholar 

  5. Chatterjee, C., Roychowdhury, V. P., Ramos, J., & Zoltowski, M. D. (1997). Self-organizing algorithms for generalized eigen-decomposition. IEEE Transactions on Neural Networks, 8(6), 1518–1530.

    Google Scholar 

  6. Chatterjee, C., & Roychowdhury, V. P. (1997). On self-organizing algorithms and networks for class-separability features. IEEE Transactions on Neural Networks, 8(3), 663–678.

    Google Scholar 

  7. Chen, L. -F., Liao, H. -Y. M., Ko, M. -T., Lin, J. -C., & Yu, G. -J. (2000). A new LDA-based face recognition system which can solve the small sample size problem. Pattern Recognition, 33(10), 1713–1726.

    Google Scholar 

  8. Chen, W. S., Yuen, P. C., & Huang, J. (2005). A new regularized linear discriminant analysis methods to solve small sample size problems. International Journal of Pattern Recognition and Artificial Intelligence, 19(7), 917–935.

    Google Scholar 

  9. Ching, W. -K., Chu, D., Liao, L. -Z., & Wang, X. (2012). Regularized orthogonal linear discriminant analysis. Pattern Recognition, 45, 2719–2732.

    Google Scholar 

  10. Chu, D., & Thye, G. S. (2010). A new and fast implementation for null space based linear discriminant analysis. Pattern Recognition, 43(4), 1373–1379.

    Google Scholar 

  11. Cruz, C. S., & Dorronsoro, J. R. (1998). A nonlinear discriminant algorithm for feature extraction and data classification. IEEE Transactions on Neural Networks, 9(6), 1370–1376.

    Google Scholar 

  12. Demir, G. K., & Ozmehmet, K. (2005). Online local learning algorithms for linear discriminant analysis. Pattern Recognition Letters, 26, 421–431.

    Google Scholar 

  13. Ding, C., & Li, T. (2007). Adaptive dimension reduction using discriminant analysis and \(K\)-means clustering. In: Proceedings of the 24th International Conference on Machine Learning (pp. 521–528), Corvallis, USA.

    Google Scholar 

  14. Duchene, L., & Leclerq, S. (1988). An optimal transformation for discriminant and principal component analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(6), 978–983.

    Google Scholar 

  15. Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7, 179–188.

    Google Scholar 

  16. Foley, D. H., & Sammon, J. W. (1975). An optimal set of discriminant vectors. IEEE Transactions on Computers, 24(3), 281–289.

    Google Scholar 

  17. Friedman, J. H. (1989). Regularized discriminant analysis. Journal of the American Statistical Association, 84(405), 165–175.

    Google Scholar 

  18. Gkalelis, N., Mezaris, V., & Kompatsiaris, I. (2011). Mixture subclass discriminant analysis. IEEE Signal Processing Letters, 18(5), 319–322.

    Google Scholar 

  19. Goldberger, J., Roweis, S., Hinton, G., & Salakhutdinov, R. (2005). Neighbourhood components analysis. In L. K. Saul, Y. Weiss, & L. Bottou (Eds.), Advances in neural information processing systems (Vol. 17, pp. 513–520). Cambridge, MA: MIT Press.

    Google Scholar 

  20. Gonzalez, A., & Dorronsoro, J. R. (2007). Natural learning in NLDA networks. Neural Networks, 20, 610–620.

    Google Scholar 

  21. Gulmezoglu, M. B., Dzhafarov, V., Keskin, M., & Barkana, A. (1999). A novel approach to isolated word recognition. IEEE Transactions on Audio and Speech Processing, 7(6), 620–628

    Google Scholar 

  22. Hamsici, O. C., & Martinez, A. M. (2008). Bayes optimality in linear discriminant analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(4), 647–657.

    Google Scholar 

  23. Hastie, T., Tibshirani, R., & Buja, A. (1994). Flexible discriminant analysis by optimal scoring. Journal of the American Statistical Association, 89(428), 1255–1270.

    Google Scholar 

  24. Hastie, T., & Tibshirani, R. (1995). Penalized discriminant analysis. Annals of Statistics, 23(1), 73–102.

    Google Scholar 

  25. Hastie, T., Tibshirani, R., & Friedman, J. H. (2001). The elements of statistical learning: Data mining, inference, and prediction. New York: Springer-Verlag.

    Google Scholar 

  26. He, X., & Niyogi, P. (2004). Locality preserving projections. In S. Thrun, L. Saul, & B. Scholkopf (Eds.), Advances in neural information processing systems (Vol. 16, pp. 153–160). Cambridge, MA: MIT Press.

    Google Scholar 

  27. He, X., Yan, S., Hu, Y., Niyogi, P., & Zhang, H.-J. (2005). Face recognition using Laplacianfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(3), 328–340.

    Google Scholar 

  28. Howland, P., & Park, H. (2004). Generalizing discriminant analysis using the generalized singular value decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(8), 995–1006.

    Google Scholar 

  29. Ji, S., & Ye, J. (2008). Generalized linear discriminant analysis: A unified framework and efficient model selection. IEEE Transactions on Neural Networks, 19(10), 1768–1782.

    Google Scholar 

  30. Jin, Z., Yang, J., Hu, Z., & Lou, Z. (2001). Face recognition based on the uncorrelated discrimination transformation. Pattern Recognition, 34(7), 1405–1416.

    Google Scholar 

  31. Kim, T.-K., & Kittler, J. (2005). Locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(3), 318–327.

    Google Scholar 

  32. Kong, H., Wang, L., Teoh, E.K., Wang, J.G., & Venkateswarlu, R. (2004). A framework of 2D Fisher discriminant analysis: application to face recognition with small number of training samples. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (Vol. 2, pp. 1083–1088), San Diego, CA.

    Google Scholar 

  33. Kwak, N., & Lee, J.-W. (2010). Feature extraction based on subspace methods for regression problems. Neurocomputing, 73, 1740–1751.

    Google Scholar 

  34. Kyperountas, M., Tefas, A., & Pitas, I. (2007). Weighted piecewise LDA for solving the small sample size problem in face verification. IEEE Transactions on Neural Networks, 18(2), 506–519.

    Google Scholar 

  35. Li, H., Jiang, T., & Zhang, K. (2006). Efficient and robust feature extraction by maximum margin criterion. IEEE Transactions on Neural Network, 17(1), 157–165.

    Google Scholar 

  36. Li, M., & Yuan, B. (2005). 2D-LDA: A statistical linear discriminant analysis for image matrix. Pattern Recognition Letters , 26, 527–532.

    Google Scholar 

  37. Li, Z., Lin, D., & Tang, X. (2009). Nonparametric discriminant analysis for face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(4), 755–761.

    Google Scholar 

  38. Liu, C., & Wechsler, H. (2000). Robust coding scheme for indexing and retrieval from large face databases. IEEE Transactions on Image Processing, 9, 132–137.

    Google Scholar 

  39. Loog, M., Duin, R. P. W., & Haeb-Umbach, R. (2001). Multiclass linear dimension reduction by weighted pairwise Fisher criteria. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, 762–766.

    Google Scholar 

  40. Lu, J., Plataniotis, K. N., & Venetsanopoulos, A. N. (2003). Face recognition using LDA based algorithms. IEEE Transactions on Neural Networks, 14(1), 195–200.

    Google Scholar 

  41. Martinez, A. M., & Kak, A. C. (2001). PCA versus LDA. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, 228–233.

    Google Scholar 

  42. Michie, D., Spiegelhalter, D.J., & Taylor, C.C. (Eds). (1994). Machine learning: Neural and statistical classification. New York: Ellis Horwood.

    Google Scholar 

  43. Mao, J., & Jain, A. K. (1995). Artificial neural networks for feature extraction and multivariate data projection. IEEE Transactions on Neural Networks, 6(2), 296–317.

    Google Scholar 

  44. Muller, K. R., Mika, S., Ratsch, G., Tsuda, K., & Scholkopf, B. (2001). An introduction to kernel-based learning algorithms. IEEE Transactions on Neural Networks, 12(2), 181–201.

    Google Scholar 

  45. Na, J. H., Park, M. S., & Choi, J. Y. (2010). Linear boundary discriminant analysis. Pattern Recognition, 43, 929–936.

    Google Scholar 

  46. Nagabhushan, P., Guru, D. S., & Shekar, B. H. (2006). (2D)\(^2\) FLD: An efficient approach for appearance based object recognition. Neurocomputing, 69, 934–940.

    Google Scholar 

  47. Nenadic, Z. (2007). Information discriminant analysis: Feature extraction with an information-theoretic objective. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(8), 1394–1407.

    Google Scholar 

  48. Okada, T., & Tomita, S. (1985). An optimal orthonormal system for discriminant analysis. Pattern Recognition, 18(2), 139–144.

    Google Scholar 

  49. Pang, S., Ozawa, S., & Kasabov, N. (2004). One-pass incremental membership authentication by face classification. In D. Zhang & A. K. Jain (Eds.), Biometric Authentication (pp. 155–161). New York: Springer-Verlag.

    Google Scholar 

  50. Peng, J., Zhang, P., & Riedel, N. (2008). Discriminant learning analysis. IEEE Transactions on Systems, Man, and Cybernetics Part B, 38(6), 1614–1625.

    Google Scholar 

  51. Qiu, X., & Wu, L. (2006). Nearest neighbor discriminant analysis. International Journal of Pattern Recognition and Artificial Intelligence, 20(8), 1245–1259.

    Google Scholar 

  52. Qiu, X., & Wu, L. (2007). Two-dimensional nearest neighbor discriminant analysis. Neurocomputing, 70, 2572–2575.

    Google Scholar 

  53. Sharma, A., & Paliwal, K. K. (2008). A gradient linear discriminant analysis for small sample sized problem. Neural Processing Letters, 27, 17–24.

    Google Scholar 

  54. Sharma, A., & Paliwal, K. K. (2008). Rotational linear discriminant analysis technique for dimensionality reduction. IEEE Transactions on Knowledge and Data Engineering, 20(10), 1336–1347.

    Google Scholar 

  55. Skurichina, M., & Duin, R.P.W. (1996). Stabilizing classifiers for very small sample size. In Proceedings of International Conference on Pattern Recognition (pp. 891–896).

    Google Scholar 

  56. Srivastava, S., & Gupta, M.R. (2006). Distribution-based Bayesian minimum expected risk for discriminant analysis. In Proceedings of IEEE International Symposium on Information Theory (pp. 2294–2298).

    Google Scholar 

  57. Sugiyama, M. (2007). Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis. Journal of Machine Learning Research, 8, 1027–1061.

    Google Scholar 

  58. Sugiyama, M., Ide, T., Nakajima, S., & Sese, J. (2010). Semi-supervised local Fisher discriminant analysis for dimensionality reduction. Machine Learning, 78, 35–61.

    Google Scholar 

  59. Swets, D. L., & Weng, J. (1996). Using discriminant eigenfeatures for image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(8), 831–836.

    Google Scholar 

  60. Wang, F., Zhao, B., & Zhang, C. (2011). Unsupervised large margin discriminative projection. IEEE Transactions on Neural Networks, 22(9), 1446–1456.

    Google Scholar 

  61. Xiang, C., Fan, X. A., & Lee, T. H. (2006). Face recognition using recursive Fisher linear discriminant. IEEE Transactions on Image Processing, 15(8), 2097–2105.

    Google Scholar 

  62. Xiong, H., Swamy, M. N. S., & Ahmad, M. O. (2005). Two-dimension FLD for face recognition. Pattern Recognition, 38(7), 1121–1124.

    Google Scholar 

  63. Yan, J., Zhang, B., & Yan et al, S. (2004). IMMC: Incremental maximum margin criterion. In Proceedings of International Conference on Knowledge Discovery and Data Mining (pp. 725–730).

    Google Scholar 

  64. Ye, J., Janardan, R., Park, C. H., & Park, H. (2004). An optimization criterion for generalized discriminant analysis on undersampled problems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(8), 982–994.

    Google Scholar 

  65. Ye, J., Janardan, R., & Li, Q. (2004). Two-dimensional linear discriminant analysis. In Advances in neural information processing systems (Vol. 17). Boston: MIT Press.

    Google Scholar 

  66. Ye, J., Li, Q., Xiong, H., Park, H., Janardan, R., & Kumar, V. (2005). IDR/QR: An incremental dimension reduction algorithm via QR decomposition. IEEE Transactions on Knowledge and Data Engineering, 17(9), 1208–1222.

    Google Scholar 

  67. Ye, J. (2005). Characterization of a family of algorithms for generalized discriminant analysis on undersampled problems. Journal of Machine Learning Research, 6, 483–502.

    Google Scholar 

  68. Ye, J., & Li, Q. (2005). A two-stage linear discriminant analysis via QR-decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(6), 929–941.

    Google Scholar 

  69. Ye, J. (2005). Generalized low rank approximations of matrices. Machine Learning, 61(1), 167–191.

    Google Scholar 

  70. Ye, J., & Xiong, T. (2006). Computational and theoretical analysis of null space and orthogonal linear discriminant analysis. Journal of Machine Learning Research, 7, 1183–1204.

    Google Scholar 

  71. Zhang, T., Fang, B., Tang, Y. Y., Shang, Z., & Xu, B. (2010). Generalized discriminant analysis: A matrix exponential approach. IEEE Transactions on Systems, Man, and Cybernetics Part B, 40(1), 186–197.

    Google Scholar 

  72. Zhang, H., Guan, C., & Li, Y. (2011). A linear discriminant analysis method based on mutual information maximization. Pattern Recognition, 44(4), 877–885.

    Google Scholar 

  73. Zhang, Y., & Yeung, D.-Y. (2011). Semisupervised generalized discriminant analysis. IEEE Transactions on Neural Networks, 22(8), 1207–1217.

    Google Scholar 

  74. Zhao, H., & Yuen, P. C. (2008). Incremental linear discriminant analysis for face recognition. IEEE Transactions on Systems, Man, and Cybernetics Part B, 38(1), 210–221.

    Google Scholar 

  75. Zheng, W. (2006). Class-incremental generalized discriminant analysis. Neural Computation, 18, 979–1006.

    Google Scholar 

  76. Zhu, M., & Martinez, A. M. (2006). Subclass discriminant analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(8), 1274–1286.

    Google Scholar 

  77. Zhu, M., & Martinez, A. M. (2008). Pruning noisy bases in discriminant analysis. IEEE Transactions on Neural Networks, 19(1), 148–157.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Enjoyor Labs, Enjoyor Inc., Hangzhou, China

    Ke-Lin Du & M. N. S. Swamy

  2. Department of Electrical and Computer Engineering, Concordia University, Montreal, QC, Canada

    Ke-Lin Du

Authors
  1. Ke-Lin Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. N. S. Swamy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ke-Lin Du .

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag London

About this chapter

Cite this chapter

Du, KL., Swamy, M.N.S. (2014). Discriminant Analysis. In: Neural Networks and Statistical Learning. Springer, London. https://doi.org/10.1007/978-1-4471-5571-3_15

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-5571-3_15

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-5570-6

  • Online ISBN: 978-1-4471-5571-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation