Abstract
Fisher Discriminant Analysis (FDA) is a subspace learning method which minimizes and maximizes the intra- and inter-class scatters of data, respectively. Although, in FDA, all the pairs of classes are treated the same way, some classes are closer than the others. Weighted FDA assigns weights to the pairs of classes to address this shortcoming of FDA. In this paper, we propose a cosine-weighted FDA as well as an automatically weighted FDA in which weights are found automatically. We also propose a weighted FDA in the feature space to establish a weighted kernel FDA for both existing and newly proposed weights. Our experiments on the ORL face recognition dataset show the effectiveness of the proposed weighting schemes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Friedman, J., Hastie, T., Tibshirani, R.: The Elements of Statistical Learning. Springer Series in Statistics, vol. 1. Springer, New York (2001). https://doi.org/10.1007/978-0-387-84858-7
Fisher, R.A.: The use of multiple measurements in taxonomic problems. Ann. Eugen. 7(2), 179–188 (1936)
Mika, S., Ratsch, G., Weston, J., Scholkopf, B., Mullers, K.R.: Fisher discriminant analysis with kernels. In: Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop, pp. 41–48. IEEE (1999)
Ghojogh, B., Karray, F., Crowley, M.: Roweis discriminant analysis: a generalized subspace learning method. arXiv preprint arXiv:1910.05437 (2019)
Zhang, Z., Dai, G., Xu, C., Jordan, M.I.: Regularized discriminant analysis, ridge regression and beyond. J. Mach. Learn. Res. 11(Aug), 2199–2228 (2010)
Díaz-Vico, D., Dorronsoro, J.R.: Deep least squares Fisher discriminant analysis. IEEE Trans. Neural Netw. Learn. Syst. (2019)
Xu, Y., Lu, G.: Analysis on Fisher discriminant criterion and linear separability of feature space. In: 2006 International Conference on Computational Intelligence and Security, vol. 2, pp. 1671–1676. IEEE (2006)
Parlett, B.N.: The Symmetric Eigenvalue Problem, vol. 20. Society for Industrial and Applied Mathematics (SIAM), Philadelphia (1998)
Ghojogh, B., Karray, F., Crowley, M.: Fisher and kernel Fisher discriminant analysis: tutorial. arXiv preprint arXiv:1906.09436 (2019)
Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)
Ghojogh, B., Karray, F., Crowley, M.: Eigenvalue and generalized eigenvalue problems: tutorial. arXiv preprint arXiv:1903.11240 (2019)
Alperin, J.L.: Local Representation Theory: Modular Representations as an Introduction to the Local Representation Theory of Finite Groups, vol. 11. Cambridge University Press, Cambridge (1993)
Loog, M., Duin, R.P., Haeb-Umbach, R.: Multiclass linear dimension reduction by weighted pairwise Fisher criteria. IEEE Trans. Pattern Anal. Mach. Intell. 23(7), 762–766 (2001)
Lotlikar, R., Kothari, R.: Fractional-step dimensionality reduction. IEEE Trans. Pattern Anal. Mach. Intell. 22(6), 623–627 (2000)
Zhang, X.Y., Liu, C.L.: Confused distance maximization for large category dimensionality reduction. In: 2012 International Conference on Frontiers in Handwriting Recognition, pp. 213–218. IEEE (2012)
Ghojogh, B., Crowley, M.: Linear and quadratic discriminant analysis: tutorial. arXiv preprint arXiv:1906.02590 (2019)
Zhang, X.Y., Liu, C.L.: Evaluation of weighted Fisher criteria for large category dimensionality reduction in application to Chinese handwriting recognition. Pattern Recogn. 46(9), 2599–2611 (2013)
Hastie, T., Tibshirani, R., Wainwright, M.: Statistical Learning with Sparsity: The Lasso and Generalizations. Chapman and Hall/CRC, London (2015)
Perlibakas, V.: Distance measures for PCA-based face recognition. Pattern Recogn. Lett. 25(6), 711–724 (2004)
Mohammadzade, H., Hatzinakos, D.: Projection into expression subspaces for face recognition from single sample per person. IEEE Trans. Affect. Comput. 4(1), 69–82 (2012)
Tizhoosh, H.R.: Opposition-based learning: a new scheme for machine intelligence. In: International Conference on Computational Intelligence for Modelling, Control and Automation, vol. 1, pp. 695–701. IEEE (2005)
Jain, P., Kar, P.: Non-convex optimization for machine learning. Found. Trends® Mach. Learn. 10(3–4), 142–336 (2017)
Nocedal, J., Wright, S.: Numerical Optimization. Springer, Berlin (2006). https://doi.org/10.1007/978-0-387-40065-5
Hamid, J.S., Greenwood, C.M., Beyene, J.: Weighted kernel Fisher discriminant analysis for integrating heterogeneous data. Comput. Stat. Data Anal. 56(6), 2031–2040 (2012)
Ah-Pine, J.: Normalized kernels as similarity indices. In: Zaki, M.J., Yu, J.X., Ravindran, B., Pudi, V. (eds.) PAKDD 2010. LNCS (LNAI), vol. 6119, pp. 362–373. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13672-6_36
AT&T Laboratories Cambridge: ORL Face Dataset. http://cam-orl.co.uk/facedatabase.html. Accessed 2019
Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.: Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19(7), 711–720 (1997)
Wang, Y.Q.: An analysis of the Viola-Jones face detection algorithm. Image Process. On Line 4, 128–148 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Ghojogh, B., Sikaroudi, M., Tizhoosh, H.R., Karray, F., Crowley, M. (2020). Weighted Fisher Discriminant Analysis in the Input and Feature Spaces. In: Campilho, A., Karray, F., Wang, Z. (eds) Image Analysis and Recognition. ICIAR 2020. Lecture Notes in Computer Science(), vol 12132. Springer, Cham. https://doi.org/10.1007/978-3-030-50516-5_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-50516-5_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-50515-8
Online ISBN: 978-3-030-50516-5
eBook Packages: Computer ScienceComputer Science (R0)