Extended Fisher Criterion Based on Auto-correlation Matrix Information

  • Hitoshi Sakano
  • Tsukasa Ohashi
  • Akisato Kimura
  • Hiroshi Sawada
  • Katsuhiko Ishiguro
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7626)

Abstract

Fisher’s linear discriminant analysis (FLDA) has been attracting many researchers and practitioners for several decades thanks to its ease of use and low computational cost. However, FLDA implicitly assumes that all the classes share the same covariance: which implies that FLDA might fail when this assumption is not necessarily satisfied. To overcome this problem, we propose a simple extension of FLDA that exploits a detailed covariance structure of every class by utilizing revealed by the class-wise auto-correlation matrices. The proposed method achieves remarkable improvements classification accuracy against FLDA while preserving two major strengths of FLDA: the ease of use and low computational costs. Experimental results with MNIST and other several data sets in UCI machine learning repository demonstrate the effectiveness of our method.

Keywords

Statistical Pattern Recognition Discriminant Axis MNIST Dataset Discriminative Feature Extractor Simple Matrix Operation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Fisher, R.A.: The use of multiple measurements in taxonomic problems. Annals of Eigenics 7, 179–188 (1936)Google Scholar
  2. 2.
    Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. John Willey and Sons (1973)Google Scholar
  3. 3.
    Belhumeur, P.N., et al.: Eigenfaces vs Fisherfaces: Recognition using class specific linear projection. IEEE Transaction of Pattern analysis and Machine PAMI 19, 711–720 (1997)CrossRefGoogle Scholar
  4. 4.
    Hastie, T., Buja, A., Tibshirani, R.: Penelized Discriminant Analysis. The Annals of Statistics 23(1), 73–102 (1995)MathSciNetMATHCrossRefGoogle Scholar
  5. 5.
    Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press (1990)Google Scholar
  6. 6.
    Baudat, G., Anouar, F.: Generalized Discriminant Analysis Using a Kernel Approach. Neural Computation 12(10), 2385–2404 (2006)CrossRefGoogle Scholar
  7. 7.
    Sierra, A.: High-order Fishers discriminant analysis. Pattern Recognition 35(6), 1291–1302 (2002)MATHCrossRefGoogle Scholar
  8. 8.
    Hastie, T., Tibshirani, R.: Discriminant Analysis by Gaussian Mixture. J. Royal Society of Statistical. Soc. B. 58, 155–176 (1996)MathSciNetMATHGoogle Scholar
  9. 9.
    Zhu, M., Martinez, A.M.: Subclass discriminant analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(8), 1274–1286 (2006)CrossRefGoogle Scholar
  10. 10.
    Gkalelis, N., Mezaris, V., Kompatsiaris, I.: Mixture subclass discriminant analysis. IEEE Signal Processing Letters 18(5), 319–322 (2011)CrossRefGoogle Scholar
  11. 11.
    Sakano, H.: A Brief History of the Subspace Methods. In: Koch, R., Huang, F. (eds.) ACCV Workshops 2010, Part II. LNCS, vol. 6469, pp. 434–435. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  12. 12.
    Decell, H.P., Mayekar, S.M.: Feature Combinations and the Divergence Criterion. Computers and Math. with Applications 3, 71–76 (1977)MATHCrossRefGoogle Scholar
  13. 13.
    Loog, M., Duin, R.P.W.: Linear dimensionality reduction via a heteroscedastic extension of LDA: the Chernoff criterion. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(6), 732–739 (2004)CrossRefGoogle Scholar
  14. 14.
    Watanabe, S., Lambert, P.F., Kulikowski, C.A., Buxton, J.L., Walker, R.: Evaluation and selection of variables in pattern recognition. Comp. & Info. Sciences 2, 91–122 (1967)Google Scholar
  15. 15.
    Lim, G., Park, C.H.: Semi-supervised Dimension Reduction Using Graph-Based Discriminant Analysis. In: 2009 Ninth IEEE International Conference on Computer and Information Technology, pp. 9–13 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Hitoshi Sakano
    • 1
  • Tsukasa Ohashi
    • 2
  • Akisato Kimura
    • 1
  • Hiroshi Sawada
    • 1
  • Katsuhiko Ishiguro
    • 1
  1. 1.NTT Communication Science LaboratoriesNTT CorporationSoraku-gunJapan
  2. 2.Graduate School of EngineeringDoshisha UniversityKyotanabe-shiJapan

Personalised recommendations