Regularized Kernel Discriminant Analysis with Optimally Scaled Data

  • Halima Bensmail
  • Hamparsum Bozdogan


Linear discriminant analysis is a well known procedure for discrimination where the linear predictors define one set of variables and a set of dummy variables representing class membership which defines the other set. Here we propose a new method of discriminating between observations using a set of mixed (i.e., categorical and/or continuous) variables. This nonparametric discriminant procedure optimally scales the data and estimates the distribution of the object scores using multivariate kernel density estimation. We propose using Bozdogan’s information-theoretic measure complexity ICOMP to select both the window width of the kernel density estimator as well as the dimension of the object scores matrix.


Linear Discriminant Analysis Window Width Kernel Density Estimator Canonical Discriminant Analysis Object Score 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bensmail, H., and Celeux, G. (1996): Regularized Discriminant Analysis Through Eigenvalue Decomposition, Journal of the American Statistical Association, Vol. 91, No. 436, pp. 1743–1748.MathSciNetCrossRefMATHGoogle Scholar
  2. Bozdogan, H. (1994): Mixture-model cluster analysis using a new informational complexity and model selection criteria. In: H. Bozdogan, ed., Multivariate Statistical Modeling, Vol. 2, Proceedings of the First US/Japan Conference on the Frontiers of Statistical Modeling: An Informational Approach, Kluwer Academic Publishers, The Netherlands, Dordrecht, 69–113.Google Scholar
  3. Bozdogan, H. (2000): Akaike’s Information Criterion and Recent Developments in Information Complexity. Journal of Mathematical Psychology 44, 62–91.MathSciNetCrossRefMATHGoogle Scholar
  4. Devroye, L. (1983): The equivalence of weak, strong and complete convergence in L1 for kernel density estimates. The Annals of Statistics, 11, 896–904.MathSciNetCrossRefMATHGoogle Scholar
  5. Gifi, A. (1990): Non linear Multivariate Analysis. Wiley Series in Probability and Mathematical Statistics, England.Google Scholar
  6. Hand, D.H. (1981): Discrimination and Classification. Wiley Series in Probability and Mathematical Statistics, England.MATHGoogle Scholar
  7. Heiser, W.J., and Meulman, J.J. (1995): Nonlinear methods for the analysis of homogeneity and heterogeneity. In: W.J Krzanowski(Ed.). Recent Advances in Descriptive Multivariate Analysis. Clarendon Press, Oxford.Google Scholar
  8. Meulman, J.J. (1986): A Distance Approach to Nonlinear Multivariate Analysis. DSWO Press, Leiden.Google Scholar
  9. Meulman, J.J. et al. (1990): Prediction of Various Grades of Cervical Preneoplasia and Neoplasia on Plastic Embedded Cytobrush Samples. Technical Report RR-90–06, Department of Data Theory, University of Leiden.Google Scholar
  10. Nishisato, S. (1980): Analysis of categorical data: dual scaling and its applcations. University of Toronto Press, Toronto Buffalo London.Google Scholar
  11. Shunmugan, K. (1977): On a modified form of Parzen estimator for nonparametric pattern recognition. Pattern Recognition, 9, 167–170.CrossRefGoogle Scholar
  12. Tatsuaka, M.M. (1988): Multivariate Analysis. Techniques for Educational and Psychological research. Macmillan publishing Company, New York. Collier Macmillan Publishers, London.Google Scholar
  13. Van der Burg, E. (1985): Homals classification of whales, Porpoises and Dolphins. Proceedings of the International Workshop on Data Analysis. Data Analysis in Real Life Environment: Ins and Outs of Solving Problems. J.F Marcotorchino, J. M. Proth, and J. Janssen(Eds.) Elsevier Science Publishers B. V. (North-Holland).Google Scholar
  14. Van der Burg, E., De Leeuw, J., and Verdegaal, R. (1988): Homogeneity analysis with k sets of variables: an alternating least square method with optimal scaling features. Psychomerika, 53, 177–197.CrossRefMATHGoogle Scholar
  15. Emden, M. H. (1971): An Analysis of Complexity. Mathematical Centre Tracts, Amsterdam.MATHGoogle Scholar
  16. Wertz, W. (1978): Statistical density estimation: A survey. Göttingen, Vandenhoek, and Ruprecht, Monographs in Applied Statistics and Econometrics, No. 13.Google Scholar

Copyright information

© Springer Japan 2002

Authors and Affiliations

  • Halima Bensmail
    • 1
  • Hamparsum Bozdogan
    • 1
  1. 1.Department of Statistics326/336 Stokely Management Ctr.KnoxvilleUSA

Personalised recommendations