How to extract predictive binary attributes from a categorical one

  • I. C. Lerman
  • J. F. Pinto da Costa
Part of the Studies in Classification, Data Analysis, and Knowledge Organization book series (STUDIES CLASS)


In this work, we present new ways of dealing with categorical attributes; in particular, the methodology introduced here concerns the use of these attributes in binary decision trees. We consider essentially two main operations; the first one consists in using the joint distribution of two or more categorical attributes in order to increase the final performance of the decision tree; the second - and the most important - operation, concerns the extraction of relatively few predictive binary attributes from a categorical attribute; specially, when the latter has a large number of values. With more than two classes to predict, most of the present binary decision tree software needs to test an exponential number of binary attributes for each categorical attribute; which can be prohibitive. Our method, ARCADE, is independent of the number of classes to be predicted, and it starts by reducing significantly the number of values of the initial categorical attribute. This is done by clustering the initial values, using a hierarchical classification method. Each cluster of values will then represent a new value of a new categorical attribute. This new attribute will then be used in the decision tree, instead of the initial one. Nevertheless, not all of the binary attributes associated with this new categorical attribute will be used; only those which are predictive. The reduction in the complexity of the search for the best binary split is therefore enormous, as will be seen in the application that we consider; that is, the old and still lively protein secondary structure prediction problem.


Decision trees categorical attributes binarization hierarchical clustering 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Almuallim, H., Akiba, Y. & Kaneda, S. (1995). On Handling Tree-Structured Attributes in Decision Tree Learning. International Conference on Machine Learning, 1995.Google Scholar
  2. Asseraf, M. (1996). Extension de la distance de Kolmogorov-Smirnov et stratégie de la coupure optimale. IVème journées de la société francophone de classification (SFC), Vannes, France. Breiman, L., Friedman, J., Olshen, A. & Stone, C. (1984) Classification and Regression Trees. Wadsworth.Google Scholar
  3. Colloc’h, N., Etchebest, C., Thoreau, E., Henrissat, B., & Mornon, J. (1993). Protein Engineering, 6, 377–382.CrossRefGoogle Scholar
  4. Lerman. I.C. (1993). Likelihood linkage analysis (LLA) classification method: An example treated by hand. Biochimie, Elsevier editions, volume 75, pp. 379–397.Google Scholar
  5. Lerman, I.C. & Pinto da Costa, J. (1996). Variables à très grand nombre de catégories dans les arbres de décision. Application à Videntification de la structure secondaire d’une protéine. Rapport de Recherche Inria, 2803, 46 pages.Google Scholar
  6. Pinto da Costa, J. (1996). Coefficients d’association et binarisation par la classification hiérarchique dans les arbres de décision. Application à l’identification de la structure secondaire des protéines. Thèse de l’Université de Rennes I.Google Scholar
  7. Quinlan, J.R. (1993). C4. 5: Programs for Machine Learning. Morgan Kaufman, California.Google Scholar
  8. Rost, B. & Sander, C. (1993). Prediction of protein secondary structures at better than 70%. J. Mol. Biology, 232, pp. 584–599.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin · Heidelberg 1998

Authors and Affiliations

  • I. C. Lerman
    • 1
  • J. F. Pinto da Costa
    • 2
    • 3
  1. 1.IRISA-INRIARennesFrance
  2. 2.Dep. Matemática AplicadaUniversidade do PortoPortugal
  3. 3.LIACC Universidade do PortoPortugal

Personalised recommendations