Text clustering to help knowledge acquisition from documents

Abstract

At the earlier stage of the knowledge acquisition process, interviews of experts produce a large amount of rich but ill-structured texts. Knowledge engineers need some tool to help them in the exploitation of all these texts. We propose the use of a statistical method, the top-down hierarchical classification and a new interpretation of its results. The initial statistical analysis proposed by M. Reinert [16, 17] gives two kinds of results: first a segmentation of texts that reflects their “semantic contexts” that we use to raise structures of texts, and second, classes of significant terms belonging to these contexts, which can be related to the experts or to these specialities. In this paper, we describe the method, its empirical validity and a comparison with similar approaches, its uses with examples and results. We conclude with some research directions to extend the exploitation of the analysis results.

All examples come from a research supported by the French “Ministère de la Recherche et de l'Espace” under contract n.92 C 0757 and the French “Ministère de l'Equipement, des Transports et du Tourisme” under contract n. 93.0003.