The lexical analysis of French

Abstract

The automatic linguistic analysis of texts requires basic information about the simple and compound words of the text. Lexical analysis is the preliminary step before syntactic analysis. We have shown that important linguistic problems appear during this basic step. Some of them cannot yet be solved (recognition of proper names, compound verbs, and so on); others, if solved during lexical analysis, facilitates the syntactic analysis by reducing the degree of ambiguity of the text.

The lexical parser is based on a program (automaton) which could be used in more general cases in order to disambiguate some strings. For example, we have described the context of the string j'; in the same way, it would be possible to describe the context of the word je, which appears only in a limited number of schemas.

We have given an enumeration of problems that have to be solved in order to recognize words. Each of these problems is well-known. What we have attempted here is to formulate them in such a form that they can be represented by finite automata and treated by the corresponding algorithms. It should be clear that the number of automata to be built, their size and the formulation of their interaction is by no mean trivial: a complex program is required simply to recognize the words of a text.

Unit 819 of the CNRS. This work has been partly financed by the Programme de Recherches Coordonnées «Informatique Linguistique» of the Ministry of Research and Technology.

FIRTECH Industries de la langue française.