Inspecting Document Collections

Abstract

The paper introduces two procedures which allow information seekers to inspect large document collections. The first method structures document collections into sensible groups. Here, three different approaches are presented: grouping based on the topology of the collection (i.e. linking and directory structure of intranet documents), grouping based on the content of the documents (i.e. similarity relation), and grouping based on the reader’s behavior when using the document collection. After the formation of groups, the second method supports readers by characterizing text through extracting short and relevant information from single documents and groups. Using statistical approaches, representative keywords of each document and also of the document groups are calculated. Later, the most important sentences from single documents and document groups are extracted as summaries. Geared to the different information needs, algorithms for indicative, informative, and thematic summaries are developed. In this process, special care is taken to generate readable and sensible summaries. Finally, we present three applications which utilize these procedures to fulfill various information-seeking needs.