Mobile Speech-to-Speech Translation of Spontaneous Dialogs: An Overview of the Final Verbmobil System

Abstract

Verbmobil is a speaker-independent and bidirectional speech-to-speech translation system for spontaneous dialogs in mobile situations. It recognizes spoken input, analyses and translates it, and finally utters the translation. The multilingual system handles dialogs in three business-oriented domains, with context-sensitive translation between three languages (German, English, and Japanese). Since Verbmobil emphasizes the robust processing of spontaneous dialogs, it poses difficult challenges to human language technology, that we discuss in this paper. We present Verbmobil as a hybrid system incorporating both deep and shallow processing schemes. We describe the anatomy of Verbmobil and the functionality of its main components. We discuss Verbmobil’s multi-blackboard architecture that is based on packed representations at all processing stages. These packed representations together with formalisms for underspecification capture the non-determinism in each processing phase, so that the remaining uncertainties can be reduced by linguistic, discourse and domain constraints as soon as they become applicable. We present Verbmobil’s multi-engine approach, eg. its use of five concurrent translation engines: statistical translation, case-based translation, substring-based translation, dialog-act based translation, and semantic transfer. Distinguishing features like the multilingual prosody module and the generation of dialog summaries are highlighted. We conclude that Verbmobil has successfully met the project goals with more than 80% of approximately correct translations and a 90% success rate for dialog tasks.