Natural multimodal interaction with realistic virtual characters provides rich opportunities for entertainment and education. In this paper we present the current VIRTUALHUMAN demonstrator system. It provides a knowledge-based framework to create interactive applications in a multi-user, multi-agent setting. The behavior of the virtual humans and objects in the 3D environment is controlled by interacting affective conversational dialogue engines. An elaborate model of affective behavior adds natural emotional reactions and presence of the virtual humans. Actions are defined in a XML-based markup language that supports the incremental specification of synchronized multimodal output. The system was successfully demonstrated during CeBIT 2006.

Pragmatically important information as e.g. dialogue acts that describe the illocution of an utterance depend in traditional processing approaches on error prone syntactic /semantic processing. We present a statistically based method for dialogue act classification that has word strings as input. An experimental evaluation shows that this method can be successfully used to determine dialogue acts. The overall recognition rate in the experiments is in the range of 65%--67% for German test data, and 74% for an experiment with English dialogues. 1. INTRODUCTION For spoken or typed language processing systems, it is often important to extract just the core intention of a sentence. The classes are sometimes described as dialogue moves [3] or, as in the VERBMOBIL system, as dialogue acts [1]. In the latter system which translates spoken time scheduling dialogues of two humans [2] these dialogue acts classify sentences e.g. as ACCEPT, REJECT, SUGGEST. They are used e.g. in a shallow transla...