Publication
SmartKom: Towards Multimodal Dialogues with Anthropomorphic Interface Agents
Wolfgang Wahlster; Norbert Reithinger; Anselm Blocher
In: Gottfried Wolf; Gunther Klein (Hrsg.). Proceedings of the International Status Conference "Human-Computer Interaction". International Status Conference Human-Computer Interaction, Germany, Pages 23-34, 10/2001.
Abstract
SmartKom is a multimodal dialogue system that combines speech, gesture, and facial
expressions for input and output. SmartKom provides an anthropomorphic and affective user
interface through its personification of an interface agent. Understanding of spontaneous
speech is combined with video-based recognition of natural gestures and facial expressions.
One of the major scientific goals of SmartKom is to design new computational methods for
the seamless integration and mutual disambiguation of multimodal input and output on a
semantic and pragmatic level. SmartKom is based on the situated delegation-oriented
dialogue paradigm, in which the user delegates a task to a virtual communication assistant,
visualized as a life-like character on a graphical display. SmartKom is a multilingual system
that analyses and generates German and English utterances. We describe the SmartKom
architecture, the use of an XML-based mark-up language for multimodal content, and the
most distinguishing features of the fully operational SmartKom 2.0 system.
1.