SmartKom: Multimodal Communication with a Life-Like Character

Wolfgang Wahlster, Norbert Reithinger, Anselm Blocher

In: Proceedings of the 7th European Conference on Speech Communication and Technology. European Conference on Speech Communication and Technology (EUROSPEECH-2001) September 3-7 Aalborg Denmark Seiten 1547-1550 3 ISCA 9/2001.


SmartKom is a multimodal dialog system that combines speech, gesture, and mimics input and output. Spontaneous speech understanding is combined with the video-based recognition of natural gestures. One of the major scientific goals of SmartKom is to design new computational methods for the seamless integration and mutual disambiguation of multimodal input and output on a semantic and pragmatic level. SmartKom is based on the situated delegation-oriented dialog paradigm, in which the user delegates a task to a virtual communication assistant, visualized as a life-like character on a graphical display. We describe the SmartKom architecture, the use of an XML-based mark-up language for multimodal content, and some of the distinguishing features of the first fully operational SmartKom demonstrator.


Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence