Believable Virtual Characters in Human-Computer Dialogs: State of The Art Report

Y. Jung, A. Kuijper, D.W. Fellner, Michael Kipp, Jan Miksatko, J. Gratch, D. Thalmann

In: 32nd Annual Conference of the European Association for Computer Graphics. Eurographics (EG-11) 32nd April 11-15 Llandudno United Kingdom ACM Press 2011.


For many application areas, where a task is most naturally represented by talking or where standard input devices are difficult to use or not available at all, virtual characters can be well suited as an intuitive man-machine interface due to their inherent ability to simulate verbal as well as nonverbal communicative behavior. This type of interface is made possible with the help of multimodal dialog systems, which extend common speech dialog systems with additional modalities just like in human-human interaction. Multimodal dialog systems consist at least of an auditive and graphical component, and communication is based on speech and nonverbal communication alike. However, employing virtual characters as personal and believable dialog partners in multimodal dialogs entails several challenges, because this not only requires a reliable and consistent motion and dialog behavior that also regards nonverbal communication and affective components, but also efficiency for the realization of applications. Besides modeling the “mind” and creating intelligent communication behavior, which is an active field of research in artificial intelligence, the visual representation of a character including its perceivable behavior, such as facial expressions and gestures, belongs to the domain of computer graphics and likewise implicates many open issues concerning natural communication. In this report we discuss current techniques for designing and implementing the presentation component of such a dialog system.

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz