Skip to main content Skip to main navigation


An experiment setup for collecting data for adaptive output planning in a multimodal dialogue system

Ivana Kruijff-Korbayová; Nate Blaylock; Ciprian Gerstenberger; Verena Rieser; Tilman Becker; Michael Kaißer; Peter Poller; Jan Schehl
In: Graham Wilcock; Kristiina Jokinen; Chris Mellish; Ehud Reiter (Hrsg.). Proceedings of the 10th European Workshop on Natural Language Generation (ENLG-05). European Workshop on Natural Language Generation (ENLG-05), August 8-10, Aberdeen, Scotland, United Kingdom, Pages 191-196, ACL, 8/2005.


We describe a Wizard-of-Oz experiment setup for the collection of multimodal interaction data for a Music Player application. This setup was developed and used to collect experimental data as part of a project aimed at building a flexible multimodal dialogue system which provides an interface to an MP3 player, combining speech and screen input and output. Besides the usual goal of WOZ data collection to get realistic examples of the behavior and expectations of the users, an equally important goal for us was to observe natural behavior of multiple wizards in order to guide our system development. The wizards' responses were therefore not constrained by a script. One of the challenges we had to address was to allow the wizards to produce varied screen output a in real time. Our setup includes a preliminary screen output planning module, which prepares several versions of possible screen output. The wizards were free to speak, and/or to select a screen output.