Publication
Redundancy Localization for the Conversationalization of Unstructured Responses
Sebastian Krause; Mikhail Kozhevnikov; Eric Malmi; Daniele Pighin
In: Proceedings of the 18th Annual Meeting of the Special Interest Group on Discourse and Dialogue. Annual SIGdial Meeting on Discourse and Dialogue (SIGdial-2017), August 15-17, Saarbrücken, Germany, Association for Computational Linguistics, 2017.
Abstract
Conversational agents offer users a natural- language interface to accomplish tasks, entertain themselves, or access information. Informational dialogue is particularly challenging in that the agent has to hold a conversation on an open topic, and to achieve a reasonable coverage it generally needs to digest and present unstructured information from textual sources. Making responses based on such sources sound natural and fit appropriately into the conversation con- text is a topic of ongoing research, one of the key issues of which is preventing the agent’s responses from sounding repetitive. Targeting this issue, we propose a new task, known as redundancy localization, which aims to pinpoint semantic overlap between text passages. To help address it systematically, we formalize the task, prepare a public dataset with fine-grained redundancy labels, and propose a model utilizing a weak training signal defined over the results of a passage-retrieval system on web texts. The proposed model demonstrates superior performance compared to a state-of-the-art entailment model and yields encouraging results when applied to a real-world dialogue.