Project

CORA4NLP

Co(n)textual Reasoning and Adaptation for Natural Language Processing

Co(n)textual Reasoning and Adaptation for Natural Language Processing

  • Duration:

Language is implicit - it omits information. Filling this information gap requires contextual inference, background- and commonsense knowledge, and reasoning over situational context. Language also evolves, i.e., it specializes and changes over time. For example, many different languages and domains exist, new domains arise, and both evolve constantly. Thus, language understanding also requires continuous and efficient adaptation to new languages and domains, and transfer to, and between, both. Current language understanding technology, however, focuses on high resource languages and domains, uses little to no context, and assumes static data, task, and target distributions.

The research in Cora4NLP aims to address these challenges. It builds on the expertise and results of the predecessor project DEEPLEE and is carried out jointly between the language technology research departments in Berlin and Saarbrücken. Specifically, our goal is to develop natural language understanding methods that enable:

  • reasoning over broader co- and contexts;
  • efficient adaptation to novel and/or low resource contexts;
  • continuous adaptation to, and generalization over, evolving contexts.

To achieve this, we pursue the following research directions:

  • memory- and language model-augmented few- and zero-shot learning;
  • self- and weakly-supervised pre-training for low-resource domains and long-tail classes;
  • multi-lingual, intra- and inter-document, and dialogue context representations;
  • integration of structured domain knowledge, background- and commonsense knowledge;
  • continual learning for open-domain and supervised tasks multi-hop contextual reasoning.

The resulting methods will be applied in the context of various natural language understanding tasks, such as information extraction, question answering, machine translation, and dialogue.

Publications about the project

Christoph Alt, David Harbecke, Yuxuan Chen, Leonhard Hennig

In: Proceedings of the 1st Workshop on Efficient Benchmarking in NLP. Annual Meeting of the Association for Computational Linguistics (ACL-2022) May 22-27 Dublin Ireland Association for Computational Linguistics 5/2022.

To the publication
Morgan Wixted, Alejandro García-Rudolph, Catalina Martínez-Costa, Saadullah Amin, Noon Pokaratsiri, Günter Neumann

In: Proceedings of the 21st Workshop on Biomedical Language Processing. Annual Meeting of the Association for Computational Linguistics (ACL-2022) BioNLP May 22-27 Dublin Ireland Pages 200-211 ACL 5/2022.

To the publication
Saadullah Amin, Günter Neumann

In: Proceedings of the 7th Workshop on Representation Learning for NLP. Annual Meeting of the Association for Computational Linguistics (ACL-2022) RepL4NLP May 22-27 Dublin Ireland Pages 111-120 ACL 5/2022.

To the publication

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz