Co(n)textual Reasoning and Adaptation for Natural Language Processing

Co(n)textual Reasoning and Adaptation for Natural Language Processing

  • Duration:
  • Application fields

Language is implicit - it omits information. Filling this information gap requires contextual inference, background- and commonsense knowledge, and reasoning over situational context. Language also evolves, i.e., it specializes and changes over time. For example, many different languages and domains exist, new domains arise, and both evolve constantly. Thus, language understanding also requires continuous and efficient adaptation to new languages and domains, and transfer to, and between, both. Current language understanding technology, however, focuses on high resource languages and domains, uses little to no context, and assumes static data, task, and target distributions.

The research in Cora4NLP aims to address these challenges. It builds on the expertise and results of the predecessor project DEEPLEE and is carried out jointly between the language technology research departments in Berlin and Saarbrücken. Specifically, our goal is to develop natural language understanding methods that enable:

  • reasoning over broader co- and contexts;
  • efficient adaptation to novel and/or low resource contexts;
  • continuous adaptation to, and generalization over, evolving contexts.

To achieve this, we pursue the following research directions:

  • memory- and language model-augmented few- and zero-shot learning;
  • self- and weakly-supervised pre-training for low-resource domains and long-tail classes;
  • multi-lingual, intra- and inter-document, and dialogue context representations;
  • integration of structured domain knowledge, background- and commonsense knowledge;
  • continual learning for open-domain and supervised tasks multi-hop contextual reasoning.

The resulting methods will be applied in the context of various natural language understanding tasks, such as information extraction, question answering, machine translation, and dialogue.


BMBF - Federal Ministry of Education and Research


BMBF - Federal Ministry of Education and Research

Share project:

Contact Person

Publications about the project

Saadullah Amin; Pasquale Minervini; David Chang; Pontus Stenetorp; Günter Neumann

In: Proceedings of the 29th International Conference on Computational Linguistics. International Conference on Computational Linguistics (COLING-2022), located at 29th International Conference on Computational Linguistics, October 12-17, Gyeongju, Korea, Republic of, International Committee on Computational Linguistics (ICCL), 10/2022.

To the publication

Tatiana Anikina; Natalia Skachkova; Joseph Renner; Priyansh Trivedi

In: Juntao Yu; Sopan Khosla; Ramesh Manuvinakurike; Lori Levin; Vincent Ng; Massimo Poesio; Michael Strube; Carolyn Rose (Hrsg.). Proceedings of the CODI-CRAC 2022 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue. Workshop on Computational Models of Reference, Anaphora and Coreference (CRAC-2022), located at COLING, October 16-17, Gyeongju, Korea, Republic of, Pages 15-27, Association for Computational Linguistics, 10/2022.

To the publication

David Harbecke; Yuxuan Chen; Leonhard Hennig; Christoph Alt

In: Proceedings of the 1st Workshop on Efficient Benchmarking in NLP. Annual Meeting of the Association for Computational Linguistics (ACL-2022), May 22-27, Dublin, Ireland, Association for Computational Linguistics, 5/2022.

To the publication

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz