Multilinguality and Language Technology

Modern NLP models and LLMs have specific flaws, despite being highly performant: First, they are black boxes: Parameters of proprietary models are not accessible at all; and even non-proprietary models are largely opaque in the sense that it is unclear where exactly specific knowledge is encoded in potentially billions of parameters. Second, there is a tendency to always increase the size of LLMs and training data to improve performance, which is especially problematic for domains or languages with fewer resources.

The E&E group of DFKI’s Research Department Multilinguality and Language Technology works on transparent and efficient NLP models. Our objective is to make the parameters and behaviour of LLMs more explainable and understandable to both end users and researchers. We try to improve LLMs with regard to data consumption, e.g. for domains or languages where data is scarce, by using structured data, new learning techniques, or other modalities; and in terms of model size, e.g. for settings where powerful hardware is not available.

We are involved in Twinning projects, where we provide knowledge transfer both on research topics and project management to newly established research institutions across Europe. We are involved in European procurement projects focusing on language resources, such as the European Language Resource Coordination and the Language Data Space.