Skip to main content Skip to main navigation

Publication

Findings of the WMT24 General Machine Translation Shared Task:\The LLM Era is Here but MT is Not Solved Yet

Tom Kocmi; Eleftherios Avramidis; Rachel Bawden; Ondvrej Bojar; Anton Dvorkovich; Christian Federmann; Mark Fishel; Markus Freitag; Thamme Gowda; Roman Grundkiewicz; Barry Haddow; Marzena Karpinska; Philipp Koehn; Benjamin Marie; Christof Monz; Kenton Murray; Masaaki Nagata; Martin Popel; Maja Popovic; Mariya Shmatova; Steinþór Steingrímsson; Vilém Zouhar
In: Philipp Koehn; Barry Haddow; Tom Kocmi; Christof Monz (Hrsg.). Proceedings of the Ninth Conference on Machine Translation. Conference on Machine Translation (WMT-24), November 15-16, Miami, Florida, USA, Association for Computational Linguistics, 11/2024.

Abstract

This overview paper presents the results of the General Machine Translation Task organised as part of the 2024 Conference on Machine Translation (WMT). In the general MT task, participants were asked to build machine translation systems for any of 11 language pairs, to be evaluated on test sets consisting of up to four different domains. In addition to participating systems, we collected translations of 8 different LLMs and 4 online translation providers. We evaluate system outputs with professional human annotators using a new protocol called Error Span Annotations.