Assessing Inter-Annotator Agreement for Translation Error Annotation

Arle Richard Lommel, Maja Popovic, Aljoscha Burchardt

In: MTE: Workshop on Automatic and Manual Metrics for Operational Translation Evaluation. International Conference on Language Resources and Evaluation (LREC-14) befindet sich LREC 14 May 26-31 Reykjavik Iceland LREC 2014.


One of the key requirements for demonstrating the validity and reliability of an assessment method is that annotators are able to apply it consistently. This paper explores inter-annotator agreement for error classification task and investigates some of the facts that contribute to low IAA.


LREC-Lommel-Burchardt-Popovic.pdf (pdf, 525 KB )

Deutsches Forschungszentrum für Künstliche Intelligenz
German Research Center for Artificial Intelligence