Self-Supervised Neural Machine Translation

Cristina España-Bonet, Josef van Genabith, Dana Ruiter

In: 57th Annual Meeting of the Association for Computational Linguistics. Annual Meeting of the Association for Computational Linguistics (ACL-2019) 57th July 28-August 2 Florence Italy ACL 2019.


We present a simple new method where an emergent NMT system is used for simultaneously selecting training data and learning internal NMT representations. This is done in a self-supervised way without parallel data, in such a way that both tasks enhance each other during training. The method is language independent, introduces no additional hyper-parameters, and achieves BLEU scores of 29.21 (en2fr) and 27.36 (fr2en) on newstest2014 using English and FrenchWikipedia data for training.


ACL_Paper_final_1.pdf (pdf, 281 KB )

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz