Publication
Multi-Modal Post-Editing of Machine Translation
Nico Herbig
PhD-Thesis, Saarland University, 5/2022.
Abstract
As Machine Translation (MT) quality continues to improve, more and more translators switch from traditional translation from scratch to Post-Editing ( PE ) of MT output, which has been shown to save time and reduce errors. Instead of mainly generating text, translators are now asked to correct errors within otherwise helpful translation proposals, where repetitive MT errors make the process tiresome, while hard-to-spot errors make PE a cognitively demanding activity. Our contribution is three-fold: first, we explore whether interaction
modalities other than mouse and keyboard could well support PE by creating and testing the MMPE translation environment. MMPE allows translators to cross out or hand-write text, drag and drop words for reordering, use spoken commands or hand gestures to manipulate text, or to combine any of these input modalities. Second, our interviews revealed that translators see value in automatically receiving additional translation support when a high Cognitive Load ( CL) is detected during PE . We therefore developed a sensor framework
using a wide range of physiological and behavioral data to estimate perceived CL and tested it in three studies, showing that multi-modal, eye, heart, and skin measures can be used to make translation environments cognition-aware. Third, we present two multi-encoder Transformer architectures for Automatic Post-Editing ( APE) and discuss how these can adapt MT output to a domain and thereby avoid correcting repetitive MT errors.