Study of Machine Translated Segment Pairs

Download our study of 73,150 translated segment pairs from translators who provided a set of segment pairs representing their daily work and approved their inclusion in this report. Their translations cover 16 unique language pairs with 5 source languages and 13 target languages across 13 different subject domains.

A translator’s job is a solitary experience. Whether hired as an employee or engaged as independent contractor, a translator works alone and in the moment. His cognitive effort, keystrokes, dictionaries, glossaries, dictation software and other tools, that is, his activities, transform one human language into another. He finds an assistive tool beneficial or detrimental to the degree it contributes to or detract from his work.

Averages of 31 Representative SetsGOOGLE
NMT
CUSTOMIZED
SMT
words per MT segment2020
BLEU score40.0373.27
exact MT match (percent)5.0%34.0%
words per exact MT match (word count)714
filtered BLEU score (no exact MT matches)39.1065.16
character edits per segment (Levenstein)4033
BLEU score – a likeness score (similar to TM fuzzy match) between MT and reference after-the-fact. Higher scores are better and 100 is an exact MT match
exact MT match – MT segment exactly matches the reference human translated segment, AKA BLEU score 100 and 0 edit-distance (Levenstein)

Machine translation (MT) technology has become a mainstream tool in the translation services market. Unfortunately, machine-translated segments are not created equal. Some are detrimental to a translator’s productivity. Therefore, a translator needs benchmarks that can predict, with a high degree of confidence, the likelihood that machine-translated segments from a given system will benefit his daily work productivity.

This study used two well-established quality estimation scores in a unique and innovative paradigm to predict and describe the impact of machine-translated segments as an assistive tool on an individual translator’s daily work. The translator can use these benchmarks to maintain or extend his competitive advantage.

For LSPs: This study opens new opportunities to enhance your competitive advantage through new relationships with translators but this report does not address them. Please contact us to learn more: Click here to contact us


Do you have questions? We’ll reply ASAP.