30.5% Better Than Google NMT For English-to-Croatian EU Legislation

This English-to-Croatian EU Legislation translator uses a Slate Desktop personalized SMT that’s 30.5% more productive than Google NMT.

A Slate Rocks customer (a translator) created a translation engine with his or her translation memories (TMs) using a personal computer. This page describes the engine and compares the translator’s Slate Desktop experience to an experience using Google’s new-and-improved neural machine translation (NMT) technology. You can read the entire report with thirth (30) more customer experiences by downloading the full report Study of Machine Translated Segment Pairs.

Slate Desktop Engine Details

The customer started with translation memories in the language pair and industry of his or her work, totaling the estimated corpus size number of segments. Slate Desktop cleaned the TMs, prepared a training corpus and built the engine. Note that these processes typically runs overnight. During that processing, Slate Desktop also extracted a representative set consisting of randomly selected segments from the training corpus.

sourceen
targethr
subject domainlegislation
estimated corpus size400,000
segments per representative set2,388
words per source segment26
words per target segment23

Benchmark Score Comparison

The segment pairs in the representative set are representative of the translator’s daily work. By focusing on one translator’s experience, these scores indicate a level of work reduction this customer will likely experience in his or her daily work using the respective MT system (Google or Slate Desktop) with 95% confidence.

 Google
NMT
Slate Desktop
SMT
words per MT segment2323
BLEU score36.6577.21
exact MT match (count)106801
exact MT match (percent)4.4%33.5%
words per exact MT match (count)616
filtered BLEU score (no exact MT matches)36.1370.53
segments requiring edit (count)2,2821,587
character edits per segment5134
total character edits116,38253,958

These scores indicate this customer using Slate Desktop will likely spend significantly less time editing MT suggestions than if he or she were using Google for this work. This is because Slate Desktop creates engines with the customers translation memories and optimizes them to predict how the customer translates. While on the other hand, Google optimizes its NMT service for millions of customers with countless demands.

Google’s three (3) longest exact MT matches

The exact MT match (count) in the Benchmark Scores table (above) is the number of segments that Google NMT successfully matched to the translator’s actual work, i.e. Google successfully predicted the translator’s actions. The three segments in this table are the exact MT match segments with the longest length. This translator can expect to experience these kinds Google NMT results while translating these kinds of project.

enhr (Google and translator)
Kobayashi, T., M. Matsuda, H. Kajiura-Kobayashi, A. Suzuki, N. Saito, M. Nakamoto, N. Shibata, and Y.Kobayashi, T., M. Matsuda, H. Kajiura-Kobayashi, A. Suzuki, N. Saito, M. Nakamoto, N. Shibata i Y.
Regulation (EU) 2017/371 of the European Parliament and of the CouncilUredba (EU) 2017/371 Europskog parlamenta i Vijeća
Active yeasts, other than culture yeast and baker’s yeastAktivni kvasci, osim kulture kvasca i pekarskog kvasca

Slate’s three (3) longest exact MT matches

The exact MT match (count) in the Benchmark Scores table (above) is the number of segments that this translator’s Slate Desktop engine successfully matched to the translator’s actual work, i.e. Slate Desktop successfully predicted the translator’s actions. The three segments in this table are the exact MT match segments with the longest length. This translator can expect to experience these kinds Slate Desktop results while translating these kinds of project.

enhr (Slate and translator)
Council Directive 2003/85/EC of 29 of September 2003 on Community measures for the control of foot–and–mouth disease repealing Directive 85/511/EEC and Decisions 89/531/EEC and 91/665/EEC and amending Directive 92/46/EEC (OJ L 306, 22.11.2003, p. 1).Direktiva Vijeća 2003/85/EZ od 29. rujna 2003. o mjerama Zajednice za suzbijanje slinavke i šapa, o stavljanju izvan snage Direktive 85/511/EEZ i odluka 89/531/EEZ i 91/665/EEZ te o izmjeni Direktive 92/46/EEZ (SL L 306, 22.11.2003., str. 1.).
Commission Implementing Regulation (EU) No 799/2014 of 24 July 2014 establishing models for annual and final implementation reports pursuant to Regulation (EU) No 514/2014 of the European Parliament and of the Council laying down general provisions on the Asylum, Migration and Integration Fund and on the instrument for financial support for police cooperation, prevention and combating crime and crisis management (see page 4 of this Official Journal).Provedbena uredba Komisije (EU) br. 799/2014 od 24. srpnja 2014. o utvrđivanju modela godišnjih i završnih izvješća o provedbi u skladu s Uredbom (EU) br. 514/2014 Europskog parlamenta i Vijeća o utvrđivanju općih odredaba o Fondu za azil, migracije te integraciju i o Instrumentu za financijsku potporu u području policijske suradnje, sprečavanja i suzbijanja kriminala te upravljanja krizama (vidjeti stranicu 4 ovoga Službenog lista).
On 13 July 2010, in accordance with Article 126(7) TFEU and Article 3(4) of Council Regulation (EC) No 1467/97 [2], the Council, on the basis of a recommendation from the Commission, addressed a recommendation to Cyprus with a view to bringing the situation of an excessive government deficit to an end by 2012.Vijeće je 13. srpnja 2010., u skladu s člankom 126. stavkom 7. UFEU-a i člankom 3. stavkom 4. Uredbe Vijeća (EZ) br. 1467/97 [2], na temelju preporuke Komisije uputilo preporuku Cipru s ciljem okončanja stanja prekomjernog državnog deficita do 2012.
Advertisements