29.1% Better Than Google NMT For English-to-Czech EU Legislation

This English-to-Czech EU Legislation translator uses a Slate Desktop personalized SMT that’s 29.1% more productive than Google NMT.

A Slate Rocks customer (a translator) created a translation engine with his or her translation memories (TMs) using a personal computer. This page describes the engine and compares the translator’s Slate Desktop experience to an experience using Google’s new-and-improved neural machine translation (NMT) technology. You can read the entire report with thirth (30) more customer experiences by downloading the full report Study of Machine Translated Segment Pairs.

Slate Desktop Engine Details

The customer started with translation memories in the language pair and industry of his or her work, totaling the estimated corpus size number of segments. Slate Desktop cleaned the TMs, prepared a training corpus and built the engine. Note that these processes typically runs overnight. During that processing, Slate Desktop also extracted a representative set consisting of randomly selected segments from the training corpus.

sourceen
targetcs
subject domainlegislation
estimated corpus size160,000
segments per representative set2,366
words per source segment27
words per target segment25

Benchmark Score Comparison

The segment pairs in the representative set are representative of the translator’s daily work. By focusing on one translator’s experience, these scores indicate a level of work reduction this customer will likely experience in his or her daily work using the respective MT system (Google or Slate Desktop) with 95% confidence.

 Google
NMT
Slate Desktop
SMT
words per MT segment2325
BLEU score44.1274.57
exact MT match (count)185819
exact MT match (percent)7.8%34.6%
words per exact MT match (count)616
filtered BLEU score (no exact MT matches)43.2167.32
segments requiring edit (count)2,1811,547
character edits per segment5344
total character edits115,59368,068

These scores indicate this customer using Slate Desktop will likely spend significantly less time editing MT suggestions than if he or she were using Google for this work. This is because Slate Desktop creates engines with the customers translation memories and optimizes them to predict how the customer translates. While on the other hand, Google optimizes its NMT service for millions of customers with countless demands.

Google’s three (3) longest exact MT matches

The exact MT match (count) in the Benchmark Scores table (above) is the number of segments that Google NMT successfully matched to the translator’s actual work, i.e. Google successfully predicted the translator’s actions. The three segments in this table are the exact MT match segments with the longest length. This translator can expect to experience these kinds Google NMT results while translating these kinds of project.

encs (Google and translator)
This Agreement shall enter into force on the first day of the second month following the date on which the Parties notify each other of the completion of the legal procedures necessary for this purpose.Tato dohoda vstupuje v platnost prvním dnem druhého měsíce následujícího po dni, kdy si strany navzájem oznámí dokončení právních postupů nezbytných pro tento účel.
These Rules of Procedure may be amended by a decision of the Association Council in accordance with Article 408(1) of the Agreement.Tento jednací řád může být změněn rozhodnutím Rady přidružení v souladu s čl. 408 odst. 1 dohody.
The Netherlands has questioned the impact of possible aid on the internal market for clubs not playing football at European level.Nizozemsko zpochybnilo dopad možné podpory na vnitřní trh pro kluby, které nehrají fotbal na evropské úrovni.

Slate’s three (3) longest exact MT matches

The exact MT match (count) in the Benchmark Scores table (above) is the number of segments that this translator’s Slate Desktop engine successfully matched to the translator’s actual work, i.e. Slate Desktop successfully predicted the translator’s actions. The three segments in this table are the exact MT match segments with the longest length. This translator can expect to experience these kinds Slate Desktop results while translating these kinds of project.

encs (Slate and translator)
Council Decision 2001/924/EC of 17 December 2001 extending the effects of the Decision establishing an exchange, assistance and training programme for the protection of the euro against counterfeiting (‘Pericles’ programme) to the Member States which have not adopted the euro as the single currency (OJ L 339, 21.12.2001, p. 55).Rozhodnutí Rady 2001/924/ES ze dne 17. prosince 2001, kterým se rozšiřuje působnost rozhodnutí o akčním programu výměny, pomoci a odborného vzdělávání za účelem ochrany eura proti padělání (program „Pericles“) na členské státy, které nepřijaly euro jako jednotnou měnu (Úř. věst. L 339, 21.12.2001, s. 55).
The first working day following receipt of the request is […].This request, made by [name of the Member State concerned/name of the relevant contracting entity] [3], concerns [brief mention of the relevant sector or activity] in [that country/indicate the Member State concerned].Prvním pracovním dnem následujícím po obdržení žádosti je […].Tato žádost, kterou podal [název dotčeného členského státu/název příslušného zadavatele] [3], se týká [stručný popis příslušného odvětví nebo činnosti] v [této zemi/uveďte dotčený členský stát].
Commission Regulation (EC) No 391/2007 of 11 April 2007 laying down detailed rules for the implementation of Council Regulation (EC) No 861/2006 as regards the expenditure incurred by Member States in implementing the monitoring and control systems applicable to the Common Fisheries Policy (OJ L 97, 12.4.2007, p. 30).Nařízení Komise (ES) č. 391/2007 ze dne 11. dubna 2007, kterým se stanoví prováděcí pravidla k nařízení Rady (ES) č. 861/2006, pokud jde o výdaje vzniklé členským státům při provádění monitorovacích a kontrolních systémů použitelných pro společnou rybářskou politiku (Úř. věst. L 97, 12.4.2007, s. 30).