26.6% Better Than Google NMT For Swedish-to-English EU Legislation

This Swedish-to-English EU Legislation translator uses a Slate Desktop personalized SMT that’s 26.6% more productive than Google NMT.

A Slate Rocks customer (a translator) created a translation engine with his or her translation memories (TMs) using a personal computer. This page describes the engine and compares the translator’s Slate Desktop experience to an experience using Google’s new-and-improved neural machine translation (NMT) technology. You can read the entire report with thirth (30) more customer experiences by downloading the full report Study of Machine Translated Segment Pairs.

Slate Desktop Engine Details

The customer started with translation memories in the language pair and industry of his or her work, totaling the estimated corpus size number of segments. Slate Desktop cleaned the TMs, prepared a training corpus and built the engine. Note that these processes typically runs overnight. During that processing, Slate Desktop also extracted a representative set consisting of randomly selected segments from the training corpus.

sourcesv
targeten
subject domainlegislation
estimated corpus size180,000
segments per representative set2,372
words per source segment22
words per target segment25

Benchmark Score Comparison

The segment pairs in the representative set are representative of the translator’s daily work. By focusing on one translator’s experience, these scores indicate a level of work reduction this customer will likely experience in his or her daily work using the respective MT system (Google or Slate Desktop) with 95% confidence.

 Google
NMT
Slate Desktop
SMT
words per MT segment2425
BLEU score47.6375.71
exact MT match (count)136731
exact MT match (percent)5.7%30.8%
words per exact MT match (count)1016
filtered BLEU score (no exact MT matches)46.4470.15
segments requiring edit (count)2,2361,641
character edits per segment4740
total character edits105,09265,640

These scores indicate this customer using Slate Desktop will likely spend significantly less time editing MT suggestions than if he or she were using Google for this work. This is because Slate Desktop creates engines with the customers translation memories and optimizes them to predict how the customer translates. While on the other hand, Google optimizes its NMT service for millions of customers with countless demands.

Google’s three (3) longest exact MT matches

The exact MT match (count) in the Benchmark Scores table (above) is the number of segments that Google NMT successfully matched to the translator’s actual work, i.e. Google successfully predicted the translator’s actions. The three segments in this table are the exact MT match segments with the longest length. This translator can expect to experience these kinds Google NMT results while translating these kinds of project.

sven (Google and translator)
Texterna till förordningarna (EU) nr 550/2010, (EU) nr 574/2010, (EU) nr 632/2010, (EU) nr 633/2010 och (EU) nr 662/2010 på isländska och norska, som ska offentliggöras i EES-supplementet till Europeiska unionens officiella tidning, ska vara giltiga.The texts of Regulations (EU) No 550/2010, (EU) No 574/2010, (EU) No 632/2010, (EU) No 633/2010 and (EU) No 662/2010 in the Icelandic and Norwegian languages, to be published in the EEA Supplement to the Official Journal of the European Union, shall be authentic.
Kommissionens förordning (EG) nr 327/98 av den 10 februari 1998 om öppnande och förvaltning av vissa tullkvoter för import av ris och brutet ris [3] har ändrats flera gånger [4] på ett väsentligt sätt.Commission Regulation (EC) No 327/98 of 10 February 1998 opening and providing for the administration of certain tariff quotas for imports of rice and broken rice [3] has been substantially amended several times [4].
Medlemsstaterna ska senast den 31 december 2011 anta och offentliggöra de lagar och andra författningar som är nödvändiga för att följa detta direktiv.Member States shall adopt and publish, by 31 December 2011 at the latest, the laws, regulations and administrative provisions necessary to comply with this Directive.

Slate’s three (3) longest exact MT matches

The exact MT match (count) in the Benchmark Scores table (above) is the number of segments that this translator’s Slate Desktop engine successfully matched to the translator’s actual work, i.e. Slate Desktop successfully predicted the translator’s actions. The three segments in this table are the exact MT match segments with the longest length. This translator can expect to experience these kinds Slate Desktop results while translating these kinds of project.

sven (Slate and translator)
Kommissionen har i enlighet med artikel 9.1 första stycket i förordning (EG) nr 510/2006 granskat Förenade kungarikets ansökan om godkännande av ändringar av produktspecifikationen för den skyddade geografiska beteckningen ”Welsh Beef”, vilken registrerades i enlighet med kommissionens förordning (EG) nr 2400/96 [2] i dess ändrade lydelse enligt förordning (EG) nr 2066/2002 [3].In accordance with the first subparagraph of Article 9(1) of Regulation (EC) No 510/2006, the Commission has examined the United Kingdom’s application for the approval of amendments to the specification for the protected geographical indication ‘Welsh Beef’ registered in accordance with Commission Regulation (EC) No 2400/96 [2], as amended by Regulation (EC) No 2066/2002 [3].
I det avseendet noterar övervakningsmyndigheten att de norska myndigheterna bara lämnat allmänna kommentarer om hur de måste genomföra åtgärderna på grund av Hurtigrutens svaga ekonomiska ställning [35] för att se till att företaget skulle fortsätta att fullgöra den allmänna trafikplikten, eftersom det vore svårt för de norska myndigheterna att hitta ett annat företag som kan tillhandahålla tjänsten (åtminstone på kort till medellång sikt).In that regard, the Authority notes that the Norwegian authorities have only made general remarks on how they had to implement the measures due to Hurtigruten’s weak financial position [35] in order to ensure that it would continue to provide the public service, as it would be difficult, for the Norwegian authorities, to find another undertaking to provide the service (at least in the short to medium term).
Rådets förordning (EG) nr 1275/2005 av den 26 juli 2005 om ändring av förordning (EG) nr 2268/2004 om införande av en slutgiltig antidumpningstull på import av volframkarbid och smält volframkarbid med ursprung i Folkrepubliken Kina (EUT L 202, 3.8.2005, s. 1).Council Regulation (EC) No 1275/2005 of 26 July 2005 amending Regulation (EC) No 2268/2004 imposing a definitive anti-dumping duty on imports of tungsten carbide and fused tungsten carbide originating in the People’s Republic of China (OJ L 202, 3.8.2005, p. 1).
Advertisements