This document presents the First Evaluation Plan for the Project “Modern Machine Translation” (MMT) and is intended to guide the project-internal evaluation process until the First Technology Assessment Report (D-5.2), which is due in M17. It will be superseded by the Second Evaluation Plan (D-5.3; M19).

In a nutshell, the goals of the project are to provide

  • better translation quality;
  • faster translations without noticeable loss of translation quality;
  • short update cycles with respect to integrating and exploiting parallel data as it is being acquired; and
  • scalability with respect to the volumes of parallel data and translation throughput that can be handled by the MT infrastructure.

To reach these goals, MMT will collect large amounts of parallel and monolingual language data (WP2), improve or replace essential components in the MT pipeline to achieve greater speed, throughput and robustness across a variety of input domains (WP3), and implement these improvements in a robust, scalable, commercial product (WP4). The mandate of WP5 is it to monitor progress towards these goals. The following sections cover assessment of the quality of data acquired in WP2, testing of the MT component(s) developed in WP3, and performance and scalability tests (for WP4), in this order.

Download the PDF

1430235416_pdf-128MMT – D5.1 – First Evaluation Plan (PDF, 134 KB)