Executive Summary

This Second Evaluation Plan (D 5.3) updates and supersedes the First Evaluation Plan (D 5.1).

Major changes from the previous plan are as follows:

  • Following the discussion at the First Review Meeting, we dropped external benchmarks from the evaluation plan. All benchmarks will now be based on data collected and used within MMT.
  • We updated the list of languages to be covered by the evaluations. We now aim to cover the 14 translation directions most relevant to our business case.

The evaluation plan covers data cleaning, automatic translation quality measures, human evaluation, comparison against commercial competitors, and performance and scalability tests.

Download the PDF

1430235416_pdf-128MMT – D5.3 – Second Evaluation Plan (PDF, 110 KB)