In this paper, we present a method of improving quality of machine translation (MT) evaluation of Czech sentences via targeted paraphrasing of reference sentences on a deep syntactic layer. For this purpose, we employ NLP framework Treex and extend it with modules for targeted paraphrasing and word order changes.
Automatic scores computed using these paraphrased reference sentences show higher correlation with human judgment than scores computed on the original reference sentences.