It seems that MT developers are not really in tune with what translators would like to have. Most likely, they aim at translating as much content as possible in as little time as possible, quality becoming a (not very important) variable. Is this really so, or are there also projects devoted to excellent quality?

  • Published: 2 years ago on 9 January 2018


  1. Aljoscha Burchardt says:

    It is true that some researchers have their own scientific goals that are not immediately matching the needs of translators. One of the reasons for this is that MT research has traditionally focussed on gisting or information translation where the goal is to understand what foreign documents or web pages are about. Public funding especially in the US had a focus on this kind of translation. Translators need something very different, namely something that works like a translation memory, ideally producing perfect translations or almost good translations and not showing trash to the users as this is a waste of their time.

    Another reason is that the prevailing engineering approach to MT research is doing statistical, i.e., data-driven research where it is necessary to get immediate automated feedback about the “quality” during engine development. Unfortunately, evaluating MT quality is as difficult if not more difficult than translation itself. In practice, MT researchers use reference texts and corpora pre-translated by translators to compare them automatically with the MT output. As one can imagine, the simple, surface-based algorithms that perform the comparison cannot really measure quality. They rather measure the “distance” to one good translation, basically how many words or sub-strings match the reference. These measures have shown to somehow correlate with quality as judged by humans, though.

    Another challenge is that the comparisons are only reliably on corpus level, they cannot assess the quality of single segments.

    For quite some time, we have been working on better ways for measuring MT quality such as the MQM framework (see our paper “Towards a Systematic and Human-Informed Paradigm for High-Quality Machine Translation” at, but the “problem” remains that real quality judgements can only be provided by humans.

    In the QT21 project we have developed specific test suites that can be used for semi-automatically assessing quality in a more analytical way.

    One development that increased the interest in better quality measures is the turn from traditional statistical MT to neural MT. As the latter systems output is more divers (“creative”), the standard reference-based measures are no longer fine grained enough to measure improvements. This gives me hope that the quality issue will eventually become more prominent and at the same time the needs of translators.

  2. I couldn’t resist commenting. Well written!

  3. Lindiwe says:

    I do agree with the notion that machine aided translation requires on one hand a lot of editing and double checking to ensure high quality however, on the other hand, I do find a lot of relief in saving time through machine translation.

    All I need presently is to master my machine translation skills regarding perfecting my utilization of the tools availed to me (e.g. Autshumato programme software) as well as improving my typing speed in order to can undertake as many projects as possible as a freelancer.

    I also feel there is a need to be exposed to more than one machine translation skills programmes for purposes of preference options. I have found limited resources on my side to have contributed to my slow progress in growing my business as I presently do not afford exposure to other programmes or softwares.

    In addition, agencies providing capacity building in machine translation related programmes need to access more funding (e.g. from the National Department of Arts & Culture) to target more novice candidates. This will allow adequate time for the skills training to ensure mastering the use of translation tools including acquiring the machines towards job creation by translators.

    High quality in machine translation can be achieved for as long as the above are given attention as once a document has gone through the phase of translation then all that is left is editing, localization and proof-reading which would have taken almost 50% of the time of a “normal” or “manual’ word for word translation exercise. This also gives time for cross referencing and consultations with the client.

    For as long as a translator ensures thorough checking of the machine translated work, the high quality required by a client will always be achieved.

    I have also observed some clients requiring that their translated work be strictly “manually” translated (i.e. no machine translated) as part of terms & conditions. Whilst the client may have very good reasons for their demand, I think this at the same time affects the delivery time in a negative way on the side of the translator.

    Whilst I may not have provided a scientific view of my response to the topic, I confirm that this remains my honest view based on my own experience during my short practice as a self taught, and emerging freelance translator.

    Thanks for the opportunity to express my view.