Round-trip translation

To compare the quality of different machine translation systems, users perform RTT and compare the resulting text to the original. The theory is that the closer the result of the RTT is to the original text, the higher the quality of the machine translation system. One of the problems with this technique is that if there is a problem with the resulting text it is impossible to know whether the error occurred in the forward translation, in the back translation, or in both. In addition, it is possible to get a good back translation from a bad forward translation.[4] A study using the automatic evaluation methods BLEU and F-score compared five different free online translation programs, evaluating the quality of both the forward translation and the back translation, and found no correlation between the quality of the forward translation and the quality of the back translation (i.e., a high quality forward translation did not always correspond to a high quality back translation). The author concluded that RTT was a poor method of predicting the quality of machine translation software.[4] This conclusion was reinforced by a more in-depth study also using automatic evaluation methods.[1] A subsequent study which included human evaluation of the back translation in addition to automatic evaluation methods found that RTT might have some ability to predict the quality of a machine translation system[5] not on a sentence-by-sentence basis but for larger texts.

It is also suggested that RTT can be used to determine whether a text is suitable for machine translation. The idea being that if RTT results in a text that is close to the original, the text is suitable for MT. If after using RTT, the resulting text is inaccurate, the source text can then be edited until a satisfactory result is achieved.[6] One of the studies looking at RTT as a means of measuring MT system quality also looked at its ability to predict whether a text was suitable for machine translation. It found that using different types of text also did not result in any correlation between the quality of the forward translation and the quality of the back translation.[4] In contrast another study using human evaluation found that there was a correlation between the quality of the forward translation and the back translation and that this correlation could be used to estimate the quality of the forward translation. This correlation could be used to estimate the quality of the forward translation and by simplifying the source text, improve the quality of the forward translation.[3]

Although the use of RTT for assessing MT system quality or the suitability of a text for MT is in doubt, it is a way to have fun with machine translation.[2] The text produced from an RTT can be comically bad. At one time websites existed for the sole purpose of performing RTT for fun.[2][7] Other variations send the text through several languages before translating it back into the original[8] or continue translating the text back and forth until it reaches equilibrium (i.e., the result of the back translation is identical to the text used for the forward translation).[9] RTT as entertainment appeared in Philip K. Dick's novel Galactic Pot-Healer. The main character runs book titles and sayings through RTT then has his friends try to guess the original. The Australian television show Spicks and Specks had a contest called "Turning Japanese" which used RTT on song lyrics. Contestants needed to correctly guess the title of the song from which the lyrics were taken.

• Evaluation of machine translation
• Paraphrasing (computational linguistics)
• Translation-quality standards
• Humour in translation
• Language Grid

• Gaspari, F. (2006) "Look Who's Translating. Impersonations, Chinese Whispers and Fun with Machine Translation on the Internet" in Proceedings of the 11th Annual Conference of the European Association of Machine Translation