Analyzing Error Analysis in Google Translate: A Case Study of a Medical Text
DOI:
https://doi.org/10.31185/eduj.Vol2.Iss43.2235Abstract
Both sparing time and efforts urged a necessity to use machine translation (MT) systems such as Google Translate to translate texts all over the world. During the last decades, GT was the focus of attention for being the most used MT system for its speed and almost accurate product. This study emerged from the fact that English and Arabic translation via MT has many errors that need to be highlighted since both languages belong to different linguistic systems. In order to evaluate the quality of GT system, this study used error analysis with a view to identifying different error types in GT. The error analysis focused on studying the mismatching syntactic, morphological, lexical and semantic components in the source text and the target text. The case study for this research is an English medical text that is chosen to evaluate the accuracy and usefulness of GT in translating the English medical terms and statements. The results revealed that the most frequently occurring errors were semantic errors (30%) and syntactic errors (30%), morphological errors ranked second (25%) while lexical errors ranked last (15%).
Downloads
References
Aqel, K. A. J. D., & Mohammed, F. (2017). A Longitudinal Analysis Study of Writing Errors Made by EFL. British Journal of Education Vol.5, issue 5, pp. 127-145.
Aziz, W., Sousa, S. C. M., & Specia, L. (2012). PET: A tool for post-editing and assessing machine translation. In The Eighth International Conference on Language Resources and Evaluation, Istanbul, Turkey.
Byrne, J. (2006). Technical translation: Usability strategies for translating technical documentation. Dordrecht: Springer.
Castilho S, Moorkens J, Gaspari F, Sennrich R, Sosoni V, Georgakopoulou P, Lohar
P, Way A, Barone AVM, Gialama M (2017). A Comparative Quality Evaluation of PBSMT and NMT using Professional Translators. In: Proceedings of MT Summit XVI. Vol. 108, issue 1, pp. 116–131.
Corder, S.P. (1974). Error Analysis, In Allen, J. L. P. and Corder, S. P. (1974). Techniques in Applied Linguistics. Oxford: Oxford University Press.
Costa A, Ling W, Lu´ıs T, Correia R, Coheur L (2015). A linguistically motivated taxonomy for Machine Translation error analysis. In Machine Translation Jounal. Vol. 29, issue 2, pp:127–161.
Crystal, D. (2003). A Dictionary of Linguistics and Phonetics. 5th ed. Blackwell Publishing Ltd.
Farr´us M, Costa-Juss`a MR, Mari˜no JB, Fonollosa JAR (2010). Linguistic-based Evaluation Criteria to Identify Statistical Machine Translation Errors. In: Proceedings of the 14th Annual Conference of the European Association for Machine Translation (EAMT 2010), Saint-Raphal, France, pp. 167–173.
Federico M., Negri M., Bentivogli L & Turchi, M. (2014). Assessing the Impact of Translation Errors on Machine Translation Quality with Mixed-effects Models. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), Doha, Qatar, pp. 1643–1653.
Jabak, O. (2019). Assessment of Arabic-English translation produced by Google Translate. International Journal of Linguistics, Literature and Translation (IJLLT), Vol. 2, Issue 4, pp. 238-247.
Karami, O. (2014). The Brief View on Google Translate Machine. Paper presented at the meeting of the 2014 Seminar in Artificial Intelligence on Natural Language, German. Reprieved from: https://www.semanticscholar.org/paper/The-brief-view-on-Google-Translate-Machine-Karami/c6f95d543c0b34c4026b9e6cf64decd94b793823
Kirchhoff K, Capurro D, Turner A (2012). Evaluating user preferences in machine translation using conjoint analysis. In: EAMT 2012: Proceedings of the 16th Annual Conference of the European Association for Machine Translation, Italy, pp. 119–126.
Koponen, M. (2010). Assessing machine translation quality with error analysis. In
MikaEL: Electronic Proceedings of the KäTu Symposium on Translation and Interpreting Studies, pp. 1-12. Retrieved from: https://sktl-fi.directo.fi/@Bin/40701/Koponen_MikaEL2010.pdf
Li, H., Graesser, A. C., & Cai, Z. (2014). Comparison of Google translation with human translation. In 27th International Florida Artificial Intelligence Research Society Conference, pp. 190–195. Association for the Advancement of Artificial Intelligence.
Popovic, M. (2018). Error Classification and Analysis for Machine Translation Quality Assessment. J. Moorkens, et al. (eds.), In Translation Quality Assessment, pp. 129-158. Springer International Publishing.
Richards, J. C. & Schmidt, R. (2002). Dictionary of language teaching and applied linguistics. 3rd, ed. London: Longman.
Rustipa, K. (2011). Contrastive Analysis, Error Analysis, Interlanguage and the Implication to Language Teaching. Stikuban University (Unisbank) Semarang. Ragam Jurnal Pengembangan Humanioura, Vol. 11, pp. 16-22. Retrieved from: https://www.semanticscholar.org/paper/Contrastive-Analysis-%2C-Error-Analysis-%2C-and-the-to-Rustipa/255ebdf90f8353aac8347ec8fcbe7e0f66fa92f2
Strevens, P. (1977). Special Purpose Language learning: a perspective. Language Teaching & Linguistics Abstracts Vol. 10 (3), pp. 145-163.
Stymne, S. (2011). Blast: A tool for error analysis of machine translation output. In Proceedings of ACL, pp. 56-61. Portland, Oregon, USA.
Van der wees, M., Bissaza, A. & Monz, C. (2015). Five Shades of Noise: Analyzing Machine Translation Errors in User-Generated Text. In Proceedings of the Workshop on Noisy User-generated Text, pp. 28–37. Retrieved from https://www.researchgate.net/publication/301449172_Five_Shades_of_Noise_Analyzing_Machine_Translation_Errors_in_User-Generated_Text
Widdowson, H.G. (1974). Literary and Scientific Uses of English. In English Language Teaching Journal. Vol. 3 (28), pp. 282-292.
t is shown that lexical and semantic errors have
most impact on sentence level ranking. Furthermore, highly ranked sentences clearly
exhibit low number of grammatical errors, however the relation between grammat-
ical errors and poorly ranked segments remained unclear. Apart from this, high
inter-annotator agreement between two annotators is reported, which contradicts
the results from former studies. The most probable factor is removing words with
position disagreement from calculation which increased the agreement between the
error types
is shown that lexical and semantic errors have
most impact on sentence level ranking. Furthermore, highly ranked sentences clearly
exhibit low number of grammatical errors, however the relation between grammat-
ical errors and poorly ranked segments remained unclear. Apart from this, high
inter-annotator agreement between two annotators is reported, which contradicts
the results from former studies. The most probable factor is removing words with
position disagreement from calculation which increased the agreement between the
errrgest correlation is observed for lexical errors and missing words.
Additional very interesting finding is that the human perception of quality does not
necessarily depend on frequency of the given error type – a sentence with a low
overall score can easily contain less missing words and/or lexical errors than an-
other sentence with
Downloads
Published
Issue
Section
License
Copyright (c) 2021 Farah Abbas Abo Al-Timen
This work is licensed under a Creative Commons Attribution 4.0 International License.
How to Cite
