Analyzing the Performance of Gemini, ChatGPT, and Google Translate in Rendering English Idioms into Arabic

Abstract

This study examines the translation of 155 idioms by different machine and AI translation

systems, namely Google Translate, ChatGPT, and Gemini. Various sources were utilized to

collect the data, including books, magazines, interviews with native English speakers, and

various websites dedicated to English idioms. This data was analyzed based on a framework

built on the taxonomy of Baker (1992). The quantitative part examined the frequency of

translation approaches each program used to render the idioms. The qualitative part focused on

selected examples to highlight the potential issues of each approach in conveying the style and

sense-based features of the idioms. The findings showed that idiom translations were done

through three main approaches: literal, sense-based, and idiom-to-idiom translation. Google

Translate had the highest percentage of literal translation at 76%, followed by ChatGPT at 53%,

while Gemini had the lowest percentage at 21%. For sense-based translations that use

nonfigurative language, Gemini was in the lead at 63%, followed by ChatGPT, with a wide gap

at 35%. Google Translate had the least sense-based renditions at a mere 11%. When it came to

translating idioms using figurative language, Gemini once again was in the lead with 16%,

followed closely by ChatGPT at 13%, with Google Translate right behind at 12%. The study

concludes that although there is vast improvement and advancement in technology, machine

translation has yet to master nonliteral language such as idioms.