Exploring the quality of AI translation in Roblox chat

Wangsi, Siti Nabila Amira and Fauzi, Erfan Muhamad and Abqoriyyah, Fourus Huznatul (2026) Exploring the quality of AI translation in Roblox chat. IDEAS: Journal of Language Teaching and Learning, Linguistics and Literature, 14 (1). pp. 4067-4084. ISSN 2548-4192

Preview

Text Siti Nabila Amira Wangsi_Exploring The Quality of AI Translation in Roblox Chat.pdf Download (617kB) | Preview

Abstract

This study evaluates the translation quality of Roblox’s AI translation system by combining translation quality assessment (TQA) with syntactic clause analysis in a multilingual online gaming environment. The study aims to examine how the system translates different clause types based on Nababan’s theory, which is assessed by parameters of accuracy, acceptability, and readability. A qualitative approach was employed using 15 representative Indonesian–English translation pairs, selected from an initial dataset of 34 chat conversations collected from real-time Roblox gameplay. The data were classified into declarative, interrogative, and imperative clauses based on the syntactic framework of Huddleston and Pullum before being evaluated using the TQA model. The results show that the Roblox AI translation system performed best on interrogative clauses, while imperative and informal declarative clauses produced lower translation quality due to frequent lexical, syntactic, and semantic errors. The overall translation quality score was 1.98 on a 1–3 scale, indicating moderate-to-low translation quality. These findings demonstrate that combining TQA with syntactic clause analysis provides a more comprehensive approach to gaming communication and highlights the need for translation models that better accommodate informal language, gaming-specific expressions, and context-dependent interactions.

Actions (login required)

View Item