Comparative Study on the Translation Quality of "Three-Character Classic" by Large Language Models and Multimodal Large Models
Abstract
With the widespread application of large language models (LLM) and multimodal large models (MLLM) in the field of machine translation, how to evaluate the translation quality of different models for ancient Chinese classics has become an urgent issue to be explored. This study takes the children's educational classic "Three Character Classic" as the research object, selects eight representative original sentences, and translates them into English using both LLM and MLLM. The translated texts are then quantitatively compared using the BLEU automatic evaluation tool. The aim of this study is to determine which type of model performs better in the English translation of classical Chinese texts and which model is more suitable for the translation tasks of Chinese classical literature. The study found that, here, fill in the actual experimental data: the BLEU and METEOR scores show which type of model performs better. This study provides empirical reference for AI-assisted English translation of Chinese ancient classics and also offers an empirical basis for the applicability of automatic translation quality assessment tools in the context of classical Chinese translation.
References
- A multi agent classical Chinese translation method based on large language models. (2025).Scientific Reports, 15(1).
- Application of automatic evaluation to human translation. (2025). KCI
- Developing an MTPE quality evaluation model for legislative translation: A case study on Chinese MTPE output of a Korean law. (2025). KCI
- Boyd, W. J., & Mitkov, R. (2025). Machine translation in the AI era: Comparing previous methods of machine translation with large language models. InProceedings of the 8th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Texts(pp. 38–51). INCOMA Ltd.
- Al Amer, S. A., Lee, M. G., & Smith, P. (2025). Comparative evaluation of machine translation models using human-translated social media posts as references: Human-translated datasets. InProceedings of the Eighth Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT 2025)(pp. 1–9). Association for Computational Linguistics.
- Canga Alonso, A., & Napoletano, M. C. (2025). The role of metrics to assess the quality of British teenage language translation into Spanish and Italian using machine translation tools.Cadernos de Tradução, 45(1).
- Lei, T. (2025). Evaluating different translation methods: A case study of Chinese graduate students in an MTI program.Cogent Arts & Humanities, 12(1), 2511386.
- Translating culture-specific items in The true story of Ah Q: A comparative evaluation of GPT-4o, KIMI, and Google Translate under multimodal prompts. (2026).SAGE Open.
- Yu, Q., & Wan, Q. (2025). Quality evaluation of English translation of ancient Chinese classics by large language models - Taking "Mencius · Li Huaiwang Shang" as an example.Chinese Science and Technology Translation, 38(3), 50-53+57.
- Comparative study on the quality of English translation of Confucian classics by large language models. (2025).Journal of Xi'an University of Electronic Technology (Social Science Edition), (3), 125-135.
- Wen, L., & Liu, Y. (2025). Evaluation of translation quality of generated texts by translation tool GPT-4 based on NLP indicators - Taking "The Analects" as an example.Journal of Changchun University, (11).
- Research on the quality evaluation of machine translation of traditional Chinese medicine terminology in the era of artificial intelligence - Taking ChatGPT-4 and Google Translation as examples. (2025).Chinese Journal of Traditional Chinese Medicine, 284-293.