Developing the Cultural Meaning Preservation Assessment (CMPA): A Multidimensional Methodology for Evaluating AI-Assisted Translation of Indigenous Cultural Knowledge Units

Jepri Jepri

Abstract


ABSTRAK

 

Kemajuan Akal Imitasi (AI) telah meningkatkan kualitas terjemahan melalui berbagai sistem terjemahan berbantuan AI yang menghasilkan terjemahan lebih cepat dan lebih akurat secara linguistik dibandingkan sebelumnya (Koehn, 2020; Toral & Way, 2018). Namun, AI masih menghadapi tantangan dalam mempertahankan makna budaya, khususnya pada Indigenous Cultural Knowledge Units (ICKUs) yang merepresentasikan identitas historis, sosial, dan budaya masyarakat adat. Penelitian ini bertujuan mengembangkan Cultural Meaning Preservation Assessment (CMPA), yaitu metode multidimensi untuk mengevaluasi kemampuan sistem AI dalam melestarikan makna budaya pada terjemahan ICKUs dari Kutai Adat Lawas. Penelitian menggunakan desain deskriptif kualitatif dengan korpus sebanyak 25 ICKUs yang diperoleh dari Badan Registrasi Wilayah Adat (BRWA) dan Indonesia Kaya. Sebagai acuan evaluasi, dikembangkan Human Gold Standard Translation (HGST) menggunakan teknik peminjaman dan deskripsi (Newmark, 1988). Selanjutnya, terjemahan yang dihasilkan oleh Google Translate, DeepL, ChatGPT, Gemini, dan Claude dievaluasi menggunakan enam dimensi CMPA, yaitu pelestarian leksikal, semantik, budaya, historis, pragmatik, dan identitas pribumi. Hasil penelitian menunjukkan bahwa meskipun sebagian besar sistem Akal Imitasi (AI) memiliki kinerja yang baik dalam kesetaraan leksikal dan semantik, pelestarian makna historis, pragmatik, dan budaya masih menjadi kelemahan utama. CMPA memberikan kontribusi metodologis sebagai kerangka kerja holistik untuk mengevaluasi pelestarian makna budaya dalam terjemahan berbantuan Akal Imitasi (AI).


Keywords: Artificial Intelligence-Assisted Translation, Cultural Meaning Preservation Assessment (CMPA), Indigenous Cultural Knowledge Units (ICKUs), Translation Quality Assessment, Cultural Meaning Preservation, Kutai Adat Lawas.

 

ABSTRACT

 

Recent advances in Artificial Intelligence (AI) have significantly improved translation quality through the development of AI-assisted translation systems capable of producing faster and more linguistically accurate translations than previous approaches (Koehn, 2020; Toral & Way, 2018). Nevertheless, AI continues to face challenges in preserving cultural meaning, particularly when translating Indigenous Cultural Knowledge Units (ICKUs), which embody the historical, social, and cultural identities of Indigenous communities. This study aims to develop the Cultural Meaning Preservation Assessment (CMPA), a multidimensional assessment framework designed to evaluate the ability of AI-assisted translation systems to preserve the cultural meaning of ICKUs derived from Kutai Adat Lawas. The study employed a qualitative descriptive research design using a corpus of 25 ICKUs collected from the Badan Registrasi Wilayah Adat (BRWA) and Indonesia Kaya. A Human Gold Standard Translation (HGST) was developed as the evaluation benchmark using borrowing and descriptive translation techniques (Newmark, 1988). Translations generated by Google Translate, DeepL, ChatGPT, Gemini, and Claude were subsequently evaluated using the six dimensions of the CMPA framework: lexical, semantic, cultural, historical, pragmatic, and Indigenous identity preservation. The findings indicate that although most AI systems perform well in achieving lexical equivalence and semantic accuracy, preserving historical, pragmatic, and culturally embedded meanings remains a major challenge. The proposed CMPA makes a methodological contribution by providing a holistic framework for evaluating cultural meaning preservation in AI-assisted translation.

Keywords: Artificial Intelligence-Assisted Translation; Cultural Meaning Preservation Assessment (CMPA); Indigenous Cultural Knowledge Units (ICKUs); Translation Quality Assessment; Cultural Meaning Preservation; Kutai Adat Lawas.


Full Text:

PDF

References


REFERENCES

Badan Registrasi Wilayah Adat: https://brwa.or.id/wa/view/WmZnRXljazJtbmc?

Baker, M. (2018). In other words: A coursebook on translation (3rd ed.). Routledge.

Bassnett, S. (2021). Translation studies (5th ed.). Routledge.

Battiste, M. (2013). Decolonizing education: Nourishing the learning spirit. Purich Publishing.

Castilho, S. (2023). Machine translation and human translation in the age of artificial intelligence. Routledge.

Creswell, J. W., & Creswell, J. D. (2023). Research design: Qualitative, quantitative, and mixed methods approaches (6th ed.). SAGE Publications.

Guerreiro, N. M., Freitag, M., Foster, G., Cherry, C., & Martins, A. F. T. (2023). Quality evaluation of large language models in machine translation. arXiv. https://arxiv.org/abs/2308.01825

Hall, S. (1997). Representation: Cultural representations and signifying practices. SAGE Publications.

House, J. (2015). Translation quality assessment: Past and present. Routledge.

https://indonesiakaya.com/pustaka-indonesia/beseprah-nuansa-kebersamaan-dalam-tradisi-sarapan-massal-warga-kutai/

Kenny, D. (2022). Machine translation for everyone: Empowering users in the age of artificial intelligence. Language Science Press.

Koehn, P. (2020). Neural machine translation. Cambridge University Press.

Liu, Y., Zhang, X., Wang, J., & Li, H. (2024). Artificial intelligence-assisted translation and cultural meaning preservation: Challenges and opportunities. Machine Translation. Advance online publication.

Lommel, A. R., Burchardt, A., & Uszkoreit, H. (2014). Multidimensional Quality Metrics (MQM): A framework for declaring and describing translation quality metrics. Tradumàtica, 12, 455–463.

Merriam, S. B., & Tisdell, E. J. (2016). Qualitative research: A guide to design and implementation (4th ed.). Jossey-Bass.

Mertens, D. M. (2020). Research and evaluation in education and psychology: Integrating diversity with quantitative, qualitative, and mixed methods (5th ed.). SAGE Publications.

Miles, M. B., Huberman, A. M., & Saldaña, J. (2020). Qualitative data analysis: A methods sourcebook (4th ed.). SAGE Publications.

Molina, L., & Hurtado Albir, A. (2002). Translation techniques revisited: A dynamic and functionalist approach. Meta, 47(4), 498–512. https://doi.org/10.7202/008033ar

Munday, J. (2016). Introducing Translation Studies (4th ed.). Routledge.

Nakata, M. (2007). Disciplining the savages, savaging the disciplines. Aboriginal Studies Press.

Newmark, P. (1988). A textbook of translation. Prentice Hall.

Nord, C. (2018). Translating as a purposeful activity: Functionalist approaches explained (2nd ed.). Routledge.

OpenAI. (2023). GPT-4 technical report. https://arxiv.org/abs/2303.08774

Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2002). BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (pp. 311–318). Association for Computational Linguistics. https://doi.org/10.3115/1073083.1073135

Patton, M. Q. (2015). Qualitative research & evaluation methods (4th ed.). SAGE Publications.

Popel, M., Tomkova, M., Tomek, J., Kaiser, Ł., Uszkoreit, J., Bojar, O., & Žabokrtský, Z. (2020). Transforming machine translation: A deep learning system reaches news translation quality comparable to human professionals. Nature Communications, 11, Article 4381. https://doi.org/10.1038/s41467-020-18073-9

Rei, R., Stewart, C., Farinha, A. C., & Lavie, A. (2020). COMET: A neural framework for MT evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 2685–2702). Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.emnlp-main.213

TAUS. (2023). Dynamic Quality Framework (DQF). https://www.taus.net

Toral, A., & Way, A. (2018). What level of quality can neural machine translation attain on literary text? In Translation Quality Assessment (pp. 263–287). Springer.

Tracy, S. J. (2020). Qualitative research methods: Collecting evidence, crafting analysis, communicating impact (2nd ed.). Wiley-Blackwell.

UNESCO. (2021). Local and indigenous knowledge systems (LINKS). https://en.unesco.org/links

United Nations. (2007). United Nations Declaration on the Rights of Indigenous Peoples. United Nations. https://www.un.org/development/desa/indigenouspeoples/declaration-on-the-rights-of-indigenous-peoples.html

Venuti, L. (2018). The translator's invisibility: A history of translation (3rd ed.). Routledge.




DOI: http://dx.doi.org/10.30872/jbssb.v10i3.28931

Refbacks

  • There are currently no refbacks.


Copyright (c) 2026 Jepri Jepri

Editorial address:

Fakultas Ilmu Budaya, Universitas Mulawarman
Jl. Ki Hajar Dewantara, Gunung Kelua, Kec. Samarinda Ulu, Kota Samarinda, Kalimantan Timur, Indonesia 75123
Email: jurnalilmubudaya.fibunmul@gmail.com
Website: http://e-journals.unmul.ac.id/index.php/JBSSB 

 

Creative Commons License

Ilmu Budaya: Jurnal Bahasa, Sastra, Seni, dan Budaya is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License