Analisis Sentimen Multikelas Bahasa Jawa: Studi Komparatif dan Ensemble Learning Model IndoBERT dan JavaBERT
DOI:
https://doi.org/10.30872/7swy2g79Keywords:
Analisis Sentimen, Bahasa Jawa, IndoBERT, JavaBERT, Ensemble LearningAbstract
Bahasa Jawa merupakan bahasa daerah dengan jumlah penutur terbesar di Indonesia, namun sumber daya pemrosesan bahasa alami (Natural Language Processing, NLP) untuknya masih sangat terbatas dibandingkan Bahasa Indonesia. Penelitian ini menyajikan studi komparatif untuk tugas analisis sentimen multikelas (Positif, Netral, Negatif) pada teks berbahasa Jawa menggunakan dua model pre-trained berbasis Transformer, yaitu IndoBERT (indobert-base-p1) dan JavaBERT (javanese-bert-small), serta model ensemble soft-voting dari keduanya. Karena korpus mentah (200.000 kalimat) tidak memiliki label, penelitian ini menerapkan pelabelan otomatis berbasis leksikon sentimen Bahasa Jawa yang disusun manual dengan penanganan negasi dan kalimat kontras. Ketiga model di-fine-tune dan dievaluasi pada data uji yang sama (20.000 kalimat), menghasilkan akurasi, presisi, recall, dan F1-score sebesar 1,00 secara konsisten. Hasil yang sangat tinggi ini dibahas secara kritis sebagai konsekuensi langsung dari kuatnya korelasi leksikal antara skema pelabelan otomatis dan fitur permukaan teks, bukan sebagai bukti pemahaman semantik mendalam. Untuk menguji generalisasi di luar distribusi latih, dilakukan pengujian kualitatif tambahan pada kalimat berdialek Ngapak (Banyumasan) yang tidak terwakili dalam leksikon maupun data latih. Pada kalimat berdialek standar, ketiga model konsisten sepakat; namun pada kalimat berdialek Ngapak, IndoBERT dan JavaBERT saling berbeda pendapat dan model ensemble nyaris seri (50,0% berbanding 50,0%), mengindikasikan keterbatasan generalisasi lintas-dialek. Temuan ini menegaskan perlunya benchmark berlabel manusia, cakupan dialek yang lebih luas, dan evaluasi di luar distribusi sebagai arah penelitian selanjutnya.References
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proc. 2019 Conf. North American Chapter Assoc. Comput. Linguistics: Human Language Technologies (NAACL-HLT), Minneapolis, MN, USA, 2019, pp. 4171-4186.
B. Wilie et al., “IndoNLU: Benchmark and resources for evaluating Indonesian natural language understanding,” in Proc. 1st Conf. Asia-Pacific Chapter Assoc. Comput. Linguistics and 10th Int. Joint Conf. Natural Language Processing (AACL-IJCNLP), Suzhou, China, 2020, pp. 843-857.
W. Wongso, D. S. Setiawan, and D. Suhartono, “Causal and masked language modeling of Javanese language using transformer-based architectures,” in Proc. 2021 Int. Conf. Advanced Computer Science and Information Systems (ICACSIS), Depok, Indonesia, 2021, pp. 1-7.
G. I. Winata et al., “NusaX: Multilingual parallel sentiment dataset for 10 Indonesian local languages,” in Proc. 17th Conf. European Chapter Assoc. Comput. Linguistics (EACL), Dubrovnik, Croatia, 2023, pp. 815-834.
W. Medhat, A. Hassan, and H. Korashy, “Sentiment analysis algorithms and applications: A survey,” Ain Shams Eng. J., vol. 5, no. 4, pp. 1093-1113, 2014.
T. G. Dietterich, “Ensemble methods in machine learning,” in Proc. 1st Int. Workshop on Multiple Classifier Systems (MCS), Cagliari, Italy, 2000, pp. 1-15.
Kaggle, “Javanese Sentiment Dataset.” [Online]. Tersedia: https://www.kaggle.com/datasets/nunoliebert/javanese-sentiment-dataset. [Diakses: 5 Agustus 2026].
B. Pang and L. Lee, “Opinion mining and sentiment analysis,” Found. Trends Inf. Retr., vol. 2, no. 1-2, pp. 1-135, 2008.
M. Taboada, J. Brooke, M. Tofiloski, K. Voll, and M. Stede, “Lexicon-based methods for sentiment analysis,” Comput. Linguist., vol. 37, no. 2, pp. 267-307, 2011.
R. Geirhos et al., “Shortcut learning in deep neural networks,” Nat. Mach. Intell., vol. 2, no. 11, pp. 665-673, 2020.
S. Gururangan et al., “Annotation artifacts in natural language inference data,” in Proc. 2018 Conf. North American Chapter Assoc. Comput. Linguistics: Human Language Technologies (NAACL-HLT), Vol. 2 (Short Papers), New Orleans, LA, USA, 2018, pp. 107-112.
R. T. McCoy, E. Pavlick, and T. Linzen, “Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference,” in Proc. 57th Annu. Meeting Assoc. Comput. Linguistics (ACL), Florence, Italy, 2019, pp. 3428-3448.
J. R. Landis and G. G. Koch, “The measurement of observer agreement for categorical data,” Biometrics, vol. 33, no. 1, pp. 159-174, 1977.
B. Settles, “Active learning literature survey,” Computer Sciences Technical Report 1648, University of Wisconsin-Madison, 2009.
G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015.
F. Koto et al., “IndoLEM and IndoBERT: A benchmark dataset and pre-trained language model for Indonesian NLP,” in Proc. 28th Int. Conf. Comput. Linguistics (COLING), Barcelona, Spain, 2020, pp. 757-770.
A. F. Aji et al., “One country, 700+ languages: NLP challenges for underrepresented languages and dialects in Indonesia,” in Proc. 60th Annu. Meeting Assoc. Comput. Linguistics (ACL), Dublin, Ireland, 2022, pp. 7226-7249.
A. Vaswani et al., “Attention is all you need,” in Proc. 31st Conf. Neural Information Processing Systems (NeurIPS), Long Beach, CA, USA, 2017, pp. 5998-6008.
C. Sun, X. Qiu, Y. Xu, and X. Huang, “How to fine-tune BERT for text classification?” in Proc. 18th China National Conf. Chinese Computational Linguistics (CCL), Kunming, China, 2019, pp. 194-206.
V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter,” arXiv preprint arXiv:1910.01108, 2019.
S. L. Blodgett, L. Green, and B. O'Connor, “Demographic dialectal variation in social media: A case study of African-American English,” in Proc. 2016 Conf. Empirical Methods in Natural Language Processing (EMNLP), Austin, TX, USA, 2016, pp. 1119-1130.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Jurnal Rekayasa Teknologi Informasi (JURTI)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
