Comparing lecturer and AI-based assessment in EFL academic writing: Hybrid framework implications
DOI:
https://doi.org/10.22219/englie.v7i1.42798Keywords:
AI-based assessment, EFL academic writing, Hybrid assessment model, Indonesian contextAbstract
Effective assessment of EFL academic writing in Indonesian universities is still difficult because lecturers have heavy workloads and provide inconsistent feedback. While AI tools like Grammarly, ChatGPT, and Gemini promise to improve efficiency, most research focuses on single platforms or Western contexts. This leaves a significant gap in understanding how different AI systems compare with human assessment across various writing aspects in Indonesia's specific EFL environment. This mixed methods study addresses this gap by comparing lecturer assessments with three AI platforms in five writing areas: grammar, coherence, organization, vocabulary, and mechanics. It also explores stakeholder perceptions. A quantitative analysis of 30 students' essays showed that AI consistently gave higher scores in technical aspects, such as grammar and mechanics (p<0.05), but lower scores in holistic dimensions like coherence and organization. There were strong correlations in grammar (r=0.85) and weak correlations in coherence (r=0.38). Qualitative findings revealed that 70.0% of participants felt lecturer assessments were fairer because of their understanding of cultural context. Although AI showed efficiency, it lacked sensitivity to Indonesian rhetorical norms. The study suggests a culturally responsive hybrid assessment model where AI handles initial technical screening, and lecturers focus on contextual evaluation. This approach balances AI's efficiency, which could reduce workloads by 60%, with human expertise in culturally relevant feedback, providing a practical framework for Indonesian EFL institutions undergoing digital transformation while maintaining educational integrity.
Downloads
References
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests (Vol. 1). Oxford University Press.
Boud, D., & Soler, R. (2016). Sustainable assessment revisited. Assessment & Evaluation in Higher Education, 41(3), 400–413. https://doi.org/10.1080/02602938.2015.1018133
Carless, D., & Winstone, N. (2023). Teacher feedback literacy and its interplay with student feedback literacy. Teaching in Higher Education, 28(1), 150–163. https://doi.org/10.1080/13562517.2020.1782372
Cohen, J. (1968). Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213.
Creswell, J. W., & Clark, V. L. P. (2017). Designing and conducting mixed methods research. Sage publications.
Dellermann, D., Ebel, P., Söllner, M., & Leimeister, J. M. (2019). Hybrid Intelligence. Business & Information Systems Engineering, 61(5), 637–643. https://doi.org/10.1007/s12599-019-00595-2
Dörnyei, Z., & Griffee, D. T. (2010). Research Methods in Applied Linguistics. TESOL Journal, 1(1), 181–183. https://doi.org/10.5054/tj.2010.215611
Hamukti, W., Andrawina, L., & Suwarsono, L. W. (2017). Analisis beban kerja dosen bidang pendidikan dan penunjang menggunakan metode knowledge conversion 5c-4c. JISI: Jurnal Integrasi Sistem Industri, 4(2), 73–84.
Harintama, F., & Muslimin, A. I. (2024). Enhancing EFL Teaching in Indonesian Islamic senior high schools through Artificial Intelligence integration. Schemata: Jurnal Pascasarjana UIN Mataram, 13(2), 111–122.
Hyland, K. (2015). Teaching and researching writing. Routledge. https://www.taylorfrancis.com/books/mono/10.4324/9781315717203/teaching-researching-writing-ken-hyland
Iorliam, A., & Ingio, J. A. (2024). A comparative analysis of generative artificial intelligence tools for natural language processing. Journal of Computing Theories and Applications, 1(3), 311–325.
Khartha, A., Nasir, S. H., Naing, I. R., Bohang, M. B. A., Dakka, L. N., Rini, H. C., Marhamah, M., Hartina, S., Alfian, H., & Kiftiah, S. (2025). Fundamentals of English Language Teaching: A Beginner’s Guide for Educators. PT Akselerasi Karya Mandiri, 264. https://e-publisher.my.id/index.php/ptakm/article/view/124
Koe, L. S., Kustandi, C., & Siregar, E. (2024). AI-driven feedback system: Implementing advanced NLP and openAI for online learning. Jurnal Inovasi Dan Teknologi Pembelajaran, 11(3), 137–148. https://doi.org/10.17977/um031v11i32024p137
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 159–174.
Mi’andri, M., Siregar, A. C., & Utami, P. Y. (2021). Sistem penilaian ujian otomatis untuk soal esai menggunakan metode Vector Space Model. JUTECH: Journal Education and Technology, 2(2), 1–15. https://doi.org/10.31932/jutech.v2i2.1273
Nurchurifiani, E., Maximilian, A., Ajeng, G. D., Wiratno, P., Hastomo, T., & Wicaksono, A. (2025). Leveraging AI-powered tools in academic writing and research: Insights from English faculty members in Indonesia. International Journal of Information and Education Technology, 15(2), 312–322. https://doi.org/10.18178/ijiet.2025.15.2.2244
Prabhakaran, V., Qadri, R., & Hutchinson, B. (2022). Cultural Incongruencies in Artificial Intelligence. Computer Science. https://doi.org/10.48550/arXiv.2211.13069
Pramukantoro, E. S. (2016). Sistem penilaian otomatis jawaban esai pada elearning belajardisini.com. Jurnal Teknologi Informasi Dan Ilmu Komputer (JTIIK), 3(4), 248-252. https://doi.org/10.25126/jtiik.201634187
Rahman, M. A. (2024). Exploring the integration of Artificial Intelligence in English as a foreign language education in Indonesia. Pedagogy: Journal of English Language Teaching, 12(2), 196–212.
Roe, J., Perkins, M., & Furze, L. (2025). From assessment to practice: Implementing the AIAS framework in EFL teaching and learning. arXiv preprint arXiv:2501.00964. https://doi.org/10.48550/arXiv.2501.00964
Ruslan, R., Gunawan, G., & Tjandra, S. (2018). Sistem Penilaian Otomatis Jawaban Esai Menggunakan Metode GLSA. Seminar Nasional Aplikasi Teknologi Informasi (SNATi). https://journal.uii.ac.id/Snati/article/download/11133/8528
Side, S., Putri, S. E., Zubair, S., & Ilyas, N. M. (2024). Pelatihan pemanfaatan artificial intelligence (AI) dalam penulisan artikel ilmiah pada guru SMAN 11 Kabupaten Pangkep. Smart Jurnal Pengabdian Kepada Masyarakat, 4(1), 58.
Singh, A. (2023). A review on objective-driven Artificial Intelligence. Cornell University. https://doi.org/10.48550/arXiv.2308.10135
Syahira, S., Kartini, K., Sulistyahadi, S., & Prafiadi, S. (2023). Persepsi mahasiswa Prodi Pendidikan Bahasa Inggris tentang penggunaan AI dalam pengajaran bahasa Inggris. Jurnal Perspektif Pendidikan, 17(2), 263–269.
Utami, S. P. T., & Winarni, R. (2023). Utilization of Artificial Intelligence technology in an academic writing class: How do Indonesian students perceive? Contemporary Educational Technology, 15(4). https://eric.ed.gov/?id=EJ1406915
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes (Vol. 86). Harvard university press.
Wang, Y., Wu, J., Chen, F., Wang, Z., Li, J., & Wang, L. (2024). Empirical assessment of AI-powered tools for vocabulary acquisition in EFL instruction. IEEE Access. https://ieeexplore.ieee.org/abstract/document/10639964/
Williams, P. (2023). AI, analytics and a new assessment model for universities. Education Sciences, 13(10), 1040.
Wulandari, M., & Purnamaningwulan, R. A. (2024). Exploring Indonesian EFL pre-service teachers’ experiences in ai-assisted teaching practicum: benefits and drawbacks. LLT Journal: A Journal on Language and Language Teaching, 27(2), 878–894.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Aqzhariady Khartha, Uswatun Husanah, Wahdaniatul Mukarrama
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with English Learning Innovation (englie) agree to the following terms:
- For all articles published in English Learning Innovation (englie), copyright is retained by the authors. Authors give permission to the publisher to announce the work with conditions. When the manuscript is accepted for publication, the authors agree to automatic transfer of the publishing right to the publisher.
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-ShareAlike 4.0 International License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
