Cognitive Diagnostic Assessment Instrument with Individual Feedback for Language Receptive Learning on Multimodal Text
DOI:
https://doi.org/10.22219/kembara.v10i1.32974Keywords:
cognitive diagnostic assessment, individual feedback, language receptive learning, multimodal textAbstract
Cognitive diagnostic assessment with individual feedback plays an important role in detecting students' cognitive strengths and weaknesses, providing directions for steps that teachers and students must take to follow up on this, as well as facilitating the clustering of students in preparing differentiated learning. This study aims to develop a cognitive diagnostic assessment instrument for receptive language learning on multimodal text with individualized feedback. This study is R&D research with an ADDIE design. The study involved grade 9 students with varying cognitive levels working through the questions, and the results were analyzed to develop individualized feedback for each student. Individualized feedback was then followed up by teachers and students in differentiated learning. We conducted interviews, questionnaires, and observations to gather data on the feasibility of the assessment instrument. Based on the results of the analysis, it can be concluded that a set of cognitive diagnostic assessment instruments along with individualized feedback produced in the study meets the requirements of validity, reliability, and readability, as well as applicability. In addition, this study shows that the cognitive diagnostic assessment with individualized feedback provides an overview of the receptive cognitive subcompetence of language in multimodal text that is mastered so that it facilitates the teacher in developing content, process, and target differentiated learning products for the next material. This research also contributes to addressing the long-standing criticism that language assessments fail to provide individualized feedback to link assessment to learning.
Downloads
References
Agnafia, D. N. (2019). Analisis Kemampuan Berpikir Kritis Siswa dalam Pembelajaran Biologi. Florea: Jurnal Biologi dan Pembelajarannya, 6(1), 45–53. http://doi.org/10.25273/florea.v6i1.4369
Al Fajri, T. A. (2020). Pentingnya Penggunaan Pendekatan Multimodal dalam Pembelajaran. Waskita: Jurnal Pendidikan Nilai dan Pembangunan Karakter, 2(1), 57–72. https://doi.org/10.21776/ub.waskita.2018.002.01.5
Andrich, D., & Marais, I. (2019). A Course in Rasch Measurement Theory. In Measuring in the educational, social and health sciences (Vol. 41). Springer. https://link.springer.com/book/10.1007/978-981-13-7496-8
Anisah, G. (2018). Validitas Instrumen Asesmen Menulis Cerpen Terintegrasi Pendidikan Karakter. AL-Ulya: Jurnal Pendidikan Islam, 3(1), 15–25. https://doi.org/10.32665/ulya.v3i1.695
Anisah, G., & Amreta, M. Y. (2023). Pengembangan Instrument Assessment as Learning Berbasis Projek untuk Pembelajaran Menyimak dan Berbicara Monologis Dialogis Bermuatan Karakter Bhineka Tunggal Ika. KEMBARA: Jurnal Keilmuan Bahasa, Sastra, dan Pengajarannya, 9(1), 76–88. https://doi.org/10.22219/kembara.v9i1.22701
Arijanty, A. (2014). Model Rasch sebagai Kerangka Acuan Penyusunan Alat Ukur. Jurnal Pendidikan dan Kebudayaan, 20(1), 109–123. https://doi.org/10.24832/jpnk.v20i1.130
Arikunto, S. (2019). Prosedur Penelitian. Rineka Cipta.
Aryadoust, V., Ng, L. Y., & Sayama, H. (2021). A Comprehensive Review of Rasch Measurement in Language Assessment: Recommendations and Guidelines for Research. Language Testing, 38(1), 6–40. https://doi.org/10.1177/0265532220927487
Astalini, A., Darmaji, D., Kurniawan, W., Anwar, K., & Kurniawan, D. (2019). Effectivenes of Using e-Module and e-Assessment. International Journal of Interactive Mobile Technologies, 13(9), 21–39. https://doi.org/10.3991/ijim.v13i09.11016
Astuti, A. P., Aziz, A., Sumarti, S. S., & Bharati, D. A. L. (2019). Preparing 21st Century Teachers: Implementation of 4C Character’s Pre-Service Teacher Through Teaching Practice. Journal of Physics: Conference Series, 1233(1), 12109. https://doi.org/10.17509/ijal.v9i2.20218
Azizah, A., & Wahyuningsih, S. (2020). Penggunaan Model RASCH untuk Analisis Instrumen Tes pada Mata Kuliah Matematika Aktuaria. Jurnal Pendidikan Matematika (JUPITEK), 3(1), 45–50. https://doi.org/10.30598/jupitekvol3iss1pp45-50
Beauty, M., Rahmawati, L. E., Markhamah, M., & Tawandorloh, K.-A. (2023). Evaluasi CIPP Pembelajaran Keterampilan Membaca di Program Studi Pendidikan Bahasa dan Sastra Indonesia: Analisis proses dan produk. KEMBARA: Jurnal Keilmuan Bahasa, Sastra, dan Pengajarannya, 9(1), 1–14. https://doi.org/10.22219/kembara.v9i1.23516
Budyartati, S. (2016). Tes Kognitif Diagnostik untuk Mendeteksi Kesulitan Belajar Siswa Sekolah Dasar. Premiere Educandum: Jurnal Pendidikan Dasar dan Pembelajaran, 1(1). https://doi.org/10.33830/penaanda.v1i2.6202
Carless, D., & Boud, D. (2018). The Development of Student Feedback Literacy: Enabling Uptake of Feedback. Assessment & Evaluation in Higher Education, 43(8), 1315–1325. https://doi.org/10.1080/02602938.2018.1463354
Chan, S.-W., Looi, C.-K., & Sumintono, B. (2021). Assessing Computational Thinking Abilities Among Singapore Secondary Students: A Rasch Model Measurement Analysis. Journal of Computers in Education, 8(1), 213–236. https://doi.org/10.1007/s40692-020-00177-2
Diakidoy, I.-A. N., Stylianou, P., Karefillidou, C., & Papageorgiou, P. (2019). The Relationship Between Listening and Reading Comprehension of Different Types of Text at Increasing Grade Levels. Reading Psychology, 26(1), 55–80. https://doi.org/10.1080/02702710590910584
Dressman, M. (2019). Multimodality and Language Learning. In The Handbook of Informal Language learning. Wiley Online Library. https://doi.org/10.1002/9781119472384.ch3
Ermiyanto, E., Asroa, I., & Ilyas, A. (2023). Asesmen Diagnostik Gaya Belajar Siswa Kelas VII di SMPN 4 Padang Panjang. MANAZHIM, 5(1), 166–177. https://doi.org/10.36088/manazhim.v5i1.2845
Gravemeijer, K., Stephan, M., Julie, C., Lin, F., &, & Ohtani, M. (2017). What Mathematics Education May Prepare Students for the Society of the Future?. International Journal of Science and Mathematics Education, 15(1), 105–123. https://doi.org/10.1007/s10763-017-9814-6
Habsy, B. A., Azizah, N. H., Viola, N. P., & Mahendra, W. (2024). Peran Guru dalam Menciptakan Lingkungan Pembelajaran yang Efektif dengan Cara Pengelolaan Kelas yang Menarik. TSAQOFAH, 4(2), 545–566. https://doi.org/10.58578/tsaqofah.v4i2.2305
Hasna, S., Azizah, M., & others. (2023). Implementasi Asesmen Diagnostik Non Kognitif Siswa Kelas 3 SD Negeri Gayamsari 02 Kota Semarang. Didaktik: Jurnal Ilmiah PGSD STKIP Subang, 9(2), 6037–6049. https://doi.org/10.36989/didaktik.v9i2.1390
Hattie, J., & Clarke, S. (2018). Visible Learning: Feedback. Routledge.
Hendaryan, & Noviadi, A. (2023). The Role of Metacognition Strategies (Metacomprehension) and Inferential Ability in Improving Reading Comprehension Ability. KEMBARA: Jurnal Keilmuan Bahasa, Sastra, dan Pengajarannya, 9(2), 363–375. https://doi.org/10.22219/kembara.v9i2.26097
Hesse, F., Care, E., Buder, J., Sassenberg, K., & G., & P. (2015). A Framework for Teachable Collaborative Problem Solving Skills. In Assessment and teaching of 21st century skills (pp. 37–56). https://doi.org/10.1007/978-94-017-9395-7_2
Hilman, I., Akmal, R., & Nugraha, F. (2023). Analisis Gaya Belajar Peserta Didik Melalui Assessment Diagnostik Non Kognitif pada Pembelajaran Diferensiasi di Sekolah Dasar. Naturalistic: Jurnal Kajian dan Penelitian Pendidikan dan Pembelajaran, 8(1), 161–167. https://doi.org/10.35568/naturalistic.v8i1.3911
Himawan, R., & Nurgiyantoro, B. (2022). Analisis Butir soal Latihan Penilaian Akhir Semester Ganjil Mata Pelajaran Bahasa Indonesia Kelas VIII SMPN 1 Bambanglipuro Bantul Menggunakan Program ITEMAN. KEMBARA: Jurnal Keilmuan Bahasa, Sastra, dan Pengajarannya, 8(1), 160–180. https://doi.org/10.22219/kembara.v8i1.20530
Huda, A. A. S., Alamsyah, A., Selvia, S., & Sangadah, N. (2023). Asesmen Diagnostik Kognitif pada Mata Pelajaran PAI Kelas 7 di SMPN 3 Lembang. Al’Ulum Jurnal Pendidikan Islam, 3(2), 213–224. https://doi.org/10.54090/alulum.298
Hui, L., de Bruin, A. B. H., Donkers, J., & van Merriënboer, J. J. G. (2021). Does Individual Performance Feedback Increase the use of Retrieval Practice? Educational Psychology Review, 33(1), 1–23. https://doi.org/10.1007/s10648-021-09604-x
Huilin, C. L. (2013). Cognitive Diagnostic Approach — A New Stage of Language Testing Research. https://api.semanticscholar.org/CorpusID:63374199
Imamyartha, D., Fitriyah, S. M., Tasnim, Z., Puspa, A., Fardhani, A. E., Wahjuningsih, E., Sundari, S., Hudori, R. F., & Arya, B. (2019). The Efficacy of 4Cs-based Reading to Foster 21st-Century Learning Competencies. Indonesian Journal of Applied Linguistics, 9(2), 275–285. https://doi.org/doi: 10.17509/ijal.v9i2.20218
Irons, A., & Elkington, S. (2021). Enhancing Learning Through Formative Assessment and Feedback. Routledge.
Jain, S., & Wallace, B. C. (2019). Attention is Not Explanation. ArXiv. https://doi.org/https://doi.org/10.48550/arXiv.1908.04626
Jang, E. E., Dunlop, M., Park, G., & Van Der Boom, E. H. (2015). How do Young Students with Different Profiles of Reading Skill Mastery, Perceived Ability, and Goal Orientation Respond to Holistic Diagnostic Feedback? Language Testing, 32(3), 359–383. https://doi.org/10.1177/0265532215570924
Jang, E. E., & Sinclair, J. (2021). Diagnostic Assessment in Language Classrooms. The Routledge Handbook of Language Testing, 187–205. https://doi.org/10.4324/9781003220756-15
Johnson, E. (2021). Face Validity. In Encyclopedia of Autism Spectrum Disorders. Springer. https://doi.org/10.1007/978-3-319-91280-6_308
Kamaruddin, A., Patmasari, A., Whanchit, W., & Suriaman, A. (2023). The Effect of IBL (Inquiry-Based Learning) Model on EFL Students’ Critical Thinking Skills. KEMBARA: Jurnal Keilmuan Bahasa, Sastra, dan Pengajarannya, 9(1), 15-23.. https://doi.org/10.22219/kembara.v9i1.22766
Karlimah, K. (2022). How Does RASCH Modeling Reveal Difficulty and Suitability Level the Fraction Test Question? Jurnal Elemen, 8(1), 66–76. https://doi.org/10.29408/jel.v8i1.4170
Kementerian Pendidikan, Kebudayaan, R. dan T. (2022). Capaian Pembelajaran Kurikulum Merdeka 2022. Kementerian Pendidikan, Kebudayaan, Riset Dan Teknologi. https://www.kemdikbud.go.id/main/blog/2021/12/kemendikbudristek-dorong-sekolah-memahami-opsi-kurikulum-prototipe-untuk-pulihkan-pembelajaran
Kim, A.-Y. (2015). Exploring Ways to Provide Diagnostic Feedback with an ESL Placement Test: Cognitive Diagnostic Assessment of l2 Reading Ability. Language Testing, 32(2), 227–258. https://doi.org/10.1177/0265532214558457
Kokotsaki, D., Menzies, V., & Wiggins, A. (2016). Project-Based Learning: A Review of the Literature. Improving Schools, 19(3), 267–277. https://doi.org/10.1177/1365480216659733
Kramer, A.-W., Schaaf, J. V, & Huizenga, H. M. (2023). How Much do You Want to Learn? High-School Students’ Willingness to Invest Effort in Valenced Feedback-Learning Tasks. Learning and Individual Differences, 108, 102375. https://doi.org/10.1016/j.lindif.2023.102375
Larenas, C. D., Díaz, A. J., Orellana, Y. R., & Villalón, M. J. S. (2021). Exploring the Principles of English Assessment Instruments. Ensaio, 29(111), 23-37. https://doi.org/10.1590/S0104-403620210002902851
Laudel, H., & Narciss, S. (2023). The Effects of Internal Feedback and Self-Compassion on The Perception of Negative Feedback and Post-Feedback Learning Behavior. Studies in Educational Evaluation, 77(1), 101237. https://doi.org/10.1016/j.stueduc.2023.101237
Leighton, J., & Gierl, M. (2007). Cognitive Diagnostic Assessment for Education: Theory and Applications. Cambridge University Press.
Magdalena, I., Fauziah, S. N., Faziah, S. N., & Nupus, F. S. (2021). Analisis Validitas, Reliabilitas, Tingkat Kesulitan dan Daya Beda Butir Soal Ujian Akhir Semester Tema 7 Kelas III SDN Karet 1 Sepatan. Bintang, 3(3), 198–214. http://dx.doi.org/10.24042/ajpm.v6i2.50
Mauliya, A. (2019). Perkembangan Kognitif pada Peserta Didik SMP Menurut Jean Piaget. ScienceEdu, 2(2), 86–91. https://doi.org/10.19184/se.v2i2.15059
Miles, Matthew B., Huberman, A. Michael. dan Saldaña, J. (2014). Qualitative Data Analysis: A Methods Sourcebook (3rd ed.). SAGE.
Min, S., & He, L. (2022). Developing Individualized Feedback for Listening Assessment: Combining Standard Setting and Cognitive Diagnostic Assessment Approaches. Language Testing, 39(1), 90–116. https://doi.org/10.1177/0265532221995475
Misbah, S. (2022). Penerapan Metode Umpan Balik (Feed Back Partner) untuk Meningkatkan Prestasi Belajar Siswa pada Mata Pelajaran Bahasa Indonesia Materi Struktur dan Kebahasaan Teks Anekdot Kelas X IPS-2 SMAN 4 Kota Bima Semester 1 Tahun Pelajaran 2020/2021. Jurnal Pendidikan dan Pembelajaran Indonesia (JPPI), 2(2), 143–154. https://doi.org/10.53299/jppi.v2i2.219
Molenda, M. (2015). In Search of the Elusive ADDIE Model. Performance Improvement, 54(2), 40–43. https://www.researchgate.net/profile/Michael-Molenda/publication/251405713_In_search_of_the_elusive_ADDIE_model/links/5b5f098ba6fdccf0b200e5b2/In-search-of-the-elusive-ADDIE-model.pdf
Müller, M. (2020). Item Fit Statistics for Rasch Analysis: Can we Trust Them? Journal of Statistical Distributions and Applications, 7(1), 1–12. https://doi.org/10.1186/s40488-020-00108-7
Musaad, F., & Suparman, S. (2023). Pengembangan e-Modul Berbasis Problem Based Learning untuk Memacu Kemampuan Berfikir Kritis Abad-21. AKSIOMA: Jurnal Program Studi Pendidikan Matematika, 12(3), 3162–3171. http://dx.doi.org/10.24127/ajpm.v12i3.6119
Muslihin, H. Y., Suryana, D., Suherman, U., Dahlan, T. H., & others. (2022). Analysis of the Reliability and Validity of the Self-Determination Questionnaire Using RASCH model. International Journal of Instruction, 15(2), 207–222.
Parmaningsih, T. J., & Saputro, D. R. S. (2021). RASCH Analysis on Item Response Theory: Review of Model Suitability. AIP Conference Proceedings, 2326(1), 13-27. https://doi.org/10.1063/5.0040305
Perea, M. (2013). Why Does the APA Recommend the Use of Serif Fonts? Psicothema, 25(1), 13–17. https://doi.org/10.7334/psicothema2012.141
Pérez-González, L. (2019). Multimodality. In Routledge Encyclopedia of Translation Studies (pp. 346–351). Routledge.
Pratama, D. (2020). Analisis Kualitas Tes Buatan Guru melalui Pendekatan Item Response Theory (IRT) Model RASCH. Tarbawy: Jurnal Pendidikan Islam, 7(1), 61–70. https://doi.org/10.21776/ub.waskita.2018.002.01.5
Prihatni, Y., Kumaidi, K., & Mundilarto, M. (2016). Pengembangan Instrumen Diagnostik Kognitif pada Mata Pelajaran IPA di SMP. Jurnal Penelitian dan Evaluasi Pendidikan, 20(1), 111–125. https://doi.org/10.21831/pep.v20i1.7524
Putri, A. O. (2021). Pengembangan Tes Diagnostik Berbentuk Uraian Berdasarkan Ranah Kognitif untuk Mengetahui Letak Kesulitan Belajar Siswa SMP pada Materi Bangun Ruang. Jurnal Pusaka, 10(1), 40–54. https://doi.org/10.35897/ps.v10i1.597
Redlo, J. M. (2021). Barriers in Teaching the Four C’s of 21st Century Competencies: Dismantling the Obstacles. In Handbook of research on barriers for teaching 21st-Century competencies and the impact of digitalization (pp. 1–14). IGI Global. https://doi.org/10.4018/978-1-7998-6967-2.ch001
Safi’i, I., Tarmini, W., & Wahdini, L. (2021). Critical Thinking in Evaluation Instruments at BSE Indonesian Language. KEMBARA: Jurnal Keilmuan Bahasa, Sastra, dan Pengajarannya, 7(2), 232–240. https://doi.org/10.22219/kembara.v7i2.17300
Saleh Alharbi, A., Abdullah Alhebshi, A., & Meccawy, Z. (2021). EFL Students’ and Teachers’ Perceptions of Google forms as a Digital Formative Assessment Tool in Saudi Secondary Schools. Arab World English Journal (AWEJ) Special Issue on CALL, 7(2), 140–154. https://dx.doi.org/10.24093/awej/call7.10
Sawaki, Y., & Koizumi, R. (2017). Providing Test Performance Feedback that Bridges Assessment and Instruction: The Case of Two Standardized English Language Tests in Japan. Language Assessment Quarterly, 14(3), 234–256. https://doi.org/10.1080/15434303.2017.1348504
Shrotryia, V. K., & Dhanda, U. (2019). Content Validity of Assessment Instrument for Employee Engagement. SAGE Open, 9(1), 234–245. https://doi.org/10.1177/2158244018821751
Stemler, S. E., & Naples, A. (2021). Rasch Measurement V. Item Response Theory: Knowing When to Cross The Line. Practical Assessment, Research & Evaluation, 26, 11.
Suarni. (2023). Implementasi Asesmen Diagnostik dalam Kurikulum Merdeka pada Mata Pelajaran Bahasa Inggris Kelas XI/Fase F di Madrasah Aliyah Negeri Kota Sorong. JKIP, 1(4), 263–270. https://doi.org/10.61116/jkip.v1i4.188
Sumintono, B., & Widhiarso, W. (2015). RASCH Modeling Application in Educational Assessment. Trimkom Publising Home.
Sumintono, B., & Widhiarso, W. (2015). Aplikasi Pemodelan RASCH pada Assessment Pendidikan. Trim Komunikata.
Sun, H., & Hwang, Y. (2023). Enhancing EFL learners’ English Reading Performance Through the Diagnostic Feedback of CDA. Korean Journal of English Language and Linguistics, 23(July), 535–553. https://doi.org/10.15738/kjell.23..202307.535
Supriyadi, S., Lia, R. M., Rusilowati, A., Isnaeni, W., Susilaningsih, E., & Suraji, S. (2022). Penyusunan Instrumen Asesmen Diagnostik untuk Persiapan Kurikulum Merdeka. Journal of Community Empowerment, 2(2), 67–73. https://doi.org/10.15294/jce.v2i2.61886
Susanti, E. (2019). Keterampilan Menyimak. Rajawali Pers.
Tang, F., & Zhan, P. (2021). Does Diagnostic Feedback Promote Learning? Evidence from a Longitudinal Cognitive Diagnostic Assessment. AERA Open, 7. https://doi.org/10.1177/23328584211060804
Taylor, C., & Noë, A. (2021). The Explanation of Behaviour. Routledge.
Toprak, T. E., & Cakir, A. (2021). Examining The L2 Reading Comprehension Ability of Adult ELLs: Developing a Diagnostic Test within the Cognitive Diagnostic Assessment framework. Language Testing, 38(1), 106–131. https://doi.org/10.1177/0265532220941470
Van Leeuwen, T. (2015). Multimodality. The Handbook of Discourse Analysis, 447–465. https://doi.org/10.1002/9781118584194.ch21
Wahyuni, L. D., Gumela, G., & Maulana, H. (2020). Interrater Reliability: Comparison of Essay’s Tests and Scoring Rubrics. Journal of Physics: Conference Series, 1(1), 1–6. https://doi.org/10.1088/1742-6596/1933/1/012081
Wang, M. (2023). Bridging Assessment and Learning: A Cognitive Diagnostic Analysis of a Large-Scale Spanish Proficiency Test. Porta Linguarum, 2023(40), 9–24. https://doi.org/10.30827/portalin.vi40.15930
Wiggins, G., & McTighe, J. (2005). Understanding by Design (expanded 2nd ed.). In Association for Supervision and Curriculum Development.
Wilson, L. O. (2016). Anderson and Krathwohl Bloom’s Taxonomy Revised: Understanding the New Version of Bloom’s Taxonomy. Quincycollege.Edu, 2–4.
Wulandari, G. A. P. T. W., Putrayasa, I. B., & Martha, I. N. (2023). Efektivitas Asesmen Diagnostik dalam Pembelajaran Berdiferensiasi pada Pelajaran Bahasa Indonesia. Nusantara: Jurnal Pendidikan Indonesia, 3(3), 433-448. https://doi.org/10.14421/njpi.2023.v3i3-5
Yeh, H. W. (2022). Effects of Diagnostic Tests on Self-Directed Language Learning Development. International Journal of Computer-Assisted Language Learning and Teaching (IJCALLT), 12(1), 1-15. https://doi.org/10.4018/ijcallt.291105
Zhang, Z. V., & Hyland, K. (2022). Fostering Student Engagement With Feedback: An Integrated Approach. Assessing Writing, 51, 100586. https://doi.org/10.1016/j.asw.2021.100586
Zheng, Y., Yu, S., & Liu, Z. (2023). Understanding Individual Differences in Lower-Proficiency Students’ Engagement with Teacher Written Corrective Feedback. Teaching in Higher Education, 28(2), 301-321. https://doi.org/10.1080/13562517.2020.1806225
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with The KEMBARA: Jurnal Keilmuan Bahasa, Sastra, dan Pengajarannya (e-Journal) agree to the following terms:
Articles are published under the Creative Commons Attribution 3.0 Unported License (CC-BY 3.0).
Under the CC-BY license, authors retain ownership of the copyright for their article, but authors grant others permission to use the content of publications in KEMBARA: Jurnal Keilmuan Bahasa, Sastra, dan Pengajarannya (e-Journal) in whole or in part provided that the original work is properly cited. Users (redistributors) of KEMBARA: Jurnal Keilmuan Bahasa, Sastra, dan Pengajarannya are required to cite the original source, including the author's names, KEMBARA: Jurnal Keilmuan Bahasa, Sastra, dan Pengajarannya (e-Journal) as the initial source of publication, year of publication, volume number and DOI (if available).
Authors may publish the manuscript in any other journal or medium but any such subsequent publication must include a notice that the manuscript was initially published by KEMBARA: Jurnal Keilmuan Bahasa, Sastra, dan Pengajarannya (e-Journal).
Authors grant KEMBARA: Jurnal Keilmuan Bahasa, Sastra, dan Pengajarannya (e-Journal) the right of first publication. Although authors remain the copyright owner, they grant the journal the irrevocable, nonexclusive rights to publish, reproduce, publicly distribute and display, and transmit their article or portions thereof in any manner.
_.png)
