Examining students’ Self-Assessment of Digital Argumentation (SADA) in e-biology class: A Rasch analysis

Marheny Lukitasari a,1, Jeffry Handhika b,2, Wasilatul Murtafiah c,3, Agita Risma Nurhikmawati d,4,* a Department of Biology Education, Universitas PGRI Madiun, Address, Jl. Setia Budi No. 85, Madiun, East Java, 63118, Indonesia b Department of Physic Education, Universitas PGRI Madiun, Address, Jl. Setia Budi No. 85, Madiun, East Java, 63118, Indonesia c Department of Mathematic Education, Universitas PGRI Madiun, Address, Jl. Setia Budi No. 85, Madiun, East Java, 63118, Indonesia d Department of English Education, Universitas PGRI Madiun, Address, Jl. Setia Budi No. 85, Madiun, East Java, 63118, Indonesia 1 marheny@unipma.ac.id*; 2 jeffry.handhika@yahoo.com; 3 wasila.mathedu@unipma.ac.id, 4 agitarnurhikmawati@unipma.ac.id* * Corresponding author


INTRODUCTION
Person who have good ability in argumentation will express thoughts in understanding problems, showing logical reasons, explaining, and defending opinions in their daily lives (Palau & Moens, 2009). This ability will increase when the teachers arrange the class to encourage students in doing so. The appearing challenges during that process were able to help students understand teachers' material (Gyenes, 2017;Stiegelmayr & Mieskes, 2018). Furthermore, argumentation is significantly needed since it is one method to train critical thinking which becomes the purpose of higher education (Goodnight, 2009). Delivering these arguments can be done orally through a direct discussion and submitted through written text.
Along with the development of technology-based media, information, and communication (ICT), the argument can be in the form of a digital format, which then called as Digital Argumentation (DA). This development affects the conditions and ways of student learning (Haelermans, 2017). DA in the form of argument or opinion written through ICT-based media such as blogs, papers, narratives or papers on a particular theme that is communicated to public. In its development, DA is not only in the form of writing or narration, but also be accompanied by photos, pictures, or videos (Pfister, 2010) as reinforcing evidence in discussing particular topic. The concept of DA is a paradigm in the development of education with the integration of technology as a format of digital revolution (Ravenscroft & McAlister, 2008).
A good argument needs to elaborate two opposing sides, namely the submission of an opinion (claim) and a rebuttal (counter claim) to build a conclusion (Lam, Hew, & Chiu, 2017). Claims and counter claims will be clearer and more structured if it is written like DA, compared to if it is expressed orally. Writing an argument sharpens the thinking process and manages students' emotions. In addition, during the process of argument writing, students need to understand the material, thus they are able to arrange sentences that are appropriate for discussion. Challenges will increase when the submitted DA receives a rebuttal supported by evidences.
DA becomes important along with the increasing implementation of e-learning, especially in higher education. Some of the benefits of e-learning include: flexible in use, flexible in time, unlimited access (Perbawaningsih, 2013), more effective and efficient, to be able to assist students in technology transfer so that they become more competitive (Górska, 2016). E-learning is also recommended in implementing blended learning by integrating it with traditional learning in the classroom because it offers a variety of advantages (Owston, 2018). The more e-learning is applied, the higher number of the research evidences that confirmed the benefits of the process and the results felt by institutions, teachers, and students. Recent research trends tend to review the use of e-learning (Ghavifekr & Rosdy, 2015;Holmström & Pitkänen, 2012), the implementation management, or the combination of e-learning models to improve student abilities. Research on evaluating the level of satisfaction and what is felt by students when studying online still need to be investigated (Castle & McGuire, 2010). Therefore, research on e-learning evaluation tools is urgently required especially those related to self-assessment of students' DA. The results of that research are able to be used as a basis for developing the teaching process and improving the education policies in the future.
The application of DA was running for two semesters at Universitas PGRI Madiun (UNIPMA) since the institution developed its own e-learning platform, e-learning UNIPMA (eLMA). However, the conventional argumentation task, both writing and oral, has been applied previously by the lecturers. Based on observation result held in the previous semester, the conventional one showed there were 68% students in the Faculty of Teaching and Education who performed poorly on it. This condition occurs mostly due to the absence of an adjusted model or method which helps students to increase their performance in writing arguments. Moreover, Indonesia along with several Asian nations practices high context culture where most argumentation is expected to come implicitly and as polite as possible; in other words, the use of non-aggressive content even omitting the truth sometimes are allowed (Samovar, Porter, & McDaniel, 2012). After the use of eLMA, the students' attitude towards expressing argument has changed. They are more comfortable in their writing supported by experts' theories and evidence because they sense an environment of anonymity (Kelly, 2016), which is less pressure and judgment from peer's gestures.
Given the importance of argumentation skills, various studies that examine students' argumentation skills have been conducted. There is research that examines the effect of argumentation models on students' argumentation skills in science (Okumus & Unal, 2012). There are also several studies that examine students' argumentation skills in online learning (Tsai & Tsai, 2014;Yeh & She, 2010). However, these studies do not clearly inform the characteristics and strengths of the DA measurement instrument. The use of questionnaires in survey research is an important part of getting a picture of the condition of the population or research sample. However, the weakness of questionnaire uses as stated by Kalita et al. (2015) is in the process of analysis. The reliability and validity of the questionnaire as a research instrument are often be a problem, especially related to the measure of particular aspects. Related to the higher number of e-learning application, thus the reliable and valid evaluation is needed (Hubackova, 2015;Misut & Pribilova, 2015). This evaluation provides feedbacks to strengthen the application in the future (Popovici & Mironov, 2015). In addition, DA instruments that suit the conditions of Indonesian students, especially at UNIPMA, have not been found. Therefore, in this study, the Self-Assessment of Digital Argumentation (SADA) was developed. SADA is a questionnaire designed to map the condition of students while implementing DA in e-learning. What students feel when writing DA cannot be ignored because it can be a clue in knowing the potential difficulties faced by them as well as the DA qualifications they reveal. Furthermore, the aim of this study were to (1) investigate how SADA can be used as an instrument for assessing students' DA, and (2) examine the qualifications and classifications of students' DA through SADA in e-Biology class.

METHOD
This study used a quantitative approach to reveal SADA profile as a self-assessment tool for students' DA in e-Biology class. Data collection was carried out by conducting a survey using SADA questionnaire filled out by students after completing learning for one semester.
The study population was 132 students of Biology Education Department-UNIPMA. The research sample was 64 students from different semesters and subjects. A total of 28 students were taken from the third semester of the academic year 2019/2020 in Cell Biology, and 36 students from the second semester of 2020/2021 who were in Cryptogam course. In addition, both courses have material complexity which then should be conducted in the form of e-learning. Sampling was done by purposive method since both classes have been consistently using eLMA since the beginning of the semester.
The instrument used was SADA questionnaire consisting of 18 questions. SADA was compiled and developed from Smith, Kiili, and Kauppinen (2016) and Viyanti, Cari, Sunarno, and Prasetyo (2016) based on indicators, namely: 1) claim/ argument delivered, 2) evidence for the argument delivered, 3) linkages of evidence and argument, 4) quality of argument, 5) refutation and 6) logical thinking. Based on these indicators, then codes were given for each statement in the questionnaire which arranged in Table 1. Each indicator was expanded in detail into three statements which then filled out by students using Likert scale indicating their conditions: 1) unsure; 2) less sure; 3) pretty sure; 4) sure and 5) highly sure. Only one response was allowed for each statement inside SADA. The data then were analyzed using Rasch model, in specific by running WINSTEPS software version 4.3.2 in which the data were mathematically converted into logit (odd logarithmic units) through logarithmic functions (Sumintono & Widhiarso, 2014). Logarithmic function is used to convert raw ordinal data, for example: Likert data, into the same interval scale by changing the raw score obtained from filling out the questionnaire into linear data. Thus, it matches the value that tends to reveal the real condition of the respondents. In this study, Rasch is used to examine the relationship between the difficulty of questions, the ability of students, and the probability of response given to each given indicator. Each statement submitted in SADA was constructed to provide an effective measurement of all indicators.
The result in figures was interpreted by examining the main components of Rasch's analysis through the Wright map that visually depict the relationship between respondents and question items in SADA (Sumintono & Widhiarso, 2014). The vertical line is a log interval scale, a unit of measurement that represents people (respondents) with questionnaire items. The very top describes questions which get answer 'unsure', while the very bottom expresses the easiest question which most likely earn answer 'highly sure). Those logits person (measure) were then classified into high, medium and low based on the normal curve formula. The results of the analysis were also strengthened by summary statistics to obtain more information related to the quality of respondents and question items from SADA.

SADA as a student's DA self-assessment tool
Traditional analysis of data using non-parametric statistics often produces incorrect conclusions. Thus, corrections to the analysis can be minimized by the Rasch model. Rasch model is not only able to detect the difficulty level of the items/ statements, but also to measure and classify the ability of respondents. Thus, by using this model, SADA provides more accurate and valid result. Based on Rasch model, the statements in SADA show that the instrument was able to distinguish the conditions felt by students while delivering DA in e-learning. Assessment tools designed online can reduce tension and encourage students to be more confident in completing their assignments (Lukitasari, Handhika, & Murtafiah, 2018). It can be seen from Figure 1 that the items C1, C2, C3, N7, N8, K10, L18 (38.89%) were located in the bottom; thus, they were answered by students with 'sure' and 'highly sure' options. All C questions (claim) are in this layer which mean that students had been responsible for mastering the material so that students were comfortable in expressing their arguments. Furthermore, there were 2 statements of N (linkage of evidence and argument) here; therefore, it can be said that most aspects of reasonable argumentation using evidence have been successfully carried out by students. K10 is classified as the easiest question because it is placed on the lowest; and both K10 and L18 represent students' achievements in being relevant to the topic and logic in their arguments. There were no items B (submission of evidence) and S (submission of the rebuttal) which answered 'sure' and 'highly sure' by the students. S13: Delivering a refutation towards an argument by attaching supporting evidences S15: Refutation is delivered approximately more than once until reaching conclusions that are agreed upon and based on evidences B6: Connecting arguments presented with supporting evidences L16: Ensuring the objectivity of arguments based on evidences S14: Refutation is delivered to strengthen opinions and thoughts related to the topic/ material being discussed B4: Delivering evidences in each argument presented B5: The evidences come from a trusted source K12: The structure of the argument is adjusted to evidences, data and related topics C1: Delivering argument in a discussion forum for each topic discussed N8: Using reasoning in delivering arguments and evidence in discussion forums C2: Considering the truth of the arguments presented in the discussion forum C3: Understanding the material before delivering an argument N7: Thinking about the relevance of the argument with the evidence presented in the discussion forum L18: Arguments delivered are logical and can be justified based on evidence K10: Every argument delivered is always related to the topic/ material being discussed K11: Paying attention to the composition of sentences arranged in the arguments presented at the discussion forum L17: The argument delivered is to gain comprehensive (not partial) understanding N9: Thinking about the rationality of each argument supported by evidence to be presented in the discussion forum. In line 0, there were items N9, K11, L17 (16.67%) which were answered 'pretty sure' by students. There were no items C (Claim / DA), B (evidence for the argument delivered), and S (refutation) in this area. On items B4, B5, B6, K12, S13, S14, S15, L16 (44.44%), the students answered 'unsure' and 'less sure'. All questions with code B (evidence for the argument delivered) and S (refutation) are above; This condition shows that in the e-learning process, even though the lecturer has directed that the DA and refutation must be accompanied by supporting evidence to reinforce and sharpen the theme discussed, this has not been done by students optimally. Item S13 is the most difficult statement to answer and this item can provide a description of the condition of students who were not comfortable providing a refutation and supporting evidence for an argument. Items code K (argument quality) and L (logical thinking) are 2 indicators scattered on the top, middle, and below which mean that these two get the most varied answers compared to other codes. The students' effort to present quality and logical arguments become considerations that were continually felt and thought out so that the logits spread throughout the map.  No item C (Claim / DA) and N (linkages of evidence and argument) were found with the answers 'less sure' and 'unsure'. That is because the submission of DA became a mandatory task given by lecturers in each discussion session. This condition provides reasons for students to feel the urgency in showing their existence by submitting DA even though sometimes they were not sure regarding the quality of their arguments. DA is proven to affect the conditions and ways of student learning (Haelermans, 2017). Giving assignments to deliver DA was also a motivation and goal for students to be active when e-learning was carried out. The existence of such motivation becomes an important element that needs to be considered in stimulating students' confidence (Mubeen, 2014).
It can be summarized (see Table 2) that students had difficulties in providing evidences and delivering refutation. To solve these, students have to practice more; outlining, using simple sentence structure, and practicing speaking skill are encouraged. It is also important to prepare the class before the d-day including to read the material, even complete the task so that students gain a holistic understanding of those. It would help students to write their DA (Lehti & Kallio, 2017).
Some examples of DA presented by the students during an online discussion are shown in Figure 2. It discussed the role of mosses in the weathering process which occurs on walls. The rising stimulus questions encouraged students to write DA based on their understanding. They directly mentioned the type of grass within the DA that can grow on a surface that lacks of water. However, they had not yet explained the weathering process related to the presence of the moss. DA 1 and DA 2 did not show any refutation and evidence. While DA 3, a student has begun to link concepts of the benefits of mosses and the process of weathering. It can be seen that every submitted DA will encourage other students to think of possible answers that are relevant by attaching the required evidence. It means that one DA is able to influence other students' perception even though it has not accompanied by evidences. And the presence of internet should open wider information; thus, it helps the students in making a better DA (Pfister, 2010).

DA classification of students through SADA in e-Biology class
The classification of students' ability can be divided into 3 categories namely high, medium, and low. Respondents with high classification answered SADA with 'sure' and 'highly sure' were 19 (29.69%). Respondents who answered 'pretty sure' are 26 (40.63%), and were classified as medium. Whereas students who answered 'less sure' and 'unsure' were 19 (29.69%) put into low (Table 3).  Table 3 shows that some students needed to improve their DA capabilities, especially in delivering the evidence-based argumentation. They were aware that should master the material in cell biology and cryptogamae courses so that they could produce a strong DA. The logits which show the overall quality of SADA is depicted by Table 4. The person (0.92) and item (0.75) reliability index described the consistency of the students in working their SADA; and it is categorized as "very good" (Sumintono & Widhiarso, 2014). The alpha Cronbach value (09.5) confirmed that SADA has high consistency as a measurement tool. This condition also indicates that most students have high certainty/ confidence in delivering DA when studying online. Basically, students need their confidence to present DA in their learning activities (Firmansyah, Komala, & Rusdi, 2018;Kalita et al., 2015). Furthermore, the person and item separation index were 3.46 and 1.74; the wider the range of the index, the better quality of the instruments have. In addition, the mean of logit person was 1.86 with the SD 2.06 (see Table 5). Since that logit is higher than 0.00, this means all students tended to agree with all SADA statements. In other words, students understand that those statement are truly needed to build a strong DA. Moreover, the mean of item was 0.00 with SD 0.45. This shows that all items' difficulty was equally scattered within the instrument. It can be summarized that the result of this study proved that SADA is able to be used to measure students' self-assessment in working their DA during e-learning. SADA also helps students to evaluate themselves so that they can strengthen their works. This encourages them to know their ability due to increasing their perception in e-learning process (Popovici & Mironov, 2015).
This kind of research still needs to be encouraged in Indonesia. The measurement of DA and the application of e-learning are two urgent conditions to be implemented continuously. Mapping student DA will be the basis for lecturers to find out how good their student DA is. A good DA level indicates that the argumentation skills of students are empowered. The empowerment of DA will make it easier for them to understand the problems and defending their opinions in their daily lives (Palau & Moens, 2009). In addition, having a high DA will also improve their critical thinking skills (Goodnight, 2009). Increasing the number of studies related to DA will also encourage the rate of digitalization of education. The digitization of education itself is expected to be achieved in the 21 st century.

CONCLUSION
The SADA instrument can be used for self-assessment of students' DA in e-Biology class. The results of the analysis using Rasch model reveals that items B (evidence for the argument delivered) and S (refutation) are the most difficult aspects for the students. On the other hand, the easiest ones consist of items C (claim) and N (linkages between evidence and argument). Whereas items K (quality of argument) and L (logical thinking) are scattered which means that some students already master it, while others do not. In addition, statistical summary shows that the Cronbach alpha value is 0.95> 0.8 indicating that the interaction between the respondent and the item is very good. Person reliability value is 0.92 and item reliability is 0.75, which means both are in the good category. These conditions represent that on average all students have high certainty/ confidence in delivering DA during e-learning. This study also depicts that there are 26.69% students classified as high, 40.63% as medium, and 29.69% as low in working their DA.
Furthermore, it is necessary to convey and acknowledge the limitations of this research that can be used as the improvement for future studies. This research is still limited to a sample of only two classes and two courses. More extensive research using more participants and courses needs to be carried out. SADA which used as an instrument to measure DA can be developed with more detailed and specific statement items so that it will measure the condition of students better.