Construction and identification of psychometric property self-regulated learning scale for university students

Previous studies state that self-regulated learning is a pivotal component in predicting students’ learning outcome. The present study aims to test the psychometric property of self-regulated learning scale to obtain a sound instrument. The procedure in this study included the scale construction, followed by psychometric property identification covering the content validity (using Aiken’s V and reliability test (Cronbach’s Alpha) of the data obtained from 203 respondents. The reliability coefficient of the final scale was 0.908, its discriminating power ranged between 0.307 and 0.626 with a mean score of 0.462. In other words, the self-regulated learning scale exhibit adequate psychometric property to measure university students’ self-regulated learning.

It can also be defined as an individual's effort to regulate control that direct and maintain individuals to achieve the 36 learning goal. 37 Self-regulated learning comprises four aspects (Pintrich, 38 2000): (1) Cognitive control, involving cognitive and 39 metacognitive activities, (2) Motivation, involving efforts to 40 maintain one's motivation. (3) Behavior, involving one's effort 41 to control his/her behaviors. (4) Context, involving one's 42 effort control the context when engaging with the classroom 43 learning activities. Furthermore, Pintrich (2000) state that 44 individuals with proper self-regulated learning skill are able 45 to set their learning goal and plan,monitor and control their 46 cognitive aspects, motivation, and behaviors to achieve the 47 goal. 48 DiFrancesca et al. (2016) State that the important difference 49 between high and low achievers lies in their self-regulated 50 learning skills, (e.g., their ability to perform metacognitive 51 control, use learning strategy, and self-efficacy). The study 52 conducted by Dörrenbächer & Perels (2016) reports that self-53 regulated learning is significantly associated with students' 54 high achievement and low test anxiety, lower neurotic level, 55 and represent a readiness to actively receive the learning 56 process. 57 Considering the important role of self-regulated learning 58 in academic achievement, it is necessary to develop a 59 quality instrument. There are currently two widely used SRL 60 instruments, Motivated Strategies for Learning Questionnaire 61 (MSLQ) and Learning  Pintrich's (2000) SRL theory that comprises four aspects: 125 cognition, motivation, behavior, and context. The first step 126 of the development was constructing the blueprint. The 127 spearman-Brown formula was applied to estimate the required 128 number of items based on the expected reliability coefficient 129 and average discriminating power index (Suryabrata, 2005). 130 Since the expected reliability coefficient of the blueprint was 131 0.85 with discriminating power of 0.40, thirty items were 132 required. As the construct comprises four aspects, each aspect 133 should be represented by eight items, divided into favorable 134 and unfavorable item groups. The total constructed items 135 were 32 items. This number of items were doubled to be 136 64 items to anticipate being dropped during the test. The 137 blueprint of developed SRL scale is displayed in Table 1. After 138 constructing the blueprint, the next step was to write down 139 the sixty-four items based on the indicators. The scale was 140 developed using 4-point likert scale (Strongly agree, agree, 141 disagree, strongly disagree). These items were assessed by 142 five professional judgments for its relevance, these experts 143 have adequate experiences related to SRL study and hold at 144 least Master degree in Psychology. The expert judgment was 145 used as the basis to calculate the content validity coefficient 146 of each item using Aiken's V formula. The next stage 147 was trial stage. In this stage, respondents were recruited 148 using purposive sampling technique. They were 203 students 149 (11.8% male, 88.2% female) pursuing vocational education 150 (1%), undergraduate degree (92%), and postgraduate degree 151 (7%). The data were collected using Google Forms distributed 152 through social media. The data were analyzed using SPSS 23 153 to examine the psychometric properties (discriminating power 154 and reliability coefficient). The factor analysis was done to 155 see its construct validity prior to the final compilation stage. 156

158
The content validity test was performed to see the relevance 159 of each items (Azwar, 2014). Aiken's V formula was applied 160 to see the content-validity coefficient based on the panel 161 judgment of the item relevance with the measured construct. 162 The score ranged between 1-5 (1 = not relevant with the 163 theoretical construct); 5 =highly relevant with the theoretical 164 construct) The five expert judgments were analyzed using 165 Aiken's V formula (V = Σs/[n(c − 1)]), and in order to 166 find Σs, the formula of Σ s = s1 + s2 + s3 was applied 167 (Azwar, 2017). Any V higher than 0.50 indicates a high 168 content validity (Azwar, 2017). Aiken's V estimation result is 169 presented in Figures 1.

170
As displayed in Figure 1, items no.1 to 30 exhibited an 171 Aiken's V value higher than 0.5, indicating a good content 172 validity. Meanwhile, figure 1 also shows two items with 173 Aiken's V value equal to and lower than 0.5 (i.e., item no. 59 174 (V = 0.5) and item 64 (V=0.45). This shows that there were 175 sixty two items (96.875%) with V coefficient value higher 176 than 0.5. Items no. 59 and 64 were dropped because their 177 values equal and lower than 0.5. Considering that there were 178 too many items left, those with V value lower than 0.6 was 179 also dropped (i.e., item 6 and item 10, V = 0.55, respectively), 180 leaving sixty items with good content validity coefficient. The 181 average content validity coefficient of these 60 items was 182 0.831, indicating good relevance with the measured construct. 183

184
The instrument was tested after performing the content 185 validity test, 60 items were considered having a good content 186 validity. All items were renumbered. The test blueprint was 187 displayed in Table 2.

188
The test was performed online using Google form (https: 189 //bit.ly/risetmahasiswa2020), involving university students.  The obtained data were analyze to see the discriminating  0.30 and to obtain an item composition that suits the initial 205 research goal. At the first stage, sixty items were analyzed and 206 results in a Cronbach's Alpha of 0.931 with discriminating 207 power ranging from -0.11 to 0.635. Eleven items with rit 208 < 0.30 were dropped, leaving 49 items with discriminating 209 power higher than 0.30. At the second stage, the cronbach 210 alpha coefficient of 49 items was 0.933 with discriminating 211 power ranging between 0.321 and 0.627. By considering 212 item proportionality of each aspect (25% for each aspect), 213 the number of items were adjusted to that of initial blue print. 214 The process resulted in 32 items with reliability coefficient of 215 0.908 and discriminating power ranging between 0.307 and 216 0.626 (average discriminating power: 0.462) (Table 3).

217
The final version of the scale consists of 32 items with 218 content validity coefficient ranging from 0.6 to 1 and average 219 V value of 0.8326 (Table 3). The validity and reliability tests 220 showed that the developed SRL scale exhibited an adequate 221 psychometric properties, indicated by reliability coefficient 222 of 0.908, average discriminating power of 0.642, and content 223 validity coefficient of 0.836. The blueprint of final version of 224 the scale is presented in table 4.

226
In this study, the construct validity of the scale was examined 227 using the exploratory factor analysis, an analysis aiming to 228 see whether the items of the scale represents the aspects 229 intended to to measure, and whether there is a relationship 230 between these aspects. Self-regulated learning theoretically 231 comprises four aspects: cognition, motivation, behavior, and 232 context (Pintrich, 2000) and we tested the final version of 233 the scale to see whether these 32 items represent those four 234 aspects.

235
The KMO value was 0.834 and the Bartlett's test of 236 sphericity significance was 0.000, allowing us to proceed 237 to the factor analysis. The 32 items were extracted into four 238 factors, considering that SRL is theoretically comprises 4 239 aspects. The total variance explained indicate that reducing 32 240 items into four factors allow the scale to account for 46.696% 241 variance. The factor loading of each factor is displayed in 242 Table 5.   power of an item refers to the extent to which an item is 277 able to separate respondents with lower trait from those with 278 higher trait.. In this regard, item A34 possesses the best 279 ability to separate individuals with low SRL from those with 280 high SRL. This unfavorable item represent the behavioral 281 aspect, which reads: "I only study when I want to". Finch et 282 al.

243
(2016) argue that item's discriminating power indicates 283 a relevance with the trait being measured. This shows that 284 item A34 in this scale measures one's self-regulated learning 285 from different direction. Respondents with low score on this 286 item indicates high self-regulated learning. However, it is 287 necessary to sum up the total score of the items before drawing 288 a conclusion. Item A34 also exhibits high content-validity 289 coefficient (V=0.9). Experts view this item highly relevant 290 with the measured construct, i.e., self-regulated learning. In 291 other words, Item A34 possesses good quality in terms of 292 content validity and the discriminating power.

293
Item A35's discriminating power was slightly above 294 the minimum requirement, i.e., 0.307, however, it is still 295 considered adequate, as the minimum discriminating power is. 296 0.30 (Azwar, 2017). According to Furr & Bacharach (2013), 297 content validity (not including the face validity) serves as 298 the important evidence when evaluating a construct validity. 299 It means that the content validity of a measure is likely to 300 determine its construct validity. In this regard, Azwar (2014) 301 states that a scale's content validity coefficient is affected 302 by the content validity of each item.Item A35 is a positive-303 worded item reads: "I know the efforts I have made in 304 learning". It has a high content validity (V=0.95). Four of 305 five experts scored this item 5, while an expert scored this 306 item 4. This judgment indicates that four experts agree that 307 A35 is relevant to measure SRL, especially the behavioral 308 aspect (2nd behavioral indicator: able to monitor learning 309 efforts). Overall, the discriminating power and the content 310 validity coefficient of item A35 is quite good.

311
Item A39 exhibited Aiken's V of 1, indicating a very high 312 content validity and considered highly relevant to measure 313 Since the item in this scale was selected based on 332 its discriminating power index, the quality of the item is  i.e., exploratory factor analysis. Their study reports the factors 364 affecting the factor analysis result, including items that are 365 not in line with their place due to inter-item correlation that 366 does not suite the measured construct. Similar condition 367 appears to occur in the present study, as writing an item is 368 challenging as it should matches the construct. In order to 369 construct an item, it is necessary to formulate the indicator 370 of each aspect. In the present study, the four SRL aspects 371 were derived into different indicators. However, the factor 372 analysis result showed that some items overlapped, despite 373 the professional judgment done to ensure the relevance. This 374 should be valuable reminder for future studies regarding the 375 item relevance with the measured construct.

376
The validity and reliability tests showed that the 377 developed SRL scale exhibited adequate psychometric 378 properties, indicated by reliability coefficient of 0.908, 379 average discriminating power of 0.642, and content validity 380 coefficient of 0.836. However, its construct validity requires 381 further evaluation. The scale's poor empirical evidence of 382 the construct validity emerges as the limitation of the present 383 study, in addition to the respondents' factor who were mostly 384 undergraduate students (92%).

386
Overall, the developed self-regulated learning scale exhibited 387 adequate psychometric properties, indicated by its reliability 388 coefficient, average discriminating power, and content validity, 389 yet it lacks strong empirical evidence of the construct 390 validity. In this regard, future studies are recommended to 391 strengthen the empirical evidence of the construct validity 392 while involving more diverse respondents.