' item parameter' Search Results
Somers' D as an Alternative for the Item–Test and Item-Rest Correlation Coefficients in the Educational Measurement Settings
item analysis pearson correlation somers' d item–total correlation item–rest correlation item discrimination power...
Pearson product–moment correlation coefficient between item g and test score X, known as item–test or item–total correlation (Rit), and item–rest correlation (Rir) are two of the most used classical estimators for item discrimination power (IDP). Both Rit and Rir underestimate IDP caused by the mismatch of the scales of the item and the score. Underestimation of IDP may be drastic when the difficulty level of the item is extreme. Based on a simulation, in a binary dataset, a good alternative for Rit and Rir could be the Somers’ D: it reaches the ultimate values +1 and –1, it underestimates IDP remarkably less than Rit and Rir, and, being a robust statistic, it is more stable against the changes in the data structure. Somers’ D has, however, one major disadvantage in a polytomous case: it tends to underestimate the magnitude of the association of item and score more than Rit does when the item scale has four categories or more.
Generalized Discrimination Index
kelley’s discrimination index item parameter item–total correlation item analysis classical test theory...
Kelley’s Discrimination Index (DI) is a simple and robust, classical non-parametric short-cut to estimate the item discrimination power (IDP) in the practical educational settings. Unlike item–total correlation, DI can reach the ultimate values of +1 and ‒1, and it is stable against the outliers. Because of the computational easiness, DI is specifically suitable for the rough estimation where the sophisticated tools for item analysis such as IRT modelling are not available as is usual, for example, in the classroom testing. Unlike most of the other traditional indices for IDP, DI uses only the extreme cases of the ordered dataset in the estimation. One deficiency of DI is that it suits only for dichotomous datasets. This article generalizes DI to allow polytomous dataset and flexible cut-offs for selecting the extreme cases. A new algorithm based on the concept of the characteristic vector of the item is introduced to compute the generalized DI (GDI). A new visual method for item analysis, the cut-off curve, is introduced based on the procedure called exhaustive splitting.
A High-Stakes Approach to Response Time Effort in Low-Stakes Assessment
accelerated failure time model piaac response time survival analysis...
Response times are one of the important sources that provide information about the performance of individuals during a test process. The main purpose of this study is to show that survival models can be used in educational data. Accordingly, data sets of items measuring literacy, numeracy and problem-solving skills of the countries participating in Round 3 of the Programme for the International Assessment of Adult Competencies were used. Accelerated failure time models have been analyzed for each country and domain. As a result of the analysis of the models in which various covariates are included as independent variables, and response time for giving correct answers is included as a dependent variable, it was found the associations between the covariates and response time for giving correct answers were concluded to vary from one domain to another or from one country to another. The results obtained from the present study have provided the educational stakeholders and practitioners with valuable information.
Number of Response Options, Reliability, Validity, and Potential Bias in the Use of the Likert Scale Education and Social Science Research: A Literature Review
likert scale literature review potential bias reliability and validity...
This study reviews 60 papers using a Likert scale and published between 2012 – 2021. Screening for literature review uses the PRISMA method. The data analysis technique was carried out through data extraction, then synthesized in a structured manner using the narrative method. To achieve credible research results at the stage of the data collection and data analysis process, a group discussion forum (FGD) was conducted. The findings show that only 10% of studies use a measurement scale with an even answer choice category (4, 6, 8, or 10 choices). In general, (90%) of research uses a measurement instrument that involves a Likert scale with odd response choices (5, 7, 9, or 11) and the most popular researchers use a Likert scale with a total response of 5 points. The use of a rating scale with an odd number of responses of more than five points (especially on a seven-point scale) is the most effective in terms of reliability and validity coefficients, but if the researcher wants to direct respondents to one side, then a scale with an even number of responses (six points) is possible. more suitable. The presence of response bias and central tendency bias can affect the validity and reliability of the use of the Likert scale instrument.
Graded Response Models on the Curiosity Measurement of Elementary School Students
curiosity measurement elementary school graded response models...
Curiosity is one of the most important characters for elementary school students. However, the facts in the field show that the measurement model used by the teacher to identify the student's curiosity is not yet available in a standardized manner. This study aims to develop a model for measuring the curiosity of elementary school students using the graded response model (GRM) approach. This research uses quantitative method with descriptive type. The research sample used was 236 elementary school students who were randomly selected. Data were collected using a questionnaire of 16 statement items using a Likert scale approach. The data were analyzed using the response item theory approach with the GRM. The results showed that the model for measuring student curiosity in elementary schools had good location parameters, a good discriminant index, a fairly good information function with a small estimation error. The curiosity measurement model in this study can be used as an alternative for teachers to identify students' curiosity in elementary schools.