Goodman–Kruskal gamma and Dimension-Corrected Gamma in Educational Measurement Settings
Although Goodman–Kruskal gamma (G) is used relatively rarely it has promising potential as a coefficient of association in educational settings..
- Pub. date: February 15, 2021
- Pages: 95-118
- 975 Downloads
- 1418 Views
- 10 Citations
Although Goodman–Kruskal gamma (G) is used relatively rarely it has promising potential as a coefficient of association in educational settings. Characteristics of G are studied in three sub-studies related to educational measurement settings. G appears to be unexpectedly appealing as an estimator of association between an item and a score because it strictly indicates the probability to get a correct answer in the test item given the score, and it accurately produces perfect latent association irrespective of distributions, degrees of freedom, number of tied pairs and tied values in the variables, or the difficulty levels in the items. However, it underestimates the association in an obvious manner when the number of categories in the item is more than four. Towards this, a dimension-corrected G (G2) is proposed and its characteristics are studied. Both G and G2 appear to be promising alternatives in measurement modelling settings, G with binary items and G2 with binary, polytomous and mixed datasets.
Keywords: Item analysis, Goodman–Kruskal gamma, Somers D, Jonckheere–Terpstra test, Pearson correlation.
References
Agresti, A. (2010). Analysis of ordinal categorical data (2nd ed.). Wiley.
Arbuthnott, J. (1710). An argument for divine providence, taken from the constant regularity observed in the births of both sexes. Philosophical Transactions of the Royal Society of London, 27(325–336), 186–190. http://doi.org/10.1098/rstl.1710.0011
Aslan, S., & Aybek, B. (2020). Testing the effectiveness of interdisciplinary curriculum-based multicultural education on tolerance and critical thinking skill. International Journal of Educational Methodology, 6(1), 43–55. https://doi.org/10.12973/ijem.6.1.43
Bai, J., & Wei, L.-L. (2009). A new method of attribute reduction based on gamma coefficient. In S.-M. Zhou & W. Wang, GCIS 2009. 2009 WRI Global Congress on Intelligent Systems (pp. 370–373). IEEE Computer Society. https://doi.org/10.1109/GCIS.2009.212
Bravais, A. (1844). Analyse Mathematique. Sur les probabilités des erreurs de situation d'un point [Mathematical analysis. On the probabilities of the point errors]. Imprimerie Royale.
Breslow, N. (1970). A generalized Kruskal–Wallis test for comparing K samples subject to unequal patterns of censorship. Biometrics/ Biometrika, 57(3), 579–594. http://doi.org/10.1093/biomet/57.3.579
Byrne, B. M. (2016). Structural Equation Modeling with AMOS. Basic concepts, applications, and programming (3rd ed.). Routledge.
Cheng, Y., Yuan, K.-H., & Liu, C. (2012). Comparison of reliability measures under factor analysis and item response theory. Educational and Psychological Measurement, 72(1), 52–67. https://doi.org/10.1177/0013164411407315
Cleff, T. (2019). Applied Statistics and Multivariate Data Analysis for Business and Economics. A Modern Approach Using SPSS, Stata, and Excel. Springer.
Conover, W. J. (1980). Practical nonparametric statistics. Wiley & Sons.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrics/ Psychometrika, 16(3), 297–334. http://doi.org/10.1007/BF02310555
Davis, J. A. (1967). A partial coefficient for Goodman and Kruskal's gamma. Journal of the American Statistical Association, 62(317), 189–193. https://doi.org/10.1080/01621459.1967.10482900
Delil, A., & Ozcan, B. N. (2019). How 8th graders are assessed through tests by mathematics teachers? International Journal of Educational Methodology, 5(3), 479–488. https://doi.org/10.12973/ijem.5.3.479
Drasgow, F. (1986). Polychoric and polyserial correlations. In S. Kotz & N. L. Johnson (Eds.), Encyclopedia of statistical sciences. (Vol. 7, pp. 68–74). John Wiley.
El-Shaarawi, A. H., & Piegorsch, W. W. (2001). Encyclopedia of Environmetrics (Volume 1). John Wiley and Sons.
Finnish National Education Evaluation Centre (2018). National assessment of learning outcomes in mathematics at grade 9 in 2002. Unpublished dataset opened for the re-analysis 18.2.2018. Finnish National Education Evaluation Centre.
Forthmann, B., Förster, N., Schütze, B., Hebbecker, K., Flessner, J., Peters, M. T., & Souvignier, E. (2020). How much g is in the distractor? Re-thinking item-analysis of multiple-choice items. Journal of Intelligence, 8(1), 1-36. https://doi.org/10.3390/jintelligence8010011
Freeman, L. C. (1986). Order-based statistics and monotonicity: A family of ordinal measures of association. Journal of Mathematical Sociology, 12(1), 49–69. https://doi.org/10.1080/0022250X.1986.9990004
Galton, F. (1889). Kinship and correlation. Statistical Science, 4(2), 81–86. http://doi.org/10.1214/ss/1177012581
Gehan, E. A. (1965). A generalized Wilcoxon test for comparing arbitrarily single-censored samples. Biometrics/ Biometrika, 52(1–2), 203–233. http://doi.org/10.1093/biomet/52.1-2.203
Gini, C. (1912). Variabilità e mutabilità. Contributo allo studio delle distribuzioni e dellerelazioni statistiche [Variability and mutability. Contribution to the study of distributions and statistical relationships]. Bologna.
Göktaş, A., & İşçi., O. A. (2011). Comparison of the most commonly used measures of association for doubly ordered square contingency tables via simulation. Methodological Notebooks / Metodološki zvezki, 8(1), 17–37.
Gonzalez, R., & Nelson, T. O. (1996). Measuring ordinal association in situations that contain tied scores. Psychological Bulletin, 119(1), 159–165. https://doi.org/10.1037/0033-2909.119.1.159
Good, K. (2015). Investigating relationships between educational technology use and other instructional elements using "big data" in higher education [Doctoral dissertation, Iowa State University]. Iowa State University Digital Repository. https://lib.dr.iastate.edu/etd/14854
Goodman, L. A., & Kruskal, W. H. (1954). Measures of association for cross classifications. Journal of the American Statistical Association, 49(268), 732–764. http://doi.org/10.1080/01621459.1954.10501231
Goodman, L. A., & Kruskal, W. H. (1979). Measures of association for cross classification. Springer-Verlag.
Green S. B., & Yang Y. (2009). Commentary on coefficient alpha: A cautionary tale. Psychometrics/ Psychometrika, 74(1), 121‒135. http://dx.doi.org/10.1007/s11336-008-9098-4
Greiner, R. (1909). Über das Fehlersystem der Kollektivmaßlehre (Of the error systemic of collectives). Journal of Mathematics and Physics / Zeitschift fur Mathematik und Physik, 57, 121–158, 225–260, 337–373.
Guttman, L. (1950). The basis for scalogram analysis. In S. A. Stouffer, L. Guttman, E. A. Suchman, P. F. Lazarsfield, S. A. Star, & J. A. Clausen (Eds.), Measurement and prediction (pp. 60 – 90). Princeton University Press.
Harrell, F. (2001). Regression Modelling Strategies. Springer.
Harrell, F. E., Califf, R. M., Pryor, D. B., Lee, K. L., & Rosati, R. A. (1982). Evaluating the yield of medical tests. Journal of the American Medical Association, 247(18), 2543–2546. http://doi.org/10.1001/jama.1982.03320430047030
Heagerty, P. J., & Zheng, Y. (2005). Survival model predictive accuracy and ROC curves. Biometrics, 61(1), 92–105. https://doi.org/10.1111/j.0006-341X.2005.030814.x
Henrysson, S. (1963). Correction of item–total correlations in item analysis. Psychometrics/ Psychometrika, 28(2), 211–218. https://doi.org/10.1007/BF02289618
Higham, P. A., & Higham, D. P. (2019). New improved gamma: Enhancing the accuracy of Goodman-Kruskal's gamma using ROC curves. Behavior Research Methods, 51(1), 108–125. https://doi.org/10.3758/s13428-018-1125-5
Higham, P. A., Zawadzka, K., & Hanczakowski, M. (2016). Internal mapping and its impact on measures of absolute and relative metacognitive accuracy. In J. Dunlosky & S. K. Tauber (Eds.), The Oxford handbook of metamemory. Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199336746.013.15
Hryniewicz, O. (2006). Goodman-Kruskal γ measure of dependence for fuzzy ordered categorical data. Computational Statistics & Data Analysis, 51(1), 323–334. https://doi.org/10.1016/j.csda.2006.04.014
IBM (2017). IBM SPSS Statistics 25 Algorithms. IBM.
Jonckheere, A. R. (1954). A distribution-free k–sample test against ordered alternatives. Biometrics/ Biometrika, 41(1–2), 133–145. http://doi.org/10.1093/biomet/41.1-2.133
Kendall, M. G. (1938). A new measure of rank correlation. Biometrics/ Biometrika, 30(1/2), 81–93. http://doi.org/10.2307/2332226
Kendall, M. G. (1948). Rank correlation methods (1st ed.). Charles Griffin & Co. Ltd.
Kendall, M. G. (1949). Rank and product–moment correlation. Biometrics/ Biometrika, 36(1/2), 177–193. https://doi.org/10.2307/2332540
Kim, J.-O. (1971). Predictive measures of ordinal association. American Journal of Sociology, 76(5), 891–907. https://doi.org/10.1086/225004
Kreiner, S., & Christensen, K. B. (2009). Item screening in graphical loglinear Rash models. Psychometrics/ Psychometrika, 76(2), 228–256. https://doi.org/10.1007/s11336-011-9203-y
Kruskal, W. H., & Wallis, W. A. (1952). Use of ranks on one-criterion variance analysis. Journal of the American Statistical Association, 47(260), 583–621. http://doi.org/10.2307/2280779
Kuder, G. F., & Richardson, M. W. (1937). The theory of the estimation of test reliability. Psychometrics/ Psychometrika, 2(3), 151–160. http://doi.org/10.1007/BF02288391
Kvålseth, T. O. (2017). An alternative measure of ordinal association as a value-validity correction of the Goodman–Kruskal gamma. Communications in Statistics - Theory and Methods, 46(21), 10582–10593. https://doi.org/10.1080/03610926.2016.1239114
Livingston, S. A., & Dorans, N. J. (2004). A graphical approach to item analysis (Research Report No. RR-04-10). Educational Testing Service. http://doi.org/10.1002/j.2333-8504.2004.tb01937.x
Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Addison–Wesley Publishing Company.
Love, T. E. (1997). Distractor selection ratios. Psychometrics/ Psychometrika, 62(1), 51–62. https://doi.org/10.1007/BF02294780
Mann, H. B. (1945). Nonparametric tests against trend. Econometrics/ Econometrica, 13(3), 245–259. https://doi.org/10.2307/1907187
Mann, H. B., & Whitney, D. R. (1947). On a test of whether one of two random variables is stochastically larger than the other. Annals of Mathematical Statistics, 18(1), 50–60. http://doi.org/10.1214/aoms/1177730491
Martin, W. S. (1973). The effects of scaling on the correlation coefficient: A test of validity. Journal of Marketing Research, 10(3), 316–318. http://doi.org/10.2307/3149702
Martin, W. S. (1978). Effects of scaling on the correlation coefficient: Additional considerations. Journal of Marketing Research, 15(2), 304–308. https://doi.org/10.1177/002224377801500219
Masson, M. E. J., & Rotello, C. M. (2009). Sources of bias in the Goodman–Kruskal gamma coefficient measure of association: Implications for studies of metacognitive processes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(2), 509–527. https://doi.org/10.1037/a0014876
McDonald, R. P. (1985). Factor analysis and related methods. Lawrence Erlbaum Associates.
Meade, A. W. (2010). Restriction of range. In N. J. Salkind (Ed.), Encyclopedia of research design (pp. 1278–1280). SAGE Publications, Inc. http://doi.org/10.4135/9781412961288.n309
Metsämuuronen, J. (2016). Item–total correlation as the cause for the underestimation of the alpha estimate for the reliability of the scale. GJRA - Global Journal for Research Analysis, 5(1), 471–477.
Metsämuuronen, J. (2017). Essentials of research methods in human sciences. SAGE Publications, Inc.
Metsämuuronen, J. (2020a). Somers’ D as an Alternative for the Item–Test and Item–Rest Correlation Coefficients in the Educational Measurement Settings. International Journal of Educational Measurement, 6(1), 207–221. https://doi.org/10.12973/ijem.6.1.207
Metsämuuronen, J. (2020b). Dimension-corrected Somers’ D for the item analysis settings. International Journal of Educational Methodology, 6(2), 297–317. https://doi.org/10.12973/ijem.6.2.297
Metsämuuronen, J. (2021). Directional nature of Goodman–Kruskal gamma and some consequences—Identity of Goodman–Kruskal gamma and Somers delta, and their connection to Jonckheere–Terpstra test statistic. ResearchGate. http://doi.org/10.13140/RG.2.2.19404.44163
Metsämuuronen, J., & Ukkola, A. (2019). Alkumittauksen menetelmällisiä ratkaisuja [Methodological solutions of zero level assessment]. Finnish Education Evaluation Centre.
Moses, T. (2017). A review of developments and applications in item analysis. In R. Bennett, & M. von Davier (Eds.), Advancing human assessment. The methodological, psychological and policy contributions of ETS (pp. 19–46). Springer Open. http://doi.org/10.1007/978-3-319-58689-2_2
Newson, R. (2002). Parameters behind “nonparametric” statistics: Kendall’s tau, Somers’ D and median differences. The Stata Journal, 2(1), 45–64. https://doi.org/10.1177/1536867X0200200103
Newson, R. (2006). Confidence intervals for rank statistics: Somers’ D and extensions. The Stata Journal, 6(3), 309–334. https://doi.org/10.1177/1536867X0600600302
Newson, R. (2008). Identity of Somers’ D and the rank biserial correlation coefficient. https://www.rogernewsonresources.org.uk/miscdocs/ranksum1.pdf
Nielsen, J. B., Kyvsgaard, J. N., Sildorf, S. M., Kreiner, S., & Svensson, J. (2017). Item analysis using Rasch models confirms that the Danish versions of the DISABKIDS® chronic-generic and diabetes-specific modules are valid and reliable. Health Qual Life Outcomes 15(1), article 44, 1–10. https://doi.org/10.1186/s12955-017-0618-8
Nielsen, T., & Santiago, P. H. R. (2020). Using graphical loglinear Rasch models to investigate the construct validity of Perceived Stress Scale. In M. S. Khine (Ed.), Rasch Measurement: Applications in Quantitative Educational Research (pp. 261–281). Springer Nature. https://doi.org/10.1007/978-981-15-1800-3_14
Olsson, U. (1980). Measuring correlation in ordered two-way contingency tables. Journal of Marketing Research, 17(3), 391–394. https://doi.org/10.1177/002224378001700315
Pearson, K. (1896). VII. Mathematical contributions to the theory of evolution.- III. Regression, heredity and panmixia. Philosophical Transactions of the Royal Society A, 187, 253–318. https://doi.org/10.1098/rsta.1896.0007
Pearson, K. (1903). I. Mathematical contributions to the theory of evolution. —XI. On the influence of natural selection on the variability and correlation of organs. Philosophical Transactions of the Royal Society A. Mathematical, Physical and Engineering Sciences, 200(321–330), 1–66. https://doi.org/10.1098/rsta.1903.0001
Raykov, T., & Marcoulides, G. A. (2013). Meta-analysis of reliability coefficients using latent variable modeling. Structural Equation Modeling, 20(2), 338‒353. http://doi.org/10.1080/10705511.2013.769396
Rousson, V. (2007). The gamma coefficient revisited. Statistics & Probability Letters 77(17), 1696–1704. https://doi.org/10.1016/j.spl.2007.04.009
Sackett, P. R., Lievens, F., Berry, C. M., & Landers, R. N. (2007). A cautionary note on the effect of range restriction on predictor intercorrelations. Journal of Applied Psychology, 92(2), 538–544. http://doi.org/10.1037/0021-9010.92.2.538
Sackett, P. R., & Yang, H. (2000). Correction for range restriction: An expanded typology. Journal of Applied Psychology, 85(1), 112–118. https://doi.org/10.1037/0021-9010.85.1.112
Sen, P. K. (1963). On the estimation of relative potency in dilution(-direct) assays by distribution-free methods. Biometrics, 19(4), 532–552. https://doi.org/10.2307%2F2527532
Shafina, A (2021). The impact of birth-order, sib-size, siblings’ sex composition on educational attainment in the Maldives. The Universal Academic Research Journal, 3(2), 87–100.
Sheskin, D. J. (2011). Handbook of parametric and nonparametric statistical procedures (5th ed.). Chapman & Hall/CRC.
Siegel, S., & Castellan, N. J., Jr. (1988). Nonparametric statistics for the behavioural sciences (2nd ed.). McGraw-Hill.
Sirkin, M. R. (2006). Statistics of the social science (3rd ed.). SAGE Publications, Inc.
Somers, R. H. (1962). A new asymmetric measure of association for ordinal variables. American Sociological Review, 27(6), 799–811. http://doi.org/10.2307/2090408
Somers, R. H. (1980). Simple approximations to null sampling variances. Goodman and Kruskal’s gamma, Kendall’s tau and Somers dyx. Sociological Methods & Research, 9(1), 115–126. https://doi.org/10.1177/004912418000900107
Terpstra, T. J. (1952). The asymptotic normality and consistency of Kendall’s test against trend, when ties are present in one ranking. Investigations of the mathematics/ Indagationes Mathematicae, 14(3), 327–333. http://doi.org/10.1016/S1385-7258(52)50043-X
Theil, H. (1950). A rank-invariant method of linear and polynomial regression analysis I, II, III. In Proceedings of the Section of Sciences - Koninklijke Nederlandsche Akademie van Wetenschappen [Royal Netherlands Academy of Sciences] (Series A. Mathematical Sciences, pp. 386–392, 521–525, 1397–1412). North-Holland.
Trizano-Hermosilla, I., & Alvarado, J. M. (2016). Best alternatives to Cronbach's alpha reliability in realistic conditions: Congeneric and asymmetrical measurements. Frontiers in Psychology, 7, 1-8. https://doi.org/10.3389/fpsyg.2016.00769
Van der Ark, L. A., & Van Aert, R. C. M. (2015). Comparing confidence intervals for Goodman and Kruskal's gamma coefficient. Journal of Statistical Computation and Simulation, 85(12), 2491–2505. https://doi.org/10.1080/00949655.2014.932791
Wholey, J., S., Hatry, H., P., & Newcomer, K. E. (Eds.) (2015). Handbook of practical program evaluation (4th ed.). Jossey-Bass.
Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1(6), 80–83. http://doi.org/10.2307/3001968
Wilson, T. P. (1974). Measures of association for bivariate ordinal hypotheses. In H. M. Blalock (Ed.), Measurement in the social sciences (pp. 327–342). Aldine.
Zaionts, C. (2020). Polychoric correlation using solver. Real Statistics Using Excel. http://www.real-statistics.com/correlation/polychoric-correlation/polychoric-correlation-using-solver/