Although regression to the mean is pervasive in data analysis, educational researchers often misconstrue it as evidence of genuine change and mistaken.
- Pub. date: February 15, 2025
- Online Pub. date: January 31, 2025
- Pages: 17-25
- 115 Downloads
- 403 Views
- 0 Citations
Although regression to the mean is pervasive in data analysis, educational researchers often misconstrue it as evidence of genuine change and mistakenly attribute random changes to treatment effects. A statistical phenomenon where extreme values naturally move closer to the average after repeated treatment, regression to the mean is especially susceptible to misinterpretations in educational studies with pretest-posttest or longitudinal designs. In such studies, observed changes are frequently assumed to be the effects of treatment, even in cases where the changes are statistical artifacts. Using a hypothetical case and two real-world studies, this paper investigates the technical challenges that regression to the mean poses and introduces a hybrid Bayesian model that mitigates its effects more effectively than conventional approaches, such as multiple baseline adjustments and formulaic corrections. In particular, the hybrid Bayesian model relies on multiple baseline measurements to minimize distortions associated with regression to the mean during the pretest phase and leverages prior knowledge—such as standard deviations and population means—to refine post-test data adjustments. It follows that the model provides educational researchers with an innovative tool for accurately evaluating interventions and enhancing the effectiveness of various research-driven educational policies and practices.
Keywords: Bayesian regression, causal inferences, pretest–posttest designs, regression to the mean.
0
References
Asbury, C. A. (1974). Selected factors influencing over- and under-achievement in young school-age children. Review of Educational Research, 44(4), 409-428. https://doi.org/10.3102/00346543044004409
Cochrane, K. M., Williams, B. A., Fischer, J. A., Samson, K. L., Pei, L. X., & Karakochuk, C. D. (2020). Regression to the mean: A statistical phenomenon of worthy consideration in anemia research. Current Developments in Nutrition, 4(10), Article nzaa152. https://doi.org/10.1093/cdn/nzaa152
Drugowitsch, J. (2013). Variational Bayesian inference for linear and logistic regression. arXiv. https://doi.org/10.48550/ARXIV.1310.5438
Galton, F. (1886). Regression towards mediocrity in hereditary stature. The Journal of the Anthropological Institute of Great Britain and Ireland, 15, 246-263. https://doi.org/10.2307/2841583
Illenberger, N., Small, D. S., & Shaw, P. A. (2019). Regression to the Mean’s Impact on the Synthetic Control Method: Bias and Sensitivity Analysis (Version 1). arXiv. https://doi.org/10.48550/ARXIV.1909.04706
Isaac, S., & Michael, W. B. (1995). Handbook in research and evaluation: A collection of principles, methods, and strategies useful in the planning, design, and evaluation of studies in education and the behavioral sciences (3rd ed.). EdITS. https://psycnet.apa.org/record/1995-98981-000
Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80(4), 237-251. https://doi.org/10.1037/h0034747
Kazdin, A. E. (2020). Single-case research designs: Methods for clinical and applied settings (3rd ed.). Oxford University Press.
Kennedy, C. H. (2022). The nonconcurrent multiple-baseline design: It is what it is and not something else. Perspectives on Behavior Science, 45, 647-650. https://doi.org/10.1007/s40614-022-00343-0
Ledford, J. R., & Gast, D. L. (Eds.). (2024). Single case research methodology: Applications in special education and behavioral sciences (4th ed.). Routledge.
Linden, A. (2013). Assessing regression to the mean effects in health care initiatives. BMC Medical Research Methodology, 13, Article 119. https://doi.org/10.1186/1471-2288-13-119
Mara, T. A. (2019). Linear regression in the Bayesian framework (Version 1). arXiv. https://doi.org/10.48550/ARXIV.1908.03329
Marsden, E., & Torgerson, C. J. (2012). Single group, pre- and post-test research designs: Some methodological concerns. Oxford Review of Education, 38(5), 583-616. https://doi.org/10.1080/03054985.2012.731208
Morton, V., & Torgerson, D. J. (2005). Regression to the mean: Treatment effect without the intervention. Journal of Evaluation in Clinical Practice, 11(1), 59-65. https://doi.org/10.1111/j.1365-2753.2004.00505.x
Murphy, K. P. (2012). Machine learning: A probabilistic perspective. MIT Press.
Nesselroade, J. R., Stigler, S. M., & Baltes, P. B. (1980). Regression toward the mean and the study of change. Psychological Bulletin, 88(3), 622-637. https://doi.org/10.1037/0033-2909.88.3.622
Nielsen, T., Karpatschof, B., & Kreiner, S. (2007). Regression to the mean effect: When to be concerned and how to correct for it. Nordic Psychology, 59(3), 231-250. https://doi.org/10.1027/1901-2276.59.3.231
Pinker, S. (2021). Rationality: What it is, why it seems scarce, why it matters. Viking.
Slocum, T. A., Joslyn, P. R., Nichols, B., & Pinkelman, S. E. (2022). Revisiting an analysis of threats to internal validity in multiple baseline designs. Perspectives on Behavior Science, 45, 681-694. https://doi.org/10.1007/s40614-022-00351-0
Smith, G. (1997). Do statistics test scores regress toward the mean? CHANCE, 10(4), 42-45. https://doi.org/10.1080/09332480.1997.10542064
Smith, G., & Smith, J. (2005). Regression to the mean in average test scores. Educational Assessment, 10(4), 377-399. https://doi.org/10.1207/s15326977ea1004_4
Stigler, S. M. (1997). Regression towards the mean, historically considered. Statistical Methods in Medical Research, 6(2), 103-114. https://doi.org/10.1177/096228029700600202
Streiner, D. V. L. (2001). Regression toward the mean: Its etiology, diagnosis, and treatment. The Canadian Journal of Psychiatry, 46(1), 72-76. https://doi.org/10.1177/070674370104600111
Thorndike, R. L. (1942). Regression fallacies in the matched groups experiment. Psychometrika, 7(2), 85–102. https://doi.org/10.1007/bf02288069
Yu, R., & Chen, L. (2015). The need to control for regression to the mean in social psychology studies. Frontiers in Psychology, 5, Article 01574. https://doi.org/10.3389/fpsyg.2014.01574