IES/NSF Pipeline-of-Evidence Protocol as Explanation for Successes and Failures of Gates Foundation Funded Initiatives
This study was designed to investigate the applicability of the IES/NSF pipeline-of-evidence protocol in ascertaining why two notable educational init.
- Pub. date: August 15, 2020
- Pages: 571-578
- 214 Downloads
- 613 Views
- 0 Citations
This study was designed to investigate the applicability of the IES/NSF pipeline-of-evidence protocol in ascertaining why two notable educational initiatives spearheaded and financially supported by the Bill and Melinda Gates Foundation achieved or not the goal of improved academic outcomes for K-12 public school students. Our interest was not whether there is a sufficient body of high quality research evidence to support the two initiatives but whether the research considered by the Gates Foundation established the likelihood that the initiatives would be successful and worth the decision to dedicate substantial funding, time, and effort required for each versus the many competing programs seeking sponsorship. We found that in the case of Intensive Partnerships for Effective Teaching, efficacy and effectiveness research were absent and the Gates foundation discontinued grant support because the initiative had not achieved the goal of improved high school graduation and college attendance among low-income minority students. In the case of Early College High School, we found empirical evidence was manifest at all but the effectiveness stage of the pipeline and the initiative continued to receive funding. Our findings support the importance of widening the net of methodologies that constitute a framework for elements needed to make predictions of effectiveness for any given intervention before investing in scale-up initiatives and the need for private foundations to be transparent in their decision-making process to enable others to scrutinize the research that informs the design of initiatives.
Keywords: Evidence-based practices, K-12 education initiatives, early college high schools, teacher evaluation, dual enrollment.
References
Aud, S., Fox, M. A., & KewalRamani, A. (2010). Status and trends in the education of racial and ethnic groups. U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. https://nces.ed.gov/programs/raceindicators/.
Berger, A., Tirl-Bicakci, L., Garet, M., Song, M., Knudson, J., Haxton, C., Zeiser, K., Hoshen, G., Ford, J., Stephan, J., Keating, K., & Cassidy, L. (2013). Early college, early success: Early College High School initiative impact study. American Institutes for Research. https://www.air.org/sites/default/files/downloads/report/ ECHSI_Impact_Study_Report_Final1_0.pdf
Bill and Melinda Gates Foundation. (2019). K-12 education. https://www.gatesfoundation.org/What-We-Do/US-Program/K-12-Education#AREASOFFOCUS
Carnevale, A. P., Rose, S. J., & Cheah, B. (2011). The college payoff: Education, occupations, lifetime earnings. Georgetown University, Center on Education and the Workforce. https://cew.georgetown.edu/cew-reports/the-college-payoff/
Cooper, H. (2010). Research synthesis and meta-analysis: A step-by-step approach (4th ed.). Sage.
Council for Exceptional Children. (2014). Council for Exceptional Children: Standards for evidence-based practices in special education. https://www.cec.sped.org/~/media/Images/Standards/CEC%20EBP%20Standards%20cover/ CECs%20Evidence%20Based%20Practice%20Standards.pdf.
Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Sage
Dee, T. W., & Wyckoff, J. (2015). Incentives, selection, and teacher performance: Evidence from IMPACT. Journal of Policy Analysis and Management, 34(2), 267-297. https://doi.org/10.1002/pam.21818
Fryer, R. G. (2011). Teacher incentives and student achievement: Evidence from New York City Public Schools. National Bureau of Economic Research, Working Paper 16850. http://www.nber.org/papers/w16850.
Gersten, R., Fuchs, L. S., Compton, D., Coyne, M., Greenwood, C., & Innocenti, M. S. (2005). Quality indicators for group experimental and quasi-experimental research in special education. Exceptional Children, 71(2), 149-164. https://doi.org/10.1177/001440290507100202
Hanushek, E. A. (1971). Teacher characteristics and gains in student achievement: Estimation using micro data. American Economic Review, 61(2), 280-288.
Hughes, K. L., Karp, M. M., Fermin, B. J., & Bailey, T. R. (2006). Pathways to college access and success. Journal for Vocational Special Needs Education, 28(3), 17-46. https://eric.ed.gov/?id=EJ854373.
Institute of Education Sciences and National Science Foundation. (2013). Common guidelines for education research and development. Authors. https://www.nsf.gov/publications/pub_summ.jsp?ods_key=nsf13126
Jiang, J. Y., Sporte, S. E., & Luppescu, S. (2015). Teacher perspectives on evaluation reform: Chicago’s REACH students. Educational Researcher, 44(2), 105-116.
Kane, T. J., & Staiger, D. O. (2012). Gathering feedback for teaching: Combining high-quality observations with student surveys and achievement gains. Bill and Melinda Gates Foundation. https://k12education.gatesfoundation.org/ resource/gathering-feedback-on-teaching-combining-high-quality-observations-with-student-surveys-and-achievement-gains-3/
Kane, T. J., Taylor, E. S., Tyler, J. H., & Wooten, A. L. (2010). Identifying effective classroom practices using student achievement data. National Bureau of Economic Research. https://www.nber.org/papers/w15803
Karp, M.M., Calcagno, J. C., Hughes, K. L., Jeong, D. W., & Bailey, T. R. (2007). The postsecondary achievement of participants in dual enrollment: An analysis of student outcomes in two states. National Research Center for Career and Technical Education, University of Minnesota. https://eric.ed.gov/?id=ED498661
Kazdin, A. E. (2011). Single-case research design: Methods for clinical and applied settings (2nd ed.). Oxford University Press.
Kleiner, B., & Lewis, L. (2005). Dual enrollment of high school students at postsecondary institutions: 2002-03. U. S. Department of Education, National Center for Education Statistics. https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2005008
Patton, M. Q. (2015). Qualitative research and evaluation methods (4th ed.). Sage.
Ravitch, S. M., & Carl, N. M. (2016). Qualitative research: Bridging the conceptual, theoretical, and methodological. Sage.
Rivkin, S., Hanushek, E., & Kain, J. (2005). Teachers, schools and academic achievement. Econometrica, 73(2), 417-458.
Saldana, J. (2016). The coding manual for qualitative researchers (3rd ed.). Sage.
Schirmer, B. R., Lockman, A. S., & Schirmer, T. N. (2016). Identifying evidence-based educational practices: Which research designs provide findings that can influence social change? Journal of Educational Research and Practice, 6(1), 33-42. https://doi.org/10.5590/jerap.2016.06.1.03
Springer, M. G., Pane, J. F., Le, V-N., McCaffrey, D. F., Burns, S. F., Hamilton, L. S., & Stecher, B. (2012). Team pay for performance: Experimental evidence from the Round Rock Pilot Project on team incentives. Educational Evaluation and Public Policy Analysis, 34(4), 367-390. https://doi.org/10.3102/0162373712439094
Stecher, B. M., Holtzman, D. J., Garet, M. S., Hamilton, L. S., Engberg, J., Steiner, E. D., Robyn, A., Baird, M. D., Gutierrez, I. A., Peet, E. D., De Los Reyes, I. B., Fronberg, K., Weinberger, G., Hunter, G. P., & Chambers, J. (2018). Improving teaching effectiveness: Final report. The Intensive Partners. American Institutes for Research. https://www.rand.org/pubs/research_reports/RR2242.html
Taylor, E. S., & Tyler, J. H. (2012). The effect of evaluation on teacher performance. American Economic Review, 102(7), 3628-3651. https://doi.org/10.1257/aer.102.7.3628
Tierney, W. B., Bailey, T. Constantine, J., Finkelstein, N., & Hurd, N. F. (2009). Helping students navigate the path to college: What high schools can do. U. S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. https://ies.ed.gov/ncee/wwc/PracticeGuide/11
U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse. (2017, February). Dual enrollment programs. https://ies.ed.gov/ncee/wwc/Docs/InterventionReports/wwc_dual_enrollment_022817.pdf
What Works Clearinghouse. (2017). Procedures and standards handbook version 4.0. https://ies.ed.gov/ncee/wwc/Handbooks