Presenting the Meta-Performance Test, a Metacognitive Battery based on Performance
Marcio Alexander Castillo-Diaz , Cristiano Mauro Assis Gomes
The self-report and think-aloud approaches are the two dominant methodologies to measure metacognition. This is problematic, since they generate respo.
- Pub. date: May 15, 2021
- Pages: 289-303
- 1487 Downloads
- 2790 Views
- 2 Citations
The self-report and think-aloud approaches are the two dominant methodologies to measure metacognition. This is problematic, since they generate respondent and confirmation biases, respectively. The Meta-Performance Test is an innovative battery, which evaluates metacognition based on the respondent's performance, mitigating the aforementioned biases. The Meta-Performance Test consists of two tests, the Meta-text, which evaluates metacognition in the domain of reading comprehension, and Meta-number, in the domain of arithmetic expressions solving. The main focus of this article is to present the development of the battery, in terms of its conceptual basis, development strategies and structure. Evidence of its content validity is also presented, through the evaluation of three experts in metacognition, two experts in Spanish language, two experts in mathematics and five students who represent the target population. The results of the judges' evaluations attested to the Meta-Performance Test content validity, and the target population declared that both the battery understanding and taking are adequate. Contributions and future research perspectives of the Meta-Performance Test in the field of metacognition are discussed.
Keywords: Metacognition, assessment, content validity, performance tests.
References
Abdelrahman, R. M. (2020). Metacognitive awareness and academic motivation and their impact on academic achievement of Ajman University students. Heliyon, 6(9), 1–8. https://doi.org/10.1016/j.heliyon.2020.e04192
Abernethy, M. (2015). Self-reports and observer reports as data generation methods: An assessment of issues of both methods. Universal Journal of Psychology, 3(1), 22–27. https://doi.org/10.13189/ujp.2015.030104
Akturk, A., & Sahin, I. (2011). Literature review on metacognition and its measurement. Procedia Social and Behavioral Sciences, 15, 3731–3736. https://doi.org/10.1016/j.sbspro.2011.04.364
Alias, M., & Sulaiman, N. (2017). Development of metacognition in higher education: Concepts and strategies. In E. Railean, A. Elçi, & A. Elçi (Eds.), Metacognition and successful learning strategies in higher education. Advances in higher education and professional development (AHEPD) book series (pp. 22-42). Information Science Reference.
Alves, F. A., Flores, R. P., Gomes, C. M. A., & Golino, H. F. (2012). Preditores do rendimento escolar: inteligência geral e crenças sobre ensino-aprendizagem [Predictors of school performance: general intelligence and beliefs about teaching-learning]. E-PSI Journal/ Revista E-PSI, 1, 97-117.
American Educational Research Association, American Psychological Association & National Council on Measurement in Education-USA (2014). Standards for educational and psychological testing. AERA Publications.
André, A. M., Gomes, C. M. A., & Loureiro, C. M. V. (2016). Escalas Nordoff Robbins: uma revisão bibliográfica [Nordoff Robbins Scales: a literature review]. Percepta, 3(2), 117-131. https://doi.org/10.34018/2318-891X.3(2)117-131
André, A. M., Gomes, C. M. A., & Loureiro, C. M. V. (2017). Equivalência de itens, semântica e operacional da versão brasileira da escala Nordoff Robbins de comunicabilidade musical [Equivalence of items, semantics and operations of the Brazilian version of the Nordoff Robbins musical communication scale]. OPUS, 23(2), 197-215. https://doi.org/10.20504/opus2017b2309
André, A. M. B., Gomes, C. M. A., & Loureiro, C. M. V. (2020a). Confiabilidade interexaminadores da Escala de relação criança-terapeuta na experiência musical coativa para validação no contexto brasileiro [Inter-rater reliability of the child-therapist relationship scale in coactive musical experience for validation in the Brazilian context]. Hodie, 20(1), 1-18. https://doi.org/10.5216/mh.v20.64243
André, A. M. B., Gomes, C. M. A., & Loureiro, C. M. V. (2020b). Equivalência de itens, semântica e operacional da “Escala de Musicabilidade: Formas de Atividade, Estágios e Qualidades de Engajamento” [Equivalence of items, semantics and operations of the “Musicality Scale: Forms of Activity, Stages and Qualities of Engagement”]. Orfeu, 5(2), 1–22. https://doi.org/10.5965/2525530405022020e0010
Baker, L. (2016). The Development of Metacognitive Knowledge and Control of Comprehension. Contributors and Consequences. In K. Mokhtari (Ed.), Improving Reading Comprehension through Metacognitive Reading Strategies Instruction (pp. 01-31). Rowman & Littlefield.
Baker, L., Millman, Z. B., & Singer Trakhman, L. M. (2020). How the construct of metacognition has contributed to translational research in education, mental health, and beyond. Translational Issues in Psychological Science, 6(1), 1-7. https://doi.org/10.1037/tps0000225
Blummer, B., & Kenton, J. (2014). Improving Student Information Search: A metacognitive approach. Elsevier.
Cai, Y., King, R. B., Law, W., & McInerney, D. M. (2019). Which comes first? Modeling the relationships among future goals, metacognitive strategies and academic achievement using multilevel cross-lagged SEM. Learning and Individual Differences, 74, 101750. https://doi.org/10.1016/j.lindif.2019.06.004
Cardoso, C. O., Seabra, A. G., Gomes, C. M. A., & Fonseca, R. P. (2019). Program for the neuropsychological stimulation of cognition in students: impact, effectiveness, and transfer effect on student cognitive performance. Frontiers in Psychology, 10, 1-16. https://doi.org/10.3389/fpsyg.2019.01784
Castillo, M. (2018). Monitoring and intelligence as predictors of general and specific academic performance in higher education [Master's thesis, Federal University of Minas Gerais, Brasil]. UFMG Digital Archive. https://repositorio.ufmg.br/handle/1843/BUOS-B3PJKN
Costa, B. C. G., Gomes, C. M. A., & Fleith, D. S. (2017). Validade da Escala de cognições acadêmicas autorreferentes: autoconceito, autoeficácia, autoestima e valor [Validity of the Self-Referential academic cognition scale: self-concept, self-efficacy, self-esteem and value]. Psychological Assessment/ Avaliação Psicológica, 16(1), 87-97. https://doi.org/10.15689/ap.2017.1601.10
Craig, K., Hale, D., Grainger, C., & Stewart, M. E. (2020). Evaluating metacognitive self-reports: systematic reviews of the value of self-report in metacognitive research. Metacognition and Learning, 15(2), 155-213. https://doi.org/10.1007/s11409-020-09222-y
Cromley, J. G., & Kunze, A. J. (2020). Metacognition in education: Translational research. Translational Issues in Psychological Science, 6(1), 15-20. https://doi.org/10.1037/tps0000218
Das-Smaal, E. A. (1990). Biases in categorization. Advances in Psychology, 68, 349–387. https://doi.org/10.1016/S0166-4115(08)61332-1
Dent, A. L., & Koenka, A. C. (2016). The relation between self-regulated learning and academic achievement across childhood and adolescence: A meta-analysis. Educational Psychology Review, 28(3), 425–474. https://doi.org/10.1007/s10648-015-9320-8
Desoete, A., Roeyers, H., & Buysse, A. (2001). Metacognition and mathematical problem solving in grade 3. Journal of Learning Disabilities, 34(5), 435–447. https://doi.org/10.1177/002221940103400505
Dinsmore, D. L., Alexander, P. A., & Loughlin, S. M. (2008). Focusing the conceptual lens on metacognition, self-regulation, and self-regulated learning. Educational Psychology Review, 20(4), 391–409. https://doi.org/10.1007/s10648-008-9083-6
Flavell, J. H. (1979). Metacognition and cognitive monitoring: a new area of cognitive-developmental inquiry. American Psychologist, 34(10), 906–911. https://doi.org/10.1037/0003-066X.34.10.906
Gascoine, L., Higgins, S., & Wall, K. (2017). The assessment of metacognition in children aged 4–16 years: A systematic review. Review of Education, 5(1), 3–57. https://doi.org/10.1002/rev3.3077
Georgiou, G., Li, J., & Das, J. P. (2017). Tower of London: What level of planning does it measure? Psychological Studies, 62(3), 261-267. https://doi.org/10.1007/s12646-017-0416-8
Golino, H. F., & Gomes, C. M. A. (2011). Preliminary internal validity evidences of two Brazilian Metacognitive Tests. International Journal of Testing, 26(1), 11-12. International Test Commission. https://www.intestcom.org/files/ti26.pdf
Golino, H. F., & Gomes, C.M.A. (2014). Psychology data from the “BAFACALO project: The Brazilian Intelligence Battery based on two state-of-the-art models – Carroll’s Model and the CHC model”. Journal of Open Psychology Data, 2(1), e6. https://doi.org/10.5334/jopd.af
Gomes, C. M. A. (2005). Uma análise dos fatores cognitivos mensurados pelo Exame Nacional do Ensino Médio (ENEM) [An analysis of the cognitive factors measured by the National High School Exam (ENEM)] [Doctoral Thesis, Federal University of Minas Gerais]. UFMG Digital Archive. http://hdl.handle.net/1843/FAEC-85RJNN
Gomes, C. M. A. (2007a). Apostando no desenvolvimento da inteligência; em busca de um novo currículo educacional para o desenvolvimento do pensamento humano [Betting on the development of intelligence; in search of a new educational curriculum for the development of human thought]. Lamparina.
Gomes, C. M. A. (2007b). Softwares educacionais podem ser instrumentos psicológicos [Educational software can be psychological tools]. School and Educational Psychology/ Psicologia Escolar e Educacional, 11(2), 391-401. https://doi.org/10.1590/S1413-85572007000200016
Gomes, C. M. A. (2010a). Avaliando a avaliação escolar: notas escolares e inteligência fluida [Evaluating school evaluation: school grades and fluid intelligence]. Psychology in Study/ Psicologia em Estudo, 15(4), 841-849. https://doi.org/10.1590/S1413-73722010000400020
Gomes, C. M. A. (2010b). Estrutura fatorial da Bateria de Fatores Cognitivos de Alta-Ordem (BaFaCalo) [Factorial Structure of the High-Order Cognitive Factors Battery (BaFaCalo)]. Psychological Assessment/ Avaliação Psicológica, 9(3), 449-459.
Gomes, C. M. A. (2011a). Abordagem profunda e abordagem superficial à aprendizagem: diferentes perspectivas do rendimento escolar [Deep approach and superficial approach to learning: different perspectives of school performance]. Psychology: Reflection and Criticism/ Psicologia: Reflexão e Crítica, 24(3), 438-447. https://doi.org/10.1590/S0102-79722011000300004
Gomes, C. M. A. (2011b). Validade do conjunto de testes da habilidade de memória de curto-prazo (CTMC) [Validity of the short-term memory ability test set]. Psychology Studies (Natal)/ Estudos de Psicologia (Natal), 16(3), 235-242. https://doi.org/10.1590/S1413-294X2011000300005
Gomes, C. M. A. (2012). Validade de construto do conjunto de testes de inteligência cristalizada (CTIC) da bateria de fatores cognitivos de alta-ordem (BaFaCAlO) [Construct validity of the set of crystallized intelligence tests (CTIC) of the battery of high-order cognitive factors (BaFaCAlO)]. General: Interinstitutional Journal of Psychology/ Gerais: Revista Interinstitucional de Psicologia, 5(2), 294-316.
Gomes, C. M. A. (2013). A construção de uma medida em abordagens de aprendizagem [The construction of a measure of learning approaches]. Psico (PUCRS-Online), 44(2), 193-203.
Gomes, C. M. A. (2020). Programa de enriquecimento istrumental: evidências de eficácia para intervenção cognitiva [Instrumental enrichment program: evidence of effectiveness for cognitive intervention]. In M. Mansur-Alves & J. B. Lopes-Silva (Eds.), Intervenção cognitiva: dos conceitos às práticas baseadas em evidências para diferentes aplicações [Cognitive intervention: from concepts to evidence-based practices for different applications] (pp. 621-639). T.Ser.
Gomes, C. M. A., Amantes, A., & Jelihovschi, E.G. (2020a). Applying the regression tree method to predict students’ science achievement. Trends in Psychology, 28(1), 99-117. https://doi.org/10.9788/s43076-019-00002-5
Gomes, C. M. A., Araujo, J., & Jelihovschi, E. G. (2020b). Approaches to learning in the non-academic context: construct validity of Learning Approaches Test in Video Game (LAT-Video Game). International Journal of Development Research, 10(11), 41842-41849. https://doi.org/10.37118/ijdr.20350.11.2020
Gomes, C. M. A., & Borges, O. N. (2007). Validação do modelo de inteligência de Carroll em uma amostra brasileira [Validation of the Carroll intelligence model in a Brazilian sample]. Psychological Assessment/ Avaliação Psicológica, 6(2), 167-179.
Gomes, C. M. A., & Borges, O. N. (2008a). Avaliação da validade e fidedignidade do instrumento crenças de estudantes sobre ensino-aprendizagem (CrEA) [Evaluation of the validity and reliability of the instrument students' beliefs about teaching-learning]. Science & Cognition/ Ciências & Cognição, 13(3), 37-50.
Gomes, C. M. A., & Borges, O. (2008b). Qualidades psicométricas de um conjunto de 45 testes cognitivos [Psychometric qualities of a set of 45 cognitive tests]. Fractal: Journal of Psychology/ Fractal: Revista de Psicologia, 20(1), 195-207. https://doi.org/10.1590/S1984-02922008000100019
Gomes, C. M. A. & Borges, O. N. (2009a). O ENEM é uma avaliação educacional construtivista? Um estudo de validade de construto [Is ENEM a constructivist educational assessment? A construct validity study]. Educational Evaluation Studies/ Estudos em Avaliação Educacional, 20(42), 73-88. https://doi.org/10.18222/eae204220092060
Gomes, C. M. A., & Borges, O. (2009c). Qualidades psicométricas do conjunto de testes de inteligência fluida [Psychometric qualities of the fluid intelligence test suite]. Psychological Assessment/ Avaliação Psicológica, 8(1), 17-32.
Gomes, C. M. A., Fleith, D. S., Marinho-Araujo, C. M., & Rabelo, M. L. (2020c). Predictors of students’ mathematics achievement in secondary education. Psychology: Theory and Research/ Psicologia: Teoria e Pesquisa, 36(1), e3638. https://doi.org/10.1590/0102.3772e3638
Gomes, C. M. A., & Gjikuria, E. (2018). Structural Validity of the School Aspirations Questionnaire (SAQ). Psychology: Theory and Research/ Psicologia: Teoria e Pesquisa, 34(1), 1-11. https://doi.org/10.1590/0102.3772e3438
Gomes, C. M. A., & Golino, H. F. (2012a). O que a inteligência prediz: diferenças individuais ou diferenças no desenvolvimento acadêmico? [What does intelligence predict: individual differences or differences in academic development?]. Psychology: Theory and Practice/ Psicologia: Teoria e Prática, 14(1), 126-139.
Gomes, C. M. A., & Golino, H. F. (2012b). Validade incremental da Escala de Abordagens de Aprendizagem (EABAP) [Incremental validity of the Learning Approach Scale (EABAP)]. Psychology: Reflection and Criticism/ Psicologia: Reflexão e Crítica, 25(4), 400-410. https://doi.org/10.1590/S0102-79722012000400001
Gomes, C., & Golino, H. (2014). Self-reports on students' learning processes are academic metacognitive knowledge. Psychology: Reflection and Criticism/ Psicologia: Reflexão e Crítica, 27(3), 472-480. https://doi.org/10.1590/1678-7153.201427307
Gomes, C., Golino, H., & Menezes, I. (2014a). Predicting school achievement rather than intelligence: Does metacognition matter? Psychology, 5(9), 1095–1110. https://doi.org/10.4236/psych.2014.59122
Gomes, C. M. A., Golino, H. F., Pinheiro, C. A. R., Miranda, G. R., & Soares, J. M. T. (2011). Validação da Escala de Abordagens de Aprendizagem (EABAP) em uma amostra Brasileira [Validation of the Learning Approaches Scale (EABAP) in a Brazilian sample]. Psychology: Reflection and Criticism/ Psicologia: Reflexão e Crítica, 24(1), 19-27. https://doi.org/10.1590/S0102-79722011000100004
Gomes, C. M. A., Golino, H. F., Santos, M. T., & Ferreira, M. G. (2014b). Formal-Logic Development Program: Effects on Fluid Intelligence and on Inductive Reasoning Stages. British Journal of Education, Society & Behavioural Science, 4(9), 1234-1248. https://doi.org/10.9734/BJESBS/2014/10757
Gomes, C. M. A., & Jelihovschi, E. (2019). Presenting the regression tree method and its application in a large-scale educational dataset. International Journal of Research & Method in Education, 43(2), 201-221. https://doi.org/10.1080/1743727X.2019.1654992
Gomes, C. M. A., Lemos, G. C., & Jelihovschi, E. G. (2020d). Comparing the predictive power of the CART and CTREE algorithms. Psychological Assessment/ Avaliação Psicológica, 19(1), 87-96. https://doi.org/10.15689/ap.2020.1901.17737.10
Gomes, C. M. A., Linhares, I. S., Jelihovschi, E. G., & Rodrigues, M. N. S. (2021). Introducing rationality and content validity of SLAT-Thinking. International Journal of Development Research, 11(1), 43264-43272. https://doi.org/10.37118/ijdr.20586.01.2021
Greene, J. A., Deekens, V. M., Copeland, D. Z., & Yu, S. (2018). Capturing and modeling self-regulated learning using think-aloud protocols. In D. H. Schunk & J. A. Greene (Eds.), Educational psychology handbook series. Handbook of self-regulation of learning and performance (pp. 323–337). Routledge/Taylor & Francis Group.
Jemstedt, A., Schwartz, B. L., & Jönsson, F. U. (2017). Ease-of-learning judgments are based on both processing fluency and beliefs. Memory, 26(6), 807-815. https://doi.org/10.1080/09658211.2017.1410849
Kofsky, E., Friedman, S., & Wallner-Allen, K. (2014). What do they really measure? A comparative analysis of planning tasks. In Friedman, S., & Kofsky, E. (Eds.), The Developmental Psychology of Planning: Why, How, and When Do We Plan? (pp. 127-156). Psychology Press.
Li, J., Zhang, B., Du, H., Zhu, Z., & Li, Y. M. (2015). Metacognitive planning: Development and validation of an online measure. Psychological Assessment, 27(1), 260-271. https://doi.org/10.1037/pas0000019
Markman, E. M. (1977). Realizing that You Don’t Understand: A Preliminary Investigation. Child Development, 48(3), 986-992.
Markman, E. M. (1979). Realizing that You Don’t Understand: Elementary School Children’s Awareness of Inconsistencies. Child Development, 50(3), 643-655. https://doi.org/10.2307/1128929
Martins, A. A., Gomes, C. M. A., Alves, A. F., & Almeida, L. S. (2018). The structure of intelligence in childhood: age and socio-familiar impact on cognitive differentiation. Psychological Reports, 121(1), 79-92. https://doi.org/10.1177/0033294117723019
Mihalca, L., Mengelkamp, C., & Schnotz, W. (2017). Accuracy of metacognitive judgments as a moderator of learner control effectiveness in problem-solving tasks. Metacognition and Learning, 12(3), 357–379. https://doi.org/10.1007/s11409-017-9173-2
Morales, J., Lau, H., & Fleming, S. M. (2018). Domain-General and Domain-Specific Patterns of Activity Supporting Metacognition in Human Prefrontal Cortex. The Journal of Neuroscience, 38(14), 3534–3546. https://doi.org/10.1523/jneurosci.2360-17.2018
Muniz, M., Gomes, C. M. A., & Pasian, S. R. (2016). Factor structure of Raven's Coloured Progressive Matrices. Psico-USF, 21(2), 259-272. https://doi.org/10.1590/1413-82712016210204
Nelson, T., & Narens, L. (1996). Why investigate Metacognition? In J. Metcalfe & A. P. Shimamura (Ed.), Metacognition. Knowing about knowing (pp. 1-27). MIT Press.
Neuenhaus, N., Artelt, C., Lingel, K., & Schneider, W. (2011). Fifth graders metacognitive knowledge: General or domain-specific? European Journal of Psychology of Education, 26(2), 163–178. https://doi.org/10.1007/s10212-010-0040-7
Norman, E., Pfuhl, G., Sæle, R. G., Svartdal, F., Låg, T., & Dahl, T. I. (2019). Metacognition in Psychology. Review of General Psychology, 23(4), 403-424. https://doi.org/10.1177/1089268019883821
Ohtani, K., & Hisasaka, T. (2018). Beyond intelligence: A meta-analytic review of the relationship among metacognition, intelligence, and academic performance. Metacognition and Learning, 13(2), 179–212. https://doi.org/10.1007/s11409-018-9183-8
Oliveira, A., & Nascimento, E. (2014). Construção de uma escala para avaliação do planejamento cognitivo [Construction of a cognitive planning assessment scale]. Psychology: Reflection and Criticism/ Psicologia: Reflexão e Crítica, 27(2), 209-218. https://doi.org/10.1590/1678-7153.201427201
Owen, A. (1997). Cognitive planning in humans: neuropsychological, neuroanatomical and neuropharmacological perspectives. Progress in Neurobiology, 53(4), 431-50. https://doi.org/10.1016/s0301-0082(97)00042-7
Peña-Ayala, A. (2015). Metacognition: Fundaments, Applications, and Trends. A Profile of the Current State-Of-The-Art. Springer.
Pazeto, T. C. B., Dias, N. M., Gomes, C. M. A., & Seabra, A. G. (2019). Prediction of arithmetic competence: role of cognitive abilities, socioeconomic variables and the perception of the teacher in early childhood education. Psychology Studies/ Estudos de Psicologia, 24(3), 225-236. https://doi.org/10.22491/1678-4669.20190024
Pazeto, T. C. B., Dias, N. M., Gomes, C. M. A., & Seabra, A. G. (2020). Prediction of reading and writing in elementary education through early childhood education. Psychology: Science and Profession/ Psicologia: Ciência e Profissão, 40, 1-14. https://doi.org/10.1590/1982-3703003205497
Pereira, B. L. S., Golino, M. T. S., & Gomes, C. M. A. (2019). Investigando os efeitos do Programa de Enriquecimento Instrumental Básico em um estudo de caso único [Investigating the effects of the Basic Instrumental Enrichment Program in a single case study]. European Journal of Education Studies, 6(7), 35-52. https://doi.org/10.5281/zenodo.3477577
Perry, J., Lundie, D., & Golder, G. (2018). Metacognition in schools: What does the literature suggest about the effectiveness of teaching metacognition in schools? Educational Review, 71(4), 483-500. https://doi.org/10.1080/00131911.2018.1441127
Pires, A. A. M., & Gomes, C. M. A. (2018). Proposing a method to create metacognitive school exams. European Journal of Education Studies, 5(8), 119-142. https://doi.org/10.5281/zenodo.2313538
Priede, C., & Farrall, S. (2010). Comparing results from different styles of cognitive interviewing: ‘Verbal probing’ vs. ‘thinking aloud’. International Journal of Social Research Methodology, 14(4), 271–287. https://doi.org/10.1080/1:3645579.2010.523187
Rhodes, M. G. (2019). Metacognition. Teaching of Psychology, 46(2), 168-175. https://doi.org/10.1177/0098628319834381
Rodrigues, C. L., Rocca, C. C. A., Serafim, A., Santos, B., & Asbahr, F. R. (2019). Impairment in planning tasks of children and adolescents with anxiety disorders. Psychiatry Research, 274, 243-246. https://doi.org/10.1016/j.psychres.2019.02.049
Rodrigues, M. N. S., & Gomes, C. M. A. (2020). Testing the hypothesis that the deep approach generates better academic performance. International Journal of Development Research, 10(12), 42925-42935. https://doi.org/10.37118/ijdr.20579.12.2020
Rosário, V. M., Gomes, C. M. A., & Loureiro, C. M. V. (2019). Systematic review of attention testing in allegedly "untestable" populations. International Journal of Psychological Research and Reviews, 2(19), 1-21. https://doi.org/10.28933/ijprr-2019-07-1905
Rouault, M., McWilliams, A., Allen, M. G., & Fleming, S. M. (2018). Human Metacognition Across Domains: Insights from Individual Differences and Neuroimaging. Personality Neuroscience, 1, 1-13. https://doi.org/10.1017/pen.2018.16
Sampaio, R. T., Loureiro, C. M. V., & Gomes, C. M. A. (2015). A Musicoterapia e o transtorno do espectro do autismo: uma abordagem informada pelas neurociências para a prática clínica [Music therapy and autism spectrum disorder: an informed approach by neurosciences for clinical practice]. Scholarly Music Journal/ Per Musi, 32, 137-170. https://doi.org/10.1590/permusi2015b3205
Schraw, G. (2008). A conceptual analysis of five measures of metacognitive monitoring. Metacognition and Learning, 4(1), 33-45. https://doi.org/10.1007/s11409-008-9031-3
Schraw, G. (2009). Measuring metacognitive judgments. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), The educational psychology series. Handbook of metacognition in education (pp. 415-429). Routledge/ Taylor & Francis Group.
Sullivan, J. R., Riccio, C. A., & Castillo, C. L. (2009). Concurrent validity of the tower tasks as measures of executive function in adults: a meta-analysis. Applied Neuropsychology, 16(1), 62–75. https://doi.org/10.1080/09084280802644243
Taub, M., Azevedo, R., Rajendran, R., Cloude, E. B., Biswas, G., & Price, M. J. (2021). How are students’ emotions related to the accuracy of cognitive and metacognitive processes during learning with an intelligent tutoring system? Learning and Instruction, 72, 101200. https://doi.org/10.1016/j.learninstruc.2019.04.001
Van der Stel, M., & Veenman, M. (2008). Relation between intellectual ability and metacognitive skillfulness as predictors of learning performance of young students performing tasks in different domains. Learning and Individual Differences, 18(1), 128–134. https://doi.org/10.1016/j.lindif.2007.08.003
Veenman, M. V. J., Van Hout-Wolters, Bernadette H. A. M., & Afflerbach, P. (2006). Metacognition and learning: Conceptual and methodological considerations. Metacognition and Learning, 1(1), 3–14. https://doi.org/10.1007/s11409-006-6893-0
Veenman, M. V. J.,Wilhelm, P., & Beishuizen, J. J. (2004). The relation between intellectual and metacognitive skills from a developmental perspective. Learning and Instruction, 14(1), 89–109. https://doi.org/10.1016/j.learninstruc.2003.10.004
Veenman, M., & Van Cleef, D. (2018). Measuring metacognitive skills for mathematics: students’ self-reports versus on-line assessment methods. ZDM, 51(4), 691-701. https://doi.org/10.1007/s11858-018-1006-5
Wetzel, E., Böhnke, J. R., & Brown, A. (2016). Response Biases. In F. T. L. Leong, D. Bartram, F. M. Cheung, K. F. Geisinger, & D. Iliescu (Eds.), The ITC international handbook of testing and assessment (pp. 349-363). Oxford University Press.
Wolcott, M. D., & Lobczowski, N. G. (2021). Using cognitive interviews and think-aloud protocols to understand thought processes. Currents in Pharmacy Teaching and Learning, 13(2), 181-188. https://doi.org/10.1016/j.cptl.2020.09.005