02.02.2027

Prof. Dr. Ronny Scherer

University of Oslo | Norway

From Parallel to Integrated: Meta-Analyses Meet Large-Scale Assessments in Educational Research

Meta-analysis and large-scale assessments in education (LSAs, such as PISA, TIMSS, ICILS, or NEPS) are two of the most influential tools for generating policy-relevant evidence. Yet they have developed largely in parallel, with few systematic attempts to connect them. This lecture argues that treating these two evidence sources separately is a missed opportunity, and it proposes a framework for bringing them together.

Two complementary strategies are presented: First, synthesizing effect sizes directly from LSA data across countries, regions, samples, and assessment cycles. Second, integrating LSA-derived effects into broader meta-analyses alongside evidence from conventional, non-LSA studies. Both strategies address recurring limitations of conventional meta-analysis, including small samples, uneven psychometric quality, and culturally selective samples, while drawing on the representativeness and measurement rigor that LSAs offer.

The framework is illustrated with empirical examples comparing effect sizes across countries, samples, and over time. These examples show how combining the two evidence streams can reveal meaningful divergences in overall effects, heterogeneity, and moderator patterns that remain hidden when LSA and non-LSA evidence are analyzed separately.

The lecture closes with open questions for the field: When should LSAs be included in meta-analyses? What do discrepancies between evidence sources tell us? And what does integrating these two traditions mean for building cumulative, longitudinal, and policy-relevant evidence in educational research?