Keynote speaker
Ronny Scherer
University of Oslo, Centre for Educational Measurement at the University of Oslo (CEMO) and Centre for Research on Equality in Education (CREATE)
Two Pillars, One Framework: How Meta-Analyses and ILSAs Can—and Should—Work Together
Meta-analyses and international large-scale assessments (ILSAs) are two of educational research’s powerful tools for generating policy-relevant evidence—yet they have largely developed in parallel. This keynote argues that treating these sources separately is a missed opportunity and presents a framework for bridging them.
Drawing on individual-participant and aggregated data meta-analysis, two complementary approaches are described: synthesizing effect sizes directly from ILSA data across countries and cycles and integrating ILSA-derived effects into broader meta-analyses alongside non-ILSA studies. Both approaches address well-known weaknesses of conventional meta-analyses—small samples, insufficient psychometric quality, and cultural selection bias—while leveraging ILSAs’ strengths in representativeness and measurement rigor.
The framework is illustrated through empirical examples examining effect sizes across countries, cycles, and subgroups. These examples reveal how integrated analyses can uncover meaningful divergences between ILSA and non-ILSA data sources in overall effects, heterogeneity, and moderator patterns—differences that remain invisible when the two evidence streams are analyzed in isolation.
The keynote closes with broader questions for the field: When should ILSAs be included in meta-analyses? What do discrepancies between evidence sources reveal? And what does integration mean for cumulative evidence in educational assessment?
Round-table Discussion
Artificial Intelligence in the Classroom: A Revolution or Just Another Tool?
The roundtable discussion explores the relationship between artificial intelligence and education, with particular attention to how these technologies are reshaping the frameworks through which learning, knowledge, and information are understood.
The rapid emergence of generative artificial intelligence not only creates new opportunities but also raises fundamental questions and dilemmas for education. The aim of the discussion is to critically examine these challenges, with a special focus on the changes affecting understanding, interpretation, and critical thinking.
The roundtable will address, among others, the following questions:
- How is the concept and value of knowledge changing in an age of rapidly accessible information?
- What impact does generative artificial intelligence have on reading comprehension, the interpretation of information, and thinking skills?
- What new challenges does it pose for educational assessment?
- What pedagogical responses can be given to situations where student performance and independent knowledge construction are more difficult to distinguish?
The discussion adopts a multidisciplinary perspective, incorporating both educational research and computer science viewpoints, thereby offering an opportunity for a deeper understanding of the phenomenon.
Participants:
Brent Duckor (San José State University, USA)
Carrie Holmberg (San José State University, USA)
Mari-Pauliina Vainikainen (University of Tampere, Finland)
Gábor Berend (University of Szeged)
Moderator:
Ádám Kocsis (University of Szeged)
Ronny Scherer
University of Oslo, Centre for Educational Measurement at the University of Oslo (CEMO) and Centre for Research on Equality in Education (CREATE)
Two Pillars, One Framework: How Meta-Analyses and ILSAs Can—and Should—Work Together
Meta-analyses and international large-scale assessments (ILSAs) are two of educational research’s powerful tools for generating policy-relevant evidence—yet they have largely developed in parallel. This keynote argues that treating these sources separately is a missed opportunity and presents a framework for bridging them.
Drawing on individual-participant and aggregated data meta-analysis, two complementary approaches are described: synthesizing effect sizes directly from ILSA data across countries and cycles and integrating ILSA-derived effects into broader meta-analyses alongside non-ILSA studies. Both approaches address well-known weaknesses of conventional meta-analyses—small samples, insufficient psychometric quality, and cultural selection bias—while leveraging ILSAs’ strengths in representativeness and measurement rigor.
The framework is illustrated through empirical examples examining effect sizes across countries, cycles, and subgroups. These examples reveal how integrated analyses can uncover meaningful divergences between ILSA and non-ILSA data sources in overall effects, heterogeneity, and moderator patterns—differences that remain invisible when the two evidence streams are analyzed in isolation.
The keynote closes with broader questions for the field: When should ILSAs be included in meta-analyses? What do discrepancies between evidence sources reveal? And what does integration mean for cumulative evidence in educational assessment?
Round-table Discussion
Artificial Intelligence in the Classroom: A Revolution or Just Another Tool?
The roundtable discussion explores the relationship between artificial intelligence and education, with particular attention to how these technologies are reshaping the frameworks through which learning, knowledge, and information are understood.
The rapid emergence of generative artificial intelligence not only creates new opportunities but also raises fundamental questions and dilemmas for education. The aim of the discussion is to critically examine these challenges, with a special focus on the changes affecting understanding, interpretation, and critical thinking.
The roundtable will address, among others, the following questions:
- How is the concept and value of knowledge changing in an age of rapidly accessible information?
- What impact does generative artificial intelligence have on reading comprehension, the interpretation of information, and thinking skills?
- What new challenges does it pose for educational assessment?
- What pedagogical responses can be given to situations where student performance and independent knowledge construction are more difficult to distinguish?
The discussion adopts a multidisciplinary perspective, incorporating both educational research and computer science viewpoints, thereby offering an opportunity for a deeper understanding of the phenomenon.
Participants:
Brent Duckor (San José State University, USA)
Carrie Holmberg (San José State University, USA)
Mari-Pauliina Vainikainen (University of Tampere, Finland)
Gábor Berend (University of Szeged)
Moderator:
Ádám Kocsis (University of Szeged)