Inflated Summary Effect Sizes in Meta-Analyses of Flipped Classroom and Collaborative Learning Interventions?

• In Event: Evidence Syntheses of Cognitive, Social, and Behavioral Processes and Practices Across Learning Contexts

Abstract

Summary effect size estimates from meta-analyses may be inflated, due to a combination of low statistical power in individual studies and a bias favouring reports with statistically significant results. This paper asks what methods meta-analysts of educational interventions employ in order to detect and adjust for small-study effects/publication bias, and whether current practice is sufficient to detect such biases. We collected and analysed data from meta-analyses of two selected types of interventions – flipped classrooms and cooperative/collaborative/team-based learning interventions. Findings reveal that most meta-analysts use two or fewer methods to detect or adjust for publication bias. None of a total of 51 meta-analyses utilised methods that model publication selection directly. Results from reanalysing study-level data from a subset of included meta-analyses are inconsistent and should be cautiously interpreted, but do seem to indicate at least some inflation.

Authors

• Torstein Låg, UIT The Arctic University of Norway

• Rannveig Grøm Sæle, UIT The Arctic University of Norway