How Methodological Features Affect Effect Sizes in Systematic Reviews in Education

• In Event: Poster Session 5
  In Poster Session: Systematic Review and Meta-Analysis SIG Poster Session 1

Abstract

The purpose of this article is to examine how methodological features such as types of publication, sample sizes, and research designs affect effect sizes in systematic reviews in education. To examine the relationship between key methodological features and effect sizes, a total of 222 studies from five recent reviews of evaluations of reading, mathematics, and science programs were used. The findings suggest that effect sizes are roughly twice larger for published articles, small-scale trials, and matched control studies than unpublished documents, large-scale studies, and randomized experiments, respectively. In addition, studies that used researcher-developed measures also produce much larger effect sizes than those that used standardized measures. Explanations for the effects of methodological features on effect sizes are discussed.

Authors

• Alan Cheung, The Chinese University of Hong Kong

• Robert Slavin, Johns Hopkins University