Meta-Analyses With Few Studies

• In Event: 61.033 - Single-Case Experimental Designs: Developments in Statistical Analysis, Effect Size Metrics, and Meta-Analysis Methods

Abstract

Objective. To determine under what circumstances usable information about the mean effect size, its standard error, and the variance of effect sizes (heterogeneity) can be determined when there are only a few studies in a meta-analysis. This situation is common in meta-analysis of single-case experimental designs.

Theoretical framework. Much of the analysis will be done in the fully Bayesian framework in order to capitalize on contexts in which prior knowledge is available.

Methods. The hblm program and the WinBUGS program will be used to generate estimates under a variety of combinations of number of studies (k), size of studies (n), variability (heterogeneity) among studies (tau). I will determine the circumstances under which reasonably accurate estimates can and cannot be obtained. I will also illustrate the techniques on some real data sets.

Results. Preliminary investigations with real data reveal that there are circumstances when even two or three studies can provide some useful information. These meta-analyses involved studies with very large sample sizes, and some had little heterogeneity.

Scientific significance. A large number of systematic reviews involve small numbers of studies. In particular, single case designs often involve only a few cases, and few studies of the same topic. As another example, most of the research summaries developed by the What Works Clearinghouse include only two studies, leading to questions about how best to represent the results.

Author

• David M. Rindskopf, City University of New York