Combining Dependent Effects From Studies Reporting Regression Analyses

• In Event: Meta-Analysis of Complex Primary Studies

Abstract

Over the years several indices have been proposed to summarize regression analyses (e.g., Aloe & Becker, 2012; Keef & Roberts, 2004; Kim, 2011). These proposed indices determine the relative importance of a single focal predictor variable to the outcome variable. However, sometimes meta-analysts want to understand the role of a set of variables in predicting the outcome. Tracey Gunter presents four methods that may be used to obtain a composite effect for two focal predictor variables that relate to the same construct from a single regression model.
For example, consider research on the effect of socio-economic status (SES) on student performance. Instead of a single predictor variable representing SES, studies may report regression models that include one predictor variable representing parental income, and another predictor variable representing parental education.
A simulation study examined the behavior of four indices under various conditions: the average of the standardized regression coefficients (ASC), the average of the standardized regression coefficients using Hedges and Olkin’s (1985) approach (AHO), the sheaf coefficient (SC), and the squared multiple semi-partial correlation coefficient (MSP), as well as their estimated variances. Some of these indices are familiar, but to date have not been put forth for use in this multivariate meta-analysis context. The conditions manipulated were the number of predictor variables in the model (2, 5, or 10), the sample size (100, 300, or 500), the correlations between the focal predictor variables in the model (.2, .5, or .8), and the correlations between the focal and non-focal predictor variables in the model (.1, .2, or .3). The bias and coefficient of variation values were computed to evaluate the performance and compare the variability, respectively, of each estimator and its variance across 10,000 replications.
Under all conditions examined, the average bias values of the ASC and AHO estimates are small even when the sample size is small. On the other hand, the average bias values of the SC and MSP estimates decrease as the sample size increases. The results also show that the ASC and AHO estimates and their estimated variances are more precise than the other indices. Furthermore the ASC and AHO methods take into account the correlations among the focal predictor variables. Therefore, when meta-analysts are interested in estimating the effect of a set of predictor variables on an outcome variable from a single regression model, the ASC or AHO procedures are preferred.

Author

• Tracey D. Gunter, Florida State University