Improving Estimation Accuracy of the Interaction Effects in Models With Composite Scores

• In Event: Composite/Factor Scores Analyses With Bias Correction: Good Alternatives to Full Structural Equation Modeling?

Abstract

Testing interaction effects is a prevailing issue in social and behavioral research. Many studies on interaction effect testing have relied on composite scores based on multiple observed items, but those studies often ignore the measurement errors of the variables. Several studies have shown that ignoring measurement errors may lead to biased estimates. However, such practice has still been commonly used among substantive studies. The reasons behind such prevalence may be due to the fact that the full SEM model with multiple indicators increases the complexity of the model (in terms of the number of parameters and variables) and hence larger sample size would be needed to warrant the application of latent variable models. Additionally, the chance of getting non-convergence results increase as the model complexity increase. Furthermore, for researchers using secondary data with only composite scores and reliability information from the manual, there is no way for them to access the individual item scores of the data.

We demonstrated two single indicator (SI) methods under the structural equation modeling (SEM) framework—the reliability-adjusted product indicator (RAPI) method and the latent moderated structural equations (LMS) method—both methods can take into account measurement errors of the exogenous composites. We further test the performance of the two SI methods with the full SEM and the conventional path model, which assumed the exogenous composites are measured without errors. Several design factors were also examined, including sample size, population reliability values, correlation between the exogenous variables, assumptions behind the measures (e.g., tau-equivalent vs. congeneric measures), types of sample reliability coefficients (i.e., Cronbach’s alpha, omega composite, Coefficient H, greatest lower bond [GLB], and fixed reliability).

Generally, both SI methods outperform the path model in terms of yielding unbiased interaction effects across all the design factors. No salient difference between the RAPI and the LMS methods was found. The path model yielded substantial biases with population reliability values at .90 and lower, indicating that correction for measurement errors is necessary even with measures with high reliability. Both SI methods along with alpha, omega, Coefficient H, and GLB performed comparably well and produced unbiased interaction effects. However, fixed reliability approach was only adequate when the specified reliability values close to the population reliability. Other findings and implications were further discussed.

Authors

• Yu-Yu Hsiao, University of New Mexico

• Oiman Kwok, Texas A&M University - College Station

• Hok Chio (Mark) Lai, University of Southern California