What is the Impact of Lag Length in Developmental Research? A Re-Analysis of Meta-Analyses Using Lag as Moderator

• In Event: Poster Session 04
  In Poster Session: PS 04 Topic - Methods, History, Theory

Abstract

Longitudinal studies are fundamental to the advancement of developmental science. A survey by Card and Little (2007) determined that 41% of studies published in developmental journals utilize a longitudinal design. Previous primary studies (Selig et al., 2012) and simulation studies (Pelz & Lew, 1970) indicate that the time span between longitudinal measurements (lag) is likely to impact the magnitude and even the direction of the effect sizes found in longitudinal research. Despite these results, many longitudinal studies choose lag based on convenience, potentially affecting the conclusions those studies draw. Card (2019) introduced Lag as Moderator Meta-Analysis (LAMMA), a tool that utilizes between-study variability in lag to assess the impact of lag length on longitudinal effect sizes. This proposal re-analyzes data from multiple previously published meta-analyses of longitudinal data to investigate how lag length may have impacted existing developmental research.

A systematic review of the literature was conducted yielding 1,167 potential meta-analyses (Figure 1). These were screened for those that 1) exclusively included longitudinal studies; 2) provided study-specific lag and longitudinal effect sizes (stability or cross-lag). This strategy yielded 16 total developmental meta-analyses. Thirteen of these met criteria for inclusion in cross-lag analyses (kstudies=267, N=190,082) while 5 met criteria for stability (kstudies=157, N=45,443). We then coded for effect size (Pearson’s r), lag length, and number of participants.

Of the 85 meta-analyses published between 2013-2018, only 11.8% focused on longitudinal data. The average lag length across studies was 28.24 months (SD=51.18) for cross-lag and 26.27 months (SD=42.27) for stability. Both the cross-lag (32.9%) and stability (65.6%) samples included high numbers of convenience lag lengths (operationalized as 6, 12, 24, 36, and 48 months). LAMMA analyses of the collective cross-lag sample indicated that the impact of lag length was significant for both the linear (r=-.26, Q(1)=100.01, p<.05) and quadratic (rlinear=-.02, Q(2)=153.62, p<.01, rquadratic=-.02, p<.01) functional forms. The same held true for the stability sample, where both the linear (r=-.14, Q(1)=38.74, p<.01) and quadratic (rlinear=-.14, Q(2)=52.06, p<.01, rquadratic=-.19, p<.01) models were significant. These results generally held true when each included meta-analysis was analyzed individually, though both the 1) magnitude of the impact of lag and 2) functional form that best modeled that impact changed depending upon the phenomenon under investigation (Figure 2).

In sum, this meta-analysis re-synthesized the results from previous developmental meta-analyses to ascertain the impact of lag length on longitudinal effect sizes. Results indicate that lag length was a significant moderator for both the collective cross-lag and stability samples and that linear and quadratic functional forms have the potential to accurately model lag. However, the preponderance of convenience lag lengths convenance restricted the range of lags observed and likely reduced our ability to find potential differences. Therefore, it is plausible that the impact of lag on developmental research is greater than the present study was able to show. These results highlight the importance of including variable time lags within and between longitudinal studies. Results of LAMMA analyses on individual meta-analyses of cross-lag and stability and their implications will be discussed.

Authors

• Rachel M Taylor, University of Georgia
  Presenting Author

• Noel Card, University of Georgia
  Non-Presenting Author