Addressing Systematic Missing Data in Causally Interpretable Meta-Analysis

• In Event: How do we account for missing and unobserved data?: Diminishing the impacts of hidden bias through novel statistical approaches

Abstract

Introduction: Causal inference in meta-analysis requires clearly defining a target population, individual participant data, and identifiability assumptions to transport evidence from clinical trials to that target. Causal inference is fraught when populations underlying trials differ from the target in terms of treatment modifiers or predictors of the outcome. Systematic missing data occurs when one or more trials did not measure variables necessary for analysis. This type of missing data is present in most meta-analytic projects and limits causal inference when missing covariates are needed to satisfy identifiability assumptions. Our team has articulated the additional identifiability assumptions required for unbiased inference in the context of systematic missing data. We will present these additional assumptions along with simulations that demonstrate the bias associated with violations of identifiability assumptions.
Methods: Covariates of interest for transportability analysis in clinical trials are treatment modifiers that differ between trial and target populations. To better simulate real-world data, simulations used covariate distributions from three adolescent HIV trials. Covariates included 3 continuous and 3 binary variables. Simulations included 8 trials of 500 participants each and varied the strength of treatment effect modifiers (standardized mean difference=.2 to 1), size of between trial differences on the modifiers (Odds Ratios=1.5 to 3), and pattern of systematic missing data (1-5 trials with missing covariate). We used three different estimators: 1) a conventional pooled estimator that only used trial data, 2) an outcome estimator (i.e., g-estimation; Steingrimsson et al., 2022) that uses trial data and data from the target population to adjust for differences in covariates, and 3) an outcome estimator that only used trials that assessed the covariate.
Results: Simulation results were consistent with expectations that bias increased as a function of stronger modifiers and larger between-population differences (Figure 1). Using trials with systematic missing data requires assuming missing covariates are ignorable given non-missing covariates. This assumption is evaluated for each trial in the analysis. When identifiability assumptions are met estimates are unbiased even as the amount of missingness increases. If a trial did not measure an important covariate, and the trial differs from the target on that covariate, it may be better to not include the trial in the estimation of the treatment effect in that target population to obtain an unbiased, albeit less precises estimate (Figure 2).
Conclusions: When combining information across trials to inform treatment recommendations in a target population it is important consider how each trial matches the target population on prominent treatment modifiers. Evaluating assumptions for each trial helps maximize information available for unbiased inference. For example, trials that are missing modifiers but are from the same population as the target can be included in analysis as well as those that differ from the target but include modifiers for use in modeling. Understanding identifiability assumptions aids evaluation of information quality as researchers seek to better understand what works for whom. These assumptions and approaches also apply to work outside of clinical trials with the covariates of interest expanding to include all predictors of the outcome of interest.

Authors

• David Barker, Brown University
  Presenting Author

• Ruofan Bie, Brown University

• Jon Steingrimsson, Brown University