Efficient Sampling for Multisite-Randomized Trials Investigating Main and Mediation Effects Under Unequal Costs

• In Event: Statistical Power and Optimal Design of Experimental Studies Investigating Main, Moderation, and Mediation Effects

Abstract

Optimal sampling frameworks attempt to identify the sampling plans that produce the maximum statistical power under the same budget by explicitly considering sampling costs. Identified optimal sample allocations can serve as important starting points, which will help navigate to final designs that balance practical considerations and requirements. In this study, we developed optimal sampling strategies for multisite-randomized trials investigating main and medication effects.

The conventional optimal design framework for investigating main and mediation effects (Raudenbush, 1997; see also Konstantopoulos, 2009, 2011; Hedges & Borenstein, 2014; Raudenbush & Liu, 2000; Cox & Kelcey, 2019; Kelcey & Shen, 2020) assumes the equal costs of sampling between treatment conditions and constrains the sampling ratio between treatment conditions as equal. Because the conventional framework does not account for the potential sampling cost differences between treatment conditions, it thus only can identify sub-optimal designs without optimizing the sampling ratio between treatment conditions.

To conquer these limitations, the literature recently developed a more flexible framework that allows the sampling costs to diversify across both levels and treatment conditions. The framework optimizes sampling ratios across levels and between treatment conditions (three citations masked). This framework leads to more powerful and cost-efficient designs identified. It has been implemented in the R package XXX (citation masked). Below we outline the extension of this framework to 1-1-1 mediation in multisite randomized trials with the treatment assignment, mediator, and outcome at the individual level. Because of space constraints, we briefly outline the methods and omit the derivation.

We derived optimal sample allocation for mediation under the Sobel test (Sobel, 1982) and the joint significance test (Hayes & Scharkow, 2013; MacKinnon et al., 2002) because the former has been historically popular, and the latter provides relatively high power. We omit the presentation of the Sobel illustration as the results showed the same conclusion that the framework can identify the optimal sample allocation.

Under the joint significance test (Hayes & Scharkow, 2013), the identified optimal design parameters are n = 18.94 and p = .21 for an illustration example (n and p are the number of individuals per site and the proportion of individuals in the treatment group). We plot the power curve under a fixed budget of 16,400 cost units, which can achieve 80% power under the optimal sample allocation with J = 62.82 clusters. Figure 1 shows that power is maximized under the optimal sample allocation and statistical power will decrease when a design departs from the optimal allocation from any direction. If we use a conventional framework that has a fixed p = .50, under the same budget, we can only achieve about 67% power (Figure 1b).

The proposed framework and the results from this study provide new tools to structure sampling conversations and ultimately improve the efficiency and cost burdens of studies. More practically, the results and accompanying software not only will facilitate more efficient designs but also encourage thoughtful consideration of cost-efficient design strategies and potentially reduce the cost burdens of large-scale experiments.

Authors

• Zuchao Shen, University of Georgia
  Presenting Author

• Benjamin Kelcey, University of Cincinnati
  Non-Presenting Author

• Wei Li, University of Florida
  Non-Presenting Author

• Walter L. Leite, University of Florida
  Non-Presenting Author