PCORI Methodology Award: Toward Improved Design and Analysis of Stepped Wedge Trials: An Estimand-Aligned and Efficiency-Focused Framework

Last updated: October 2026

Background and Basic Idea

Stepped wedge cluster randomized trials (SW-CRTs) are increasingly adopted for comparative effectiveness and implementation science research. In a SW-CRT, all clusters start under the control condition and cross over to the intervention at randomly assigned, staggered time points. While around 30 to 40 SW-CRT protocols are published each year, the analysis of these trials has relied almost exclusively on mixed-effects models or generalized estimating equations (GEE), with the treatment effect read off as a single regression coefficient. Because the multilevel and longitudinal structure allows different analytic decisions to answer different questions, this convention leaves open a fundamental question, namely “what exactly is being estimated, and for whom?”, especially when the model is imperfectly specified. Meanwhile, in our review of SW-CRTs published in 2016-2022, more than 70% adjusted for baseline covariates in the primary analysis, despite limited guidance on how to do so without changing the question being answered. This webpage summarizes our team’s work on causal inference methods for SW-CRTs (supported in large part by a PCORI Methodology Award; see the acknowledgement below), under the philosophy that “estimands drive the statistical analysis of SW-CRTs, but not the other way around.” The development is largely based on the potential outcomes framework and follows two principles. The analysis should be estimand-aligned, in that the estimand is defined first and without reference to any regression model, and the estimator remains valid for that estimand even when the working model is incorrect. The analysis should also be efficiency-focused, in that baseline covariates are leveraged to improve precision, which is essential given the limited number of clusters in a typical SW-CRT. The central finding is that, when cluster-period sizes are variable and informative (i.e., the treatment effect varies with the cluster-period size), there is no longer a single “treatment effect” in a SW-CRT, and the target of inference depends on two choices:

  • Unit of inference (individual-average versus cluster-average): whether each individual or each cluster receives equal weight in the average; this is the distinction already familiar from the estimands literature for parallel-arm CRTs;
  • Direction of aggregation (horizontal versus vertical): whether outcomes are pooled across periods before taking the contrast (horizontal), or each period is treated as a mini-CRT and the period-specific effects are then averaged with equal weight (vertical); this is the less familiar dimension that is unique to the multi-period structure of SW-CRTs.

Crossing these two choices leads to four estimands: the horizontal individual-average treatment effect (h-iATE), the horizontal cluster-average treatment effect (h-cATE), the vertical individual-average treatment effect (v-iATE), and the vertical cluster-average treatment effect (v-cATE). All four are defined over the rollout periods where both treated and control clusters are present, because the all-control first period and the all-treated last period carry no randomized contrast. The four estimands coincide when cluster-period sizes are non-informative, but can differ markedly otherwise.

Figure 1. A schematic illustration of different treatment effect definitions in a hypothetical cross-sectional SW-CRT with 8 clusters and 5 periods. Time periods form the horizontal dimension and clusters form the vertical dimension. Horizontal estimands pool outcomes across the rollout periods, whereas vertical estimands first define the effect within each period and then give each period equal weight. Within each direction, the individual-average and cluster-average versions differ in whether individuals or clusters are weighted equally. More details in Fang et al (2026).

The procedure involves the following steps: (1) Pre-specify the estimand that matches the scientific question, before choosing any model; (2) Fit a familiar working outcome model with pre-specified baseline covariates, such as a linear mixed model, a generalized linear mixed model, or GEE; (3) Apply model-robust standardization to convert the fitted working model into an estimand-aligned effect estimate, with a leave-one-cluster-out jackknife variance estimator for inference; (4) Compare estimates across the four estimands as a check on informative cluster-period sizes. In our simulations and reanalyses, the treatment coefficient from the working model can be noticeably biased for these estimands under informative sizes (sometimes more so with richer random-effects structures), whereas the model-robust standardization estimator remains nearly unbiased with close to nominal coverage regardless of the working model, and is more efficient than the unadjusted nonparametric estimator. To support implementation, an R package (MRStdLCRT, available on CRAN) and annotated R source code (swcrtcov, for model-assisted ANCOVA estimators) have been developed by our team, and all software, code and case-study tutorials are centralized on a dedicated GitHub page. At this stage, MRStdLCRT covers continuous and binary outcomes (on the mean difference, risk difference, risk ratio or odds ratio scale) and builds on the assumption that the treatment effect does not depend on the duration of exposure, which is arguably the most common working assumption in practice. The swcrtcov code is currently released as annotated source code rather than a formal package. Extensions to dynamic, anticipation, and multiple-intervention settings are addressed in separate publications listed below.

Publications and Software

We have developed a collection of new estimands, estimators and theoretical results for defining and estimating average causal effects in SW-CRTs, addressing different aspects of complications encountered in statistical practice. These publications include:

Figure 2. Panel (a): Estimated treatment effects on PTSD symptom severity in the Trauma Survivors Outcomes and Support (TSOS) trial using the model-robust standardization (MRS) estimators and the treatment coefficient (Coef) estimators under six working models, as well as an unadjusted estimator. Panel (b): Estimated odds ratios of 30-day mortality in the Acute Coronary Syndrome Quality Improvement in Kerala (ACS-QUIK) trial. Each panel corresponds to one of the four estimands. It is apparent that the MRS estimates carry similar magnitude across working models for each estimand, whereas the Coef estimates vary more substantially with the model choice. Details in Fang et al (2026).

The estimands above assume that the intervention takes full effect once rolled out. Given that this may not hold in many pragmatic SW-CRTs, our team has also studied causal effects that vary with exposure time, arise before rollout, or involve more than one intervention. These publications include:

Given that the estimand questions in SW-CRTs are rooted in those for simpler cluster randomized designs, our team has also provided a set of foundational results on estimands and model-robust covariate adjustment in parallel-arm cluster randomized trials and cluster randomized crossover trials (CRXO trials), which served as building blocks for the stepped wedge developments. These publications include:

In companion, our team contributed to a community-wide effort to bring the estimands framework into the practice of cluster randomized trials, culminating in the consensus-based CRT-Estimands Framework as an extension of the ICH E9(R1) addendum. These publications include:

Finally, a credible causal analysis starts from an efficient design. For planning SW-CRTs, we have contributed sample size methods that leverage a baseline measurement of the outcome, optimal and cost-efficient design selection, and systematic reviews of current practice. These publications include:

Beyond the average causal effects, for subgroup causal effects, the study team is continuing to explore calibrated Bayesian methods with mixed-effects Bayesian Additive Regression Trees (BART), and the current developments in this area are:

To illustrate these methods in the context of real studies, we reanalyzed recently completed pragmatic SW-CRTs with data shared by our stakeholder partners, including the Trauma Survivors Outcomes and Support (TSOS) trial, the Randomized Evaluation of Default Access to Palliative Services (REDAPS) trial, the Lumbar Imaging with Reporting of Epidemiology (LIRE) trial, and the Prognosticating Outcomes and Nudging Decisions in the Electronic Health Record (PONDER) trial. The case-study tutorials walk a reader from study design (SWCRTbaseline Shiny app), through model-assisted and model-robust analysis (MRStdLCRT package and swcrtcov code), to flexible Bayesian subgroup analysis (mixed-effects BART), and are all available on the project GitHub page.

Figure 3. Example output from the MRStdLCRT R package, which displays the unadjusted and model-robust standardization estimates with 95% confidence intervals for h-iATE, h-cATE, v-iATE and v-cATE side by side in a simulated SW-CRT with a continuous outcome. Details in the package tutorial.

Primary Investigator Team

  • Principal Investigator: Fan Li, PhD, Associate Professor, Department of Biostatistics, Yale School of Public Health
  • Co-Investigator: Patrick Heagerty, PhD, Professor, Department of Biostatistics, University of Washington
  • Co-Investigator: Liangyuan Hu, PhD, Associate Professor, Department of Biostatistics and Epidemiology, Rutgers University
  • Co-Investigator: Michael Harhay, PhD, Associate Professor, Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania

Disclaimer and Acknowledgement

The research summarized on this webpage was supported in large part by a competitive Patient-Centered Outcomes Research Institute (PCORI) Award ME-2022C2-27676, “Toward Improved Design and Analysis of Stepped Wedge Trials: An Estimand-aligned and Efficiency-focused Framework”. The official PCORI webpage can be found here, where one can also locate the peer-reviewed version of the Final Research Report once it is online (forthcoming). All members of the study team expressed sincere gratitude to PCORI for supporting this work and making it possible, thereby enabling the development of this new strand of methodological research. To our knowledge, this work represents the first effort to bring estimands and model-robust causal inference into the stepped wedge cluster randomized trials.

The Investigator Team would also like to express their deep gratitude to all collaborators and stakeholder partners who have contributed their expertise and dedication throughout this award, including Drs. Jeffrey Jarvik (PI of the LIRE trial), Douglas Zatzick (PI of the TSOS trial), Scott Halpern and Katherine Courtright (PIs of the PONDER-ICU and REDAPS trials), Thomas Travison, and Monica Taljaard (members of the Design and Statistics Core at the NIA IMPACT Collaboratory). It has been a genuine privilege to work with such an engaged and interdisciplinary team. The stakeholder discussions, together with the continued feedback from the Biostatistics and Study Design Core of the NIH Pragmatic Trials Collaboratory and the Design and Statistics Core of the NIA IMPACT Collaboratory, have profoundly shaped the direction of this research and have motivated ideas that extended well beyond the initial proposal. We are truly thankful for their partnership.

Disclaimer: The statements in this website are solely the responsibility of the authors and study team and do not necessarily represent the views of PCORI, its Board of Governors, or Methodology Committee. The principal investigator Fan Li extends special thanks to Hao Wang and Yukang Zeng for their valuable assistance in developing the initial version of this webpage.