Primary Submission Category: Sensitivity Analysis
Application of the Robustness of Inference to Replacement (RIR) to Differential Attrition in an RCT
Authors: Kenneth Frank,
Presenting Author: Kenneth Frank*
Differential attrition is one of the most serious sources of bias in estimates of treatment effects in randomized experiments (e.g., Hewitt et al., 2010; WWC Standards Handbook 4.1, 2020). In theory, if even only one case drops out of either the treatment or control for a systematic, non-random reason, the principle of randomization as a basis for estimation and inference is compromised. While loss of one case is unlikely to overturn results in most instances (unless its outcome would have been extreme), the question generally concerns how robust an inference from an RCT is to differential, non-random attrition from treatment and control. While there are many techniques for imputing the attritted data, ultimately there will be some aspects of the attritted data that are unobservable. The purpose of this paper is to characterize the unobserved conditions in the attritted data such that, if combined with observed data, it would nullify an inference of an effect of the predictor of interest (X) on the outcome (Y). We do so for a non-parametric approach based on differences in means on the outcome in the attritted data and then for a parametric approach based on the correlation between treatment and outcome in the attritted data that uses statistical significance as a threshold.