Primary Submission Category: Causal Inference and Bias/Discrimination
Pseudo-Robust Solutions to Lord’s Paradox
Authors: Robert Larzelere, Hua Lin,
Presenting Author: Robert Larzelere*
Analyses of residualized vs. simple change scores often produce contradictory treatment estimates in non-randomized longitudinal studies, a problem known as Lord’s [, 1967, Lord] Paradox. Under some assumptions, the two estimates bracket the unbiased causal effect [Angrist, 2009, Pischke], and the econometric practice of testing robustness across contrasting analyses has been recommended [Duncan, 2014]. Unfortunately, robust consistency across analyses of residualized and simple difference scores often occurs artifactually, which we call pseudo-robustness. We present four pseudo-robust solutions to Lord’s Paradox: (1) Making the pretest a covariate in difference-score analyses makes their treatment effects identical to ANCOVA’s treatment effects. (2) Pretest matching makes the two treatment effects equal to each other and to ANCOVA’s treatment effect before matching, whether biased or not. (3) Centering pretest and posttest scores on pretest group means also produces equivalent treatment effects, but ones equivalent to the treatment effect of difference-score analysis before centering, whether biased or not. (4) Two treatments for depression still appear robustly harmful in secondary analyses of the most at-risk of three subgroups even after propensity-score matching. We need to distinguish appropriate robustness and causal-estimate bracketing from pseudo-robustness in these 2-occasion analyses and in more complex longitudinal analyses.