Primary Submission Category: Bayesian Causal Inference
When IRT Point Scores Stand in for Latent Confounders: How Test Information Shapes Bias and Interval Validity in Causal Adjustment
Authors: Weiran Li, Bruno Zumbo, Xiangyi Liao,
Presenting Author: Weiran Li*
Applied evaluations often adjust for baseline differences using test based scores even when treatment uptake is driven by an unobserved latent variable that also predicts outcomes. Let theta denote this baseline latent confounder. Because theta is unobserved, analysts often plug in an IRT point score, typically the EAP, and treat it as error free. Fully joint IRT measurement, treatment, and outcome models can address latent confounding, but they are often hard to specify, fit, and communicate in routine evaluations, creating a gap between recommended joint modeling and common workflows. We link instrument information and targeting to both bias and interval validity of causal adjustment with IRT point scores. We simulate data from a coherent model with 2PL measurement, logistic treatment selection driven by theta, and a linear outcome with a nonzero treatment effect, varying test length, discrimination, and difficulty targeting. Unadjusted analyses show large bias, about 0.26 to 0.28, and near zero nominal 95 percent coverage. Plug in EAP adjustment reduces bias under high information, but interval validity depends sharply on where the test is informative: under low information, coverage collapses to about 0.04 to 0.08, and even under higher information coverage remains below nominal at about 0.69 to 0.81. These results motivate uncertainty propagation via posterior draws and principled pooling while retaining existing analytic workflows.