Primary Submission Category: Applicants in Social Sciences
A Win Ratio Approach to Nonparametric, Scale-Free Measure of Causal Effects on Validated Latent Traits
Authors: Beom Kwon, Hyunseung Kang,
Presenting Author: Beom Kwon*
Many studies in education, political science, and psychology aim to estimate treatment effects on validated latent traits, such as ability, attitudes, and disease severity, measured through multiple item responses. Most existing approaches are fully parametric, requiring correct specification of both the latent-trait measurement model and the structural model for causal effects. These approaches can lead to biased estimates when either model is mis-specified. Worse, the causal estimand may depend on the scale of the latent trait, which often lacks an intrinsic scale. In this work, we propose a win ratio approach for measuring treatment effects on validated latent traits. We show that the win ratio is nonparametric, agnostic to the underlying measurement model, and scale-free. We also establish necessary and sufficient conditions under which it can be related to existing causal effect measures. For estimation, we apply an influence function based estimator that accommodates flexible machine-learning methods for nuisance functions without requiring a correctly specified item-response model. Under mild assumptions, the estimator is doubly robust, asymptotically normal, and semiparametrically efficient. Extensive simulation studies under various measurement models, including Rasch, 2PL, 3PL, and graded response models, along with a real data analysis of the effect of game-based learning on math ability, demonstrate that our approach is more robust and stable than existing method.