Primary Submission Category: Causal Fairness, and Bias/Discrimination
A Formal Causal Perspective on Outcome Tests for Discrimination
Authors: Kai Cooper, Dean Knox,
Presenting Author: Kai Cooper*
Outcome tests detect discrimination by comparing success rates across groups: differing standards should produce different success rates. But inframarginality bias (ignorance of individual heterogeneity) undermines this—success rates mix obvious and borderline cases, obscuring the differential nature of the decision standard. Recent work shows even comparing marginal cases requires strong structural assumptions. Their remedy, via an econometric Roy Model, improves on earlier game-theoretic approaches but needs restrictive assumptions which are difficult to justify in practice. We propose probabilistic epistemic causal models (ECMs), extending structural causal models by giving a mathematical representation of decision makers’ beliefs about counterfactual outcomes across possible worlds via graphical models defined by Single World Intervention Graphs. Our approach permits the analyst to target a broader range of estimands of interest in empirical studies of discrimination without requiring game-theoretic notions of equilibrium or parametric specifications of decision rules. Additionally, since decisions depend on both group membership and case characteristics, the decision node acts as a collider. In turn, we offer an avenue to understand outcome tests through collider bias and causal interaction. We demonstrate our findings across a range of domains including: college admissions, bail decisions, and traffic stops and searches.
Â