Primary Submission Category: Randomized Designs and Analyses
Compound Causal Selection Decisions: An Almost SURE Approach
Authors: Timothy Sudijono, Jiafeng Chen, Lihua Lei, Liyang Sun, Tian Xie,
Presenting Author: Timothy Sudijono*
This paper proposes methods for producing selection decisions in a Gaussian sequence model. These decisions directly target the following problem: given unknown, fixed parameters $mu_{1:n}$ and known $sigma_{1:n}$ with observations $Y_i sim textsf{N}(mu_i, sigma_i^2)$, the decision maker would like to
select a subset of indices S so as to maximize utility $sum_{iin S} (mu_i – K_i)$,
for some known costs K. This problem appears in the management of experimentation programs: large collections of related A/B tests in the online technology industry where selections are made based on ATE estimates. It also appears as a special case of empirical welfare maximization, where subpopulations are selected on the basis of CATE estimates.
Inspired by Stein’s unbiased risk estimate (SURE), we introduce an almost unbiased estimator, called ASSURE, for the expected utility of a proposed
decision rule. ASSURE allows a user to choose a welfare-maximizing rule from a
pre-specified class by optimizing the estimated welfare, thereby producing selection
decisions that borrow strength across noisy estimates. We show that ASSURE produces
decision rules that are asymptotically no worse than the optimal but infeasible decision
rule in the pre-specified class. We apply ASSURE to the
selection of Census tracts for economic opportunity, the identification of discriminating
firms, and the analysis of p-value decision procedures in A/B testing.