Primary Submission Category: Machine Learning and Causal Inference
Differentially Private Two-Stage Empirical Risk Minimization
Authors: Joowon Lee, Guanhua Chen,
Presenting Author: Joowon Lee*
We propose a differentially private algorithm for two-stage empirical risk minimization (ERM), designed to balance privacy, utility, and efficiency. In the first stage, we compute data-dependent sample weights to balance covariate distributions between treatment groups, ensuring control of confounding factors. In the second stage, the resulting weighted ERM is solved with objective perturbation to obtain a private optimal model parameter.
Our main application is individualized treatment rules (ITRs) with privacy guarantees. For this application, it is important to use weights that balance covariate distributions between treatment groups to control confounding factors, so the first stage cannot be privatized. This precludes the standard composition method of privatizing multi-stage pipelines for private ITR. Our proposed method requires privatizing only the second stage and uses deterministic perturbation analysis for the first stage. We establish guarantees for efficient differential privacy and utility of our method. Our general framework applies to a wide range of ITR problems with inverse probability weights and distributional covariate balancing weights.