Primary Submission Category: Machine Learning and Causal Inference
Double Descent in Double Machine Learning
Authors: Jann Spiess, Guido Imbens, Amar Venugopal,
Presenting Author: Jann Spiess*
Motivated by a literature on the double-descent phenomenon in machine learning, we consider highly over-parameterized models in causal estimation using double machine learning. We build upon our earlier work on double descent in causal imputation (https://arxiv.org/abs/2305.00700; NeurIPS 2023) to investigate high-dimensional linear regression for estimating outcome regressions and inverse propensity score weights in doubly-robust causal estimators. Specifically, we consider linear-regression models with many more covariates than sample size. In such models, there may be so many free parameters that the model interpolates the training data perfectly. For outcome models in doubly-robust estimation, we find that such interpolating regressions can outperform simple ones, provided that we use appropriate sample splitting. For inverse propensity score weighting, we instead find that estimation strategies that plug in estimated propensity scores from overparameterized models fail dramatically. As a potential remedy, we instead investigate the direct estimation of balancing weights using the automated estimation of Riesz representers with interpolating linear models.