Primary Submission Category: Policy Learning
Nonparametric Estimation of Optimal Just-In-Time Adaptive Interventions for Distal Outcomes
Authors: Jack Wolf, Nandita Mitrta, Ashkan Ertefaie,
Presenting Author: Jack Wolf*
Mobile and wearable technologies enable the delivery of just-in-time adaptive interventions (JITAIs)—interventions that adapt treatment delivery to an individual’s rapidly changing internal state and context in real-time, real-world settings. However, existing methods for estimating optimal policies do not scale to the complexity of these designs and estimating optimal JITAIs remains challenging. In particular, JITAI settings typically involve dozens of decision points per individual, which cannot be handled using standard longitudinal causal inference methods. Advanced reinforcement learning approaches often optimize discounted sums of proximal outcomes and cannot support common questions in behavioral and clinical studies regarding end-of-study distal outcomes, which reflect long-term success rather than immediate effects. To address these challenges and align with scientific objectives, we make two methodological contributions. First, we develop a nonparametrically efficient inverse probability weighting approach for estimating optimal JITAIs for distal outcomes. Second, we introduce a data-driven policy tilting procedure that mitigates numerical positivity violations common in settings with a large number of decision points to improve finite-sample performance. We apply the proposed framework to Project MARS, a micro-randomized trial for smoking cessation that evaluated mobile health prompts recommending self-regulatory strategies to support quit attempts.