Trusting the Untestable: Validation and Diagnostics for the Doubly Robust Models

This post from Lyft explains how they validate and diagnose Doubly Robust (AIPW) models used for causal inference when A/B testing is not feasible.

• •AIPW (Augmented Inverse Propensity Weighting) is used to estimate Average Treatment Effect (ATE) without randomization, relying on an outcome model and a propensity score model
• •The doubly robust property ensures consistent ATE estimates if at least one of the two sub-models is correctly specified
• •A quasi-experimentation platform enforces mandatory confounder selection, with pre-defined SQL-based sets that users can customize
• •Downsampling bias correction is applied using Ballinari (2024): propensity score conversion and outcome reweighting to recover population-level ATE
• •A diagnostic scorecard includes propensity overlap histograms and covariate balance charts to verify common support and adjustment quality