Beyond A/B Testing: Using Surrogacy and Region-Splits to Measure Long-Term Effects in Marketplaces

This article presents Lyft's two-step methodology for estimating long-term market-mediated effects of marketplace policy changes using a surrogacy and region-split approach.

• •The Foundational Models team built a framework to quantify how changes to rider prices or driver incentives produce indirect long-term behavioral effects through market dynamics
• •Step 1 uses residualized regressions to estimate how policy decisions shift the distribution of negative user experiences (wait time, surge pricing, driver idleness) by removing cyclical/seasonal baselines
• •Step 2 applies Augmented Inverse Probability Weighting (AIPW), a doubly robust causal estimator, to map short-term negative experiences to long-term outcomes via a surrogacy index
• •Switch-back experiments validate Step 1 and user-split experiments validate Step 2 independently