Lyft’s Feature Store: Architecture, Optimization, and Evolution

This article describes the architecture, optimization, and evolution of Lyft's Feature Store, a core ML infrastructure platform serving 60+ use cases across the rideshare stack.

• •The system is structured as a "platform of platforms" with three main components: Batch, Online, and Streaming features
• •Batch features are defined via Spark SQL queries and JSON configs, auto-generating Airflow DAGs on Astronomer that handle computation, storage, data quality checks, and feature discovery
• •The online serving layer (dsfeatures) uses DynamoDB as primary store, a ValKey LRU write-through cache for low-latency access, and OpenSearch for embedding features
• •Streaming features use Apache Flink reading from Kafka/Kinesis, processed through a dedicated ingest service (spfeaturesingest) that writes to dsfeatures