Receive daily AI-curated summaries of engineering articles from top tech companies worldwide.
Endigest AI Core Summary
This article describes the architecture, optimization, and evolution of Lyft's Feature Store, a core ML infrastructure platform serving 60+ use cases across the rideshare stack.
•The system is structured as a "platform of platforms" with three main components: Batch, Online, and Streaming features
•Batch features are defined via Spark SQL queries and JSON configs, auto-generating Airflow DAGs on Astronomer that handle computation, storage, data quality checks, and feature discovery
•The online serving layer (dsfeatures) uses DynamoDB as primary store, a ValKey LRU write-through cache for low-latency access, and OpenSearch for embedding features
•Streaming features use Apache Flink reading from Kafka/Kinesis, processed through a dedicated ingest service (spfeaturesingest) that writes to dsfeatures
•
Developer experience centers on SQL-first design with simple JSON configs, Go and Python SDKs exposing full CRUD operations, and metadata-driven feature governance
This summary was automatically generated by AI based on the original article and may not be fully accurate.