All Tech Blogs Explore Tags Send Feedback

Endigest

© 2026 Endigest. All rights reserved.

About
Privacy
Terms
Contact
RSS

Data Engineering Articles

7 articles

Related Tags

Announcements(2)

(1)

Data Strategy(1)

Data Engineering•2026-05-23

Scaling for MHHS: how Octopus Energy achieved a 50x cost reduction in margin data engineering

Octopus Energy re-architected its data pipeline to handle Half-Hourly Settlement regulations, achieving a 50x cost reduction.

Data Engineering

Data Engineering•2026-04-16

Building with Databricks Document Intelligence and Lakeflow

Databricks Document Intelligence and Lakeflow enable enterprises to automate Intelligent Document Processing by unlocking data trapped in PDFs and documents within the data platform.

Data Engineering

Data Engineering•2026-04-07

How MakeMyTrip Achieved Millisecond Personalization at Scale with Databricks

MakeMyTrip, India's largest online travel agency, achieved millisecond-level personalization latency using Databricks' Real-Time Mode (RTM) in Apache Spark Structured Streaming.

Data Engineering

Data Engineering•2026-03-19

Announcing General Availability of Real-Time Mode for Apache Spark Structured Streaming on Databricks

Databricks announces General Availability of Real-Time Mode (RTM) for Apache Spark Structured Streaming, enabling millisecond-level latency without leaving the Spark ecosystem.

Data Engineering

Data Engineering•2026-03-16

Breaking the Microbatch Barrier: The Architecture of Apache Spark Real-Time Mode

Apache Spark 4.1 introduces Real-Time Mode (RTM) for Structured Streaming, enabling millisecond-level latency without abandoning the microbatch architecture.

Data Engineering

Data Engineering•2026-03-02

Real-Time Mode: Ultra-low latency streaming on Spark APIs without a second engine

Apache Spark introduces Real-Time Mode (RTM) for Structured Streaming, enabling sub-second latency without requiring a second engine like Apache Flink.

Data Engineering

Data Engineering•2026-02-23

Announcing General Availability of Zerobus Ingest

This post announces the general availability of Zerobus Ingest, a real-time data ingestion product targeting organizations at scale.

Data Engineering