Batch vs streaming
Batch processes chunks on a schedule, while streaming processes events continuously with low latency.
- Batch is simpler
- Streaming lowers freshness delay
- Trade latency for complexity
Batch vs streaming
Tagged with streaming
3 cards
Kafka basics
Kafka is a distributed log used for durable event streaming and decoupled producers and consumers.
- Topics store ordered partitions
- Consumers track offsets
- Great for event-driven pipelines
Kafka basics
What is late arriving data?
Late arriving data shows up after the expected processing window and can break simple incremental assumptions.
- Common in event systems
- Needs watermark or backfill strategy
- Affects correctness of incremental models
What is late arriving data?