Handling late data in streaming windows
WHAT IT TESTS: event-time stream processing. OUTLINE: use event-time windows with watermarks to bound lateness, allow a grace period before finalizing, and route data later than that to a side output.
WHAT IT TESTS: whether you understand event time, watermarks, and the accuracy-latency tradeoff in streaming. ANSWER OUTLINE: aggregate on event time, not processing time, so windows reflect when events actually occurred; use watermarks to estimate progress and tolerate a configured amount of lateness, keeping window state open during a grace period so late events update results; finalize and emit when the watermark passes, and route events later than the allowed lateness to a side output or dead-letter sink rather than silently dropping.
Read the original → interview
- #streaming
- #flink
- #watermarks
- #event-time
- #data-engineering
Get five bites like this every day.
Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.