tezvyn:

Handling late data in streaming windows

Source: interviewintermediate

WHAT IT TESTS: event-time stream processing. OUTLINE: use event-time windows with watermarks to bound lateness, allow a grace period before finalizing, and route data later than that to a side output.

WHAT IT TESTS: whether you understand event time, watermarks, and the accuracy-latency tradeoff in streaming. ANSWER OUTLINE: aggregate on event time, not processing time, so windows reflect when events actually occurred; use watermarks to estimate progress and tolerate a configured amount of lateness, keeping window state open during a grace period so late events update results; finalize and emit when the watermark passes, and route events later than the allowed lateness to a side output or dead-letter sink rather than silently dropping.

Read the original → interview

Get five bites like this every day.

Tezvyn delivers a daily feed of 60-second tech bites with quizzes to lock in what you learn.

Handling late data in streaming windows · Tezvyn