Cards/Data PlatformSystem Design · Day 86Oct 26, 2026

The Event That Arrives After We Closed The Books

A late event stamped 10:00:42 arriving after the 10:00 total was published, with event time and processing time shown as two clocks, processing-time windows putting the same event in different buckets, and a watermark closing the window

We count orders per minute. At 10:01 we publish the total for 10:00 and move on. Then an event stamped 10:00:42 arrives. A phone was offline, a broker retried, a consumer lagged. The minute we already reported was wrong, and we didn't know it.

Streaming has two clocks. Event time is when the thing happened. Processing time is when we saw it. Windowing on processing time is easy to build and quietly wrong — the same event lands in a different bucket depending on the network conditions that day.

A watermark is the system's working belief about how far event time has progressed: we think we have now seen everything up to 10:00:45. The window closes and emits when the watermark passes its end.

Where that belief is set is the whole decision. A short watermark gives fast results and lets more events arrive too late to count. A long one is more accurate and delays everything downstream. Allowed lateness holds the window open a while longer and emits an update when the stragglers turn up.

Completeness and timeliness can't both be maximised. We're choosing how long to wait before being willing to be wrong.

Completeness and timeliness can't both be maximised. We're choosing how long to wait before being willing to be wrong.