Detect a three-action user sequence in real-time at scale

WHAT THIS TESTS: This question evaluates your ability to design a distributed system for stateful pattern matching over high-volume, unbounded event streams with strict latency requirements. The interviewer wants to see that you understand complex event processing concepts, event-time semantics, keyed state management, and the operational trade-offs between correctness, cost, and speed at scale.

A GOOD ANSWER COVERS: First, ingestion and serialization: use a distributed log like Kafka or Kinesis to collect action events with user-id keys to preserve ordering. Second, stream processing layer: deploy a CEP-capable engine such as Apache Flink, Kafka Streams, or Siddhi to evaluate the three-action sequence. Model the pattern as a finite state machine or use a built-in CEP operator with a strict event-time window, for example thirty to sixty seconds, to bound state. Third, state storage: maintain per-user partial matches in a local keyed state store backed by a changelog or remote KV store like RocksDB to survive restarts. Fourth, delivery: push the detected pattern to the user device immediately via WebSocket, SSE, or a mobile push gateway rather than relying on polling. Fifth, trade-offs: call out that event-time windows and exactly-once semantics add latency and memory overhead; large windows increase state size and GC pressure; out-of-order events require watermarking and may delay triggers; and shared devices or session resets can cause false positives.

COMMON WRONG ANSWERS: Proposing to run a periodic SQL query over a data warehouse or OLAP store is a critical red flag because batch latency is measured in minutes, not milliseconds. Suggesting a simple application-level cache without addressing failover, backpressure, or event reordering reveals inexperience with stream processing. Ignoring the distinction between processing time and event time leads to incorrect matches when events arrive out of order. Overlooking delivery semantics, such as assuming the client polls an API every few seconds, fails the low-latency requirement.

LIKELY FOLLOW-UPS: How would you handle actions that cross a session boundary or a device switch? What happens if the second action arrives five minutes late? How do you scale the state store when per-user state grows? How do you prevent duplicate tutorial triggers under exactly-once requirements? What is your recovery plan if the stream processor restarts mid-sequence?

ONE CONCRETE EXAMPLE: Imagine a ride-sharing app where the sequence is open map, drop a pin, and tap schedule. Events are keyed by rider-id and flow into a Flink job. The job uses a CEP pattern with a forty-second event-time window and a watermark of five seconds. Partial matches are stored in Flink's keyed state backed by RocksDB with incremental checkpoints to S3. When the third action matches, the job writes a trigger record to a Redis pub-sub channel that the mobile gateway forwards over an open WebSocket. The trade-off is that riders on flaky networks may trigger watermarks late, delaying the tutorial by a few seconds, but the system avoids false positives from out-of-order events.