Design a real-time personalized notification trigger system
Event-driven scheduling with backpressure.
Stream events to a delayed queue, expose a rule UI to non-technical users, and deliver idempotently.
Cron polling a database without ordering or rate limits.
What's really being asked
This question tests whether you understand the difference between fire-and-forget messaging and time-bound scheduled delivery at scale. The interviewer wants to see that you can decouple event ingestion from execution, handle backpressure, and design for operability by non-technical teams. They also care about data consistency, exactly-once semantics, and graceful degradation under load.
The full answer
First, an event ingestion layer such as Kafka or Kinesis that captures user actions with ordering guarantees per user. Second, a distributed delay queue or scheduler like RabbitMQ delayed messages, SQS with visibility timeouts, or a custom time-wheel service that can hold millions of triggers without polling. Third, a rule engine and UI that lets marketers define triggers, audiences, and templates without deploying code, ideally compiling rules into a lightweight DSL or config evaluated by workers. Fourth, idempotent delivery workers that fetch due jobs, resolve user data, render personalized content, and send via push providers while respecting rate limits and opt-outs. Fifth, observability including per-campaign latency metrics, dead-letter queues for failed sends, and circuit breakers for downstream provider outages.
The mistakes people make
The biggest red flag is proposing a cron job that polls a relational database for rows where scheduled_time is less than now. This creates a hot shard on the time index, misses ordering guarantees, and falls over under high cardinality. Another mistake is skipping idempotency, which leads to duplicate notifications if the worker retries. A third error is ignoring the non-technical requirement by suggesting every rule change requires an engineer to modify code and redeploy.
What usually comes next
The interviewer may ask how you handle a user who returns and completes the action before the 30-minute timer fires. You should answer with a cancellation mechanism, either by removing the scheduled job or by checking state at delivery time. They may also ask about fairness between users or campaigns, which leads to token buckets or per-user rate limiters. Another follow-up is global scale and timezone handling, which requires scheduling in UTC and localizing at render time.
A concrete example
Imagine a cart abandonment flow. When a user adds an item, the checkout service emits a CartUpdated event. A stream processor writes a scheduled notification record into a delay queue with a 30-minute TTL. If the user checks out within that window, a CheckoutCompleted event triggers a cancellation request that removes the job from the queue. After 30 minutes, a worker pulls the job, verifies the cart is still abandoned, fetches the user's preferences, renders a localized push, and sends it through Firebase or APNS. If the send fails, the worker retries with exponential backoff up to a limit, then moves the job to a dead-letter queue for manual inspection.
Interview question
To support millions of scheduled personalized notifications without overwhelming downstream providers, which architectural pattern should you prefer?
- a.Stream events into a delay queue and process due jobs with idempotent, rate-limited workersCorrect
- b.Push events immediately to Firebase and APNS with client-side retries
- c.Buffer all jobs in a single Redis instance and pop them with a cron script
- d.Poll a relational database every minute for due rows and send them in a loop
Why? this is the answer
The card recommends decoupling ingestion from execution via an event stream and distributed delay queue, using idempotent workers that respect rate limits. Option D is tempting because database polling is a common pattern, but the card explicitly flags it as a red flag that creates hot shards and lacks ordering guarantees.
Just read this? Test yourself on what you have been reading.
Read the original → cloud.google.com
- #system design
- #event-driven
- #notifications
- #growth engineering
- #scalability
You just looked this up. Could you explain it out loud?
That is the part interviews actually test. Tezvyn takes questions like this one and gives you what the interviewer is really checking, the answer that lands, and the mistake that ends the conversation, in the four minutes before your next meeting.
The iPhone app is on the way
We are building it. Until it lands, nothing here is held back from you: every interview card, your saved cards, streaks and the job board all work in Safari, plus hundreds of free practice quizzes of thirty questions each. Sign in and it all carries over to the app the day it arrives.
Want it as an icon? Tap Share at the bottom of Safari, then Add to Home Screen. It opens full screen and the cards you have read stay available offline.
We are hiring for this. Open roles that interview on system design — each one lists the topics its interview covers.
See open roles